[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bacula-devel] Accurate Incremental backup problem


Hello,

ok, i've started bacula-fd in debug mode

leodolter:~# nohup /opt/bacula/sbin/bacula-fd -f -d 200 -u root -g bacula \
   -c /opt/bacula/etc/bacula-fd.conf >& /tmp/bacula-fd.log

then i've started 2 consecutive incremental backups, and again looked at /usr/bin/wget,
(i can send u 350MB debug file if u like)

leodolter:~# fgrep '/usr/bin/wget' /tmp/bacula-fd.log 
leodolter-fd: htable.c:288-3680 Insert hp->next=0 hp->hash=0x59616a69 hp->key=/usr/bin/wget
leodolter-fd: htable.c:294-3680 Leave insert index=25223 num_items=50014 key=/usr/bin/wget
leodolter-fd: backup.c:1210-3680 No strip for /usr/bin/wget
leodolter-fd: accurate.c:182-3680 /usr/bin/wget      st_mtime differs
leodolter-fd: backup.c:307-3680 FT_REG saving: /usr/bin/wget
leodolter-fd: backup.c:402-3680 bfiled: sending /usr/bin/wget to stored
leodolter-fd: backup.c:1210-3680 No strip for /usr/bin/wget
leodolter-fd: bfile.c:835-3680 open file /usr/bin/wget
leodolter-fd: htable.c:288-3681 Insert hp->next=0 hp->hash=0x59616a69 hp->key=/usr/bin/wget
leodolter-fd: htable.c:294-3681 Leave insert index=25223 num_items=50014 key=/usr/bin/wget
leodolter-fd: backup.c:1210-3681 No strip for /usr/bin/wget
leodolter-fd: accurate.c:182-3681 /usr/bin/wget      st_mtime differs
leodolter-fd: backup.c:307-3681 FT_REG saving: /usr/bin/wget
leodolter-fd: backup.c:402-3681 bfiled: sending /usr/bin/wget to stored
leodolter-fd: backup.c:1210-3681 No strip for /usr/bin/wget
leodolter-fd: bfile.c:835-3681 open file /usr/bin/wget

as u see , st_mtime is the reason.
but i am very sure that st_mtime did not change between incremental backups!
verified that st_time did not change using a simple perl script
called it before and after incremental jobs.

so please tell me which st_mtime are compared?
(maybe there is a hash code bug in accurrate.c,  just thinking
because i dont understand)

ulrich@leodolter:~$ bin/ul_bacula_lstat.pl /usr/bin/wget 
dev      =      65024 P4A
ino      =    1851549 HECd
mode     =      33261 IHt
nlink    =          1 B
uid      =          0 A
gid      =          0 A
rdev     =          0 A
size     =     226260 3PU
atime    = 1222265400 BI2ko4
mtime    = 1220829421 BIxGDt
ctime    = 1222070763 BI11Hr
blksize  =       4096 BAA
blocks   =        456 HI
sha1     = FyVUN2j34bKjITbKF5khOovbiGs


mysql> select File.JobId, Filename.Name,Path.Path,File.LStat,File.MD5 from File
   join Filename using (FilenameId)
   join Path using (PathId) where File.JobId in (3574,3605,3637,3669,3680,3681) and Filename.Name = 'wget';
+-------+------+-----------+------------------------------------------------------------+-----------------------------+
| JobId | Name | Path      | LStat                                                      | MD5                         |
+-------+------+-----------+------------------------------------------------------------+-----------------------------+
|  3574 | wget | /usr/bin/ | P4A HEEh IHt B A A A 3PU BAA HI BIz28l BIaVNu BIfhPi A A E | 8dz2rHajxE25WMWnQ2U6DUja/MI | 
|  3637 | wget | /usr/bin/ | P4A HECd IHt B A A A 3PU BAA HI BI2CXZ BIxGDt BI11Hr A A E | FyVUN2j34bKjITbKF5khOovbiGs | 
|  3669 | wget | /usr/bin/ | P4A HECd IHt B A A A 3PU BAA HI BI2DL7 BIxGDt BI11Hr A A E | FyVUN2j34bKjITbKF5khOovbiGs | 
|  3680 | wget | /usr/bin/ | P4A HECd IHt B A A A 3PU BAA HI BI2kMY BIxGDt BI11Hr A A E | FyVUN2j34bKjITbKF5khOovbiGs | 
|  3681 | wget | /usr/bin/ | P4A HECd IHt B A A A 3PU BAA HI BI2kWI BIxGDt BI11Hr A A E | FyVUN2j34bKjITbKF5khOovbiGs | 
+-------+------+-----------+------------------------------------------------------------+-----------------------------+


Please have a closer look at this
Ulrich


On Wed, 2008-09-24 at 14:01 +0200, Eric Bollengier wrote:
> Hello,
> 
> 
> On Wednesday 24 September 2008 13:34:53 Kern Sibbald wrote:
> > Hello,
> >
> > On Wednesday 24 September 2008 11:52:41 Ulrich Leodolter wrote:
> > > Hi,
> > >
> > > I looks like Accurate Incremental backups work not like expected.
> > >
> > > I am using current bacula trunk, last update 2008-09-16.
> > > No config changes since last update.
> > >
> > > Below is a Job list where i tried to track down the problem.
> > > On Monday Sep 22 i did a system update (apt-get), so 20794 Files are ok
> > > for Job 3637. But i did not make major changes on Sep 23, so 21725 Files
> > > for Job 3669 are not expected!
> > >
> > >
> > > JobId JobName Status Level Client Pool StartTime EndTime Duration Files
> > > Bytes 3574 Backup-leodolter OK F leodolter-fd DiskBackup 08-Sep-21 02:05
> > > 08-Sep-21 02:26 00:21:45 178,917   4.5 GB 3605 Backup-leodolter OK I
> > > leodolter-fd DiskBackup 08-Sep-22 02:05 08-Sep-22 02:06 00:01:27     202
> > > 13.3 MB 3637 Backup-leodolter OK I leodolter-fd DiskBackup 08-Sep-23
> > > 02:05 08-Sep-23 02:07 00:02:34  20,794 187.6 MB 3669 Backup-leodolter OK
> > > I leodolter-fd DiskBackup 08-Sep-24 02:05 08-Sep-24 02:07 00:02:42 
> > > 21,725 192.6 MB
> > >
> > >
> > > Now i looked at some special file /usr/bin/wget (see mysql select below)
> > > As far as i can see only st_atime (11'th field in LStat) changed from 23
> > > Sep (JobId 3637) to 24 Sep (JobId 3669).
> > >
> > > Until now i thought st_atime is not used for Accurate backups.
> >
> > To the best of my knowledge, st_atime is not used by Bacula to make any
> > decision whether or not to backup the file.  The tests are strictly against
> > st_mtime and st_ctime (depending on how you configure Bacula).
> >
> > It sounds to me like you have a problem of an anti-virus program that
> > is "touching" some files or some other change to your system happened.
> > Running verify InitCatalog, then later Catalog can verify this ...
> >
> > > For me i looks like Accurate Incremental backups are always made against
> > > last Full while ignoring previous Accurate Incremental jobs (maybe i am
> > > completely wrong)
> >
> > I haven't taken a careful look at all the details you presented above, but
> > Accurate works very much like existing backups -- that is it looks at
> > st_mtime and st_ctime (if I am not mistaken), and if the file needs to be
> > backed up for the "normal" backup reasons, it will be.  What Accurate does
> > is in *addition* to the "normal" rules, it will check to see if a file has
> > been deleted or has been added with an old date, and in those cases, it
> > ensures that files that are deleted are so marked, and any file that is
> > missed by the normal backup will also be backed up.
> >
> > I'll let Eric confirm this.
> 
> I confirm :)
> 
> Ulrich, you can try to run the the FD at debuglevel 200, and you will see why
> the file has been backuped.
> 
> Bye
> 
> > If it is not working like I described above, then please let me know and we
> > can look at it in more details.
> >
> > Best regards,
> >
> > Kern
> >
> > > mysql> select File.JobId, Filename.Name,Path.Path,File.LStat,File.MD5
> > > from File join Filename using (FilenameId)
> > >   join Path using (PathId)
> > >  where File.JobId in (3574,3605,3637,3669) and Filename.Name = 'wget';
> > >
> > > +-------+------+-----------+---------------------------------------------
> > >-- -------------+-----------------------------+
> > >
> > > | JobId | Name | Path      | LStat
> > > |
> > > |             | MD5                         |
> > >
> > > +-------+------+-----------+---------------------------------------------
> > >-- -------------+-----------------------------+
> > >
> > > |  3574 | wget | /usr/bin/ | P4A HEEh IHt B A A A 3PU BAA HI BIz28l
> > > | BIaVNu BIfhPi A A E | 8dz2rHajxE25WMWnQ2U6DUja/MI | 3637 | wget |
> > > | /usr/bin/ | P4A HECd IHt B A A A 3PU BAA HI BI2CXZ BIxGDt BI11Hr A A E
> > > | |
> > > | FyVUN2j34bKjITbKF5khOovbiGs | 3669 | wget | /usr/bin/ | P4A HECd IHt B
> > > | A A A 3PU BAA HI BI2DL7 BIxGDt BI11Hr A A E |
> > > | FyVUN2j34bKjITbKF5khOovbiGs |
> > >
> > > +-------+------+-----------+---------------------------------------------
> > >-- -------------+-----------------------------+ 3 rows in set (0.00 sec)
> > >
> > >
> > > Please have a look at this
> > > Thanks
> 
> 
> 
-- 
Ulrich Leodolter <ulrich.leodolter@xxxxxxxx>
Oesterreichische Bibliothekenverbund und Service GmbH
Bruennlbadgasse 17/2A, A-1090 Wien
Fax +43 1 4035158-30
Tel +43 1 4035158-21
Web http://www.obvsg.at

Attachment: ul_bacula_lstat.pl
Description: Perl program

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-devel mailing list
Bacula-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/bacula-devel


This mailing list archive is a service of Copilot Consulting.