[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bacula-devel] Using the OS to get the list of files changed


Le Sunday 12 October 2008 17:21:12 Dan Langille, vous avez écrit :
> With incremental and differential backups, the goal is a list of files
> that have changed since a given point in time.   Traditionally, this
> list is obtained by using find (AFAIK).  On filesystems with large
> numbers of files,the list creation can take considerable time.  How can
> we improve the performance of this stage?
>
> For some time, I have wanted the filesystem (or OS) to give me a list of
> files changed since time X.  I've just heard Pawel Jakub Dawidek speak
> at NYCBSDCon about ZFS.  I asked him if he thought getting such a list
> was possible.  He suggested it may be.  Strategy: ZFS is able to return
> a file name given a particular block.  That is, if you know a particular
> block has changed, ZFS will tell you what file it belongs to.
>
> When using ZFS snapshots, the system keeps a list of blocks that have
> changed since the last snapshot.  It may be feasible to traverse this
> list of block to obtain the list of files required by Bacula.
>
> My thoughts:
>
> 1 - I'm not proposing to apply this strategy to Bacula internally.
> However, an external script would be very useful.  Bacula already
> provides a mechanism for a FileSet to be specified by an external
> script.  I'm thinking: combine the two approaches to provide faster
> incremental/differential backups.
>
> 2 - This strategy may be applicable to other filesystems that provide
> similar mechanisms (list of blocks changed, getting file names for those
> blocks).  It was suggested during the talk that auditing, etc may
> provide the needed hooks.
>
> I hope this triggers some brain storms from people.

On linux (and perhaps freebsd and solaris), you can use either STAP or Inotify 
interface (with 10 lines of perl or stap script) to get the list of files 
that have been modified between 2 backups. 

http://en.wikipedia.org/wiki/Systemtap
http://en.wikipedia.org/wiki/Inotify

And important, you can use whatever you want as filesystem backend and you 
don't need to convert a block address into a filename :-)

Bye

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-devel mailing list
Bacula-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/bacula-devel


This mailing list archive is a service of Copilotco.