[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bacula-devel] [Bacula-users] Programming bounty

Hello Alan,

As you can see, I have copied the lists so that everyone will get the benefit 
of this information if they want it ...

On Wednesday 18 June 2008 14:40:02 Alan Brown wrote:
> On Wed, 18 Jun 2008, Kern Sibbald wrote:
> > The two projects (Synthetic and Accurate) are quite distinct, and no
> > snapshot is needed on Linux systems, while it is normally done on Win32
> > systems.
> I don't mean snapshot in the LVM sense, but in terms of recording the
> filesystem structure at a given point in time, in order to avoid a couple
> of irritating but major issues.
> What has always been a problem for us are the following 2 issues with
> current operation.
> 1: Files deleted after a Full backup will be restored. This causes user
> confusion about stuff they throught they'd thrown out or moved elsewhere
> suddenly reappearing.
> 2: Directory trees moved around within a filesystem after a Full backup
> have a tendency to partially reappear in their old location and partially
> in the new one, depending how many files have been written since the Full
> backup.
> This causes similar user confusion. I make a point of warnng them about
> this kind of issue whenever I'm asked to restore more than a few files.
> It can be a _major_ issue;
>  - as one example I've just forced a few (silly) users to implement
> hierarchical directory structures after finding that a couple of
> filesystems were grinding to a halt due to directories with somewhere
> around 1 million files in a directory. Obviously I don't want that
> original directory restored.
>  - another example, after creating those hierarchical directory
> structures, the users then moved the trees around. Again I don't want a
> few hundred thosand files appearing in one part of the restored filesystem
> and a few hundred thousand appearig in another (or worse, restoring more
> files than there is room in the filesystem...)

All the above problems are resolved by doing Accurate backups in version 2.5.x 
and later (current development tree).  This code is in and working, though as 
with all new code, we need additional beta testing.

> > For the synthetic full, a better name IMO is consolidation -- it simply
> > consolidates a bunch of previous backup jobs into a new single synthetic
> > backup  job that appears to be a Full backup.
> You're probably right, but the consolidated backup should not include
> files which have been deleted since the last Full backup (consolidated or
> real) was made.

A consolidation job will consolidate all files that were previously backed up.  
If the backups were "normal" (2.4.x and below), the deleted files will be 
included, because a "normal" backup does not recognize deleted files.  If the 
backups were "accurate" (if enabled in 2.5.x and above and using 2.5 
clients), then the deleted files will be correctly deleted (actually not 
copied during the consolidation).  The consolidation does not use the client, 
so it cannot turn a "normal" backup into an "accurate" backup, but if the 
original backups were all "accurate" then all the information is contained in 
the backups and when files are deleted, an entry is put in the catalog so 
that they will not re-appear.

> > It is *very* close to a
> > Migration (even closer to a Copy) job, but with a whole lot less options.
> Migration can be tricky. I've made it work between pools, but our main
> requirement tends to be to remove jobs within a pool (eg, copy existing
> jobs off archive tapes onto new media within the same pool)

A Consolidation job will be very similar to a Copy job (Migration without 
delete) in the way that it requires two pools.   I don't know how to make 
Migration, Copy, Consolidation, or Archive (when we finally implement 
archiving) work with a single pool, so for the near term they must all 
require two pools.

By the way, I see that some vendors are referring to Consolidation jobs as 
virtual full backups rather than synthetic full backups  :-)



> Regards
> Alan

Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
Bacula-devel mailing list

This mailing list archive is a service of Copilotco.