[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bacula-devel] Copy Jobs Selection


Hello,

For those of you using the new Copy feature to make copies of backups, here is 
some information that may help:

>From looking at the code, Bacula should be able to select disk or tape quite 
easily if you specify a "pool=xxx" on your restore command line (in bat, one 
simply selects from a drop down box).  In that case, Bacula will restrict 
finding JobIds to restore (for a current or time based restore) to those Jobs 
that wrote to the specified pool.

I haven't actually tested this, but it *should* allow you to relatively easily 
select which Copy you want to use for restore.

Another option is to simply specify the JobIds manually (not very good if you 
want to select the most current backup, which may be a number of jobids).

Although you can specify Storage=xxx on the restore command line, it does not 
seem to be used in the selection of the JobIds, so specifying it may or may 
not be useful.  Before the final release, I will look into allowing a Storage 
specification to select which backup is used as well.  However, using pool is 
probably a better choice.

Finally, I am considering implementing an option equivalent to the SQL command 
that Ulrich is using so that you can "automatically" Copy only those jobs 
that have not already been copied.  I won't guarantee that this will be 
implemented before the next release, but it is clearly needed.

Best regards,

Kern

On Wednesday 21 May 2008 17:44:44 Ulrich Leodolter wrote:
> Hello,
>
> 1) Restore Report
> =================
>
> Today my bacula test installation has one full and one incremental
> backup on disk, and one full copy on tape for each client.
>
> Here is the restore log for my /home on a linux (debian lenny) machine.
> i checked the restore files using rsync, and it looks ok.
>
> I started the restore job using bconsole.   bacula selects full backups
> only from disk, which is ok for this situation.
> but i am unsure if this selection is made by programm logic or simply
> by database storage order of jobs.
>
> there is an other question: what happens after disk volumes are recycled
> or purged,   i can test this maybe on friday.
>
> first impression: restore works but unsure why :-)
>
>
> 21-May 16:16 troll-dir JobId 32: Start Restore Job
> RestoreFiles.2008-05-21_16.16.07
> 21-May 16:16 troll-dir JobId 32: Using Device "FileStorage"
> 21-May 16:16 troll-sd JobId 32: Ready to read from volume "Backup-0004"
> on device "FileStorage" (/data/bacula/files).
> 21-May 16:16 troll-sd JobId 32: Forward spacing Volume "Backup-0004" to
> file:block 0:1154434920.
> 21-May 16:16 troll-sd JobId 32: End of Volume at file 0 on device
> "FileStorage" (/data/bacula/files), Volume "Backup-0004"
> 21-May 16:16 troll-sd JobId 32: Ready to read from volume "Backup-0005"
> on device "FileStorage" (/data/bacula/files).
> 21-May 16:16 troll-sd JobId 32: Forward spacing Volume "Backup-0005" to
> file:block 0:64716.
> 21-May 16:17 troll-sd JobId 32: End of Volume at file 0 on device
> "FileStorage" (/data/bacula/files), Volume "Backup-0005"
> 21-May 16:17 troll-sd JobId 32: Ready to read from volume "Backup-0028"
> on device "FileStorage" (/data/bacula/files).
> 21-May 16:17 troll-sd JobId 32: Forward spacing Volume "Backup-0028" to
> file:block 0:173046766.
> 21-May 16:18 troll-sd JobId 32: End of Volume at file 0 on device
> "FileStorage" (/data/bacula/files), Volume "Backup-0028"
> 21-May 16:18 troll-sd JobId 32: End of all volumes.
> 21-May 16:18 troll-dir JobId 32: Bacula troll-dir 2.3.21 (12May08):
> 21-May-2008 16:18:01
>   Build OS:               i686-pc-linux-gnu redhat Enterprise release
>   JobId:                  32
>   Job:                    RestoreFiles.2008-05-21_16.16.07
>   Restore Client:         leodolter-fd
>   Start time:             21-May-2008 16:16:31
>   End time:               21-May-2008 16:18:01
>   Files Expected:         84,989
>   Files Restored:         84,991
>   Bytes Restored:         2,197,244,991
>   Rate:                   24413.8 KB/s
>   FD Errors:              0
>   FD termination status:  OK
>   SD termination status:  OK
>   Termination:            Restore OK
>
> 21-May 16:18 troll-dir JobId 32: Begin pruning Jobs.
> 21-May 16:18 troll-dir JobId 32: No Jobs found to prune.
> 21-May 16:18 troll-dir JobId 32: Begin pruning Files.
> 21-May 16:18 troll-dir JobId 32: No Files found to prune.
> 21-May 16:18 troll-dir JobId 32: End auto prune.
>
>
>
> 2) Copy Report
> ==============
>
> After the restore i started the CopyDiskToTape job again using
> the job definition below.  i used sql to selected all
> successful backup jobs which are not referenced in another
> successful backup job.  this does exactly what i want
> and runs copy to tape for each incremental backup.
>
> Job {
>   Name = "CopyDiskToTape"
>   Type = Copy
>   Client = troll-fd
>   Level = Full                  # must be defined, but is ignored
>   FileSet = "Full Set"          # must be defined, but is ignored
>   Pool = DiskBackup
>   Storage = File
>   Messages = Standard
>   Selection Type = SQLQuery
>   Selection Pattern = "SELECT Job.JobId FROM Job,Pool WHERE Job.Type =
> 'B' and Job.JobStatus = 'T' and Pool.Name = 'DiskBackup' and Job.PoolId
> = Pool.PoolId and PriorJobId = 0 and Job.JobId not in ( select
> PriorJobId from Job where Type = 'B' and Job.JobStatus = 'T' and
> PriorJobId != 0 )"
>   Maximum Concurrent Jobs = 10
>   SpoolData = yes
> }
>
> On Tue, 2008-05-20 at 19:00 +0200, Kern Sibbald wrote:
> > Hello,
> >
> > I am very pleased to see that someone is trying out this feature, and
> > even more surprised that it works, because although I implemented the
> > code to do the Copy, I have never had the time to run it.
> >
> > One thing that you should test VERY carefully is restores after a Copy. 
> > I do not know exactly what Bacula is going to do when it finds multiple
> > jobs that started at *exactly* the same time.  It could very likely get
> > confused.  My intention was to test this later and to try to find some
> > solution that allows the user to make some intelligent choice of what is
> > used for the restore. Please do not underestimate the potential problems
> > with restore and this new feature.
> >
> > Concerning your question about how to Copy only those jobs that have not
> > already been copied:  well, that is an extremely good question, and now
> > that you mention it, this is clearly something that users are going to
> > want to do, so Bacula *should* have some easy way to do it.  I need to do
> > a bit of research over the next couple of days to see what the best way
> > to do it is -- it is unfortunately not a question I can answer without
> > careful examination of the code (in particular the entries in the SQL
> > database).  I'll also see if I can look into the problems of restores as
> > well.
> >
> > If you haven't heard back from my by Friday of this week, it means that I
> > have gotten distracted (too many things going on with Bacula these days),
> > so please in that case send me a reminder email.
> >
> > Please let me know what sort of results you get with doing restores.
> >
> > Best regards,
> >
> > Kern
> >
> > On Tuesday 20 May 2008 16:52:14 Ulrich Leodolter wrote:
> > > hello,
> > >
> > > yesterday i dropped bacula 2.2.8 test installation
> > > and compiled/installed the current bacula version from svn
> > > (Revision: 6995) ... because i like to test "Copy Jobs".
> > >
> > > i also simplified the config while migrating 5 test clients
> > > and defined two pools
> > >
> > > Pool {
> > >   Name = DiskBackup
> > >   Pool Type = Backup
> > >   Recycle = yes
> > >   RecyclePool = DiskBackup
> > >   AutoPrune = yes
> > >   Volume Retention = 30 days
> > >   Maximum Volume Bytes = 2G
> > >   Label Format = Backup-
> > >   Next Pool = DiskCopy
> > > }
> > >
> > > Pool {
> > >   Name = DiskCopy
> > >   Pool Type = Copy
> > >   Storage = Tape
> > >   Copy Pool = DiskBackup
> > >   Recycle = yes
> > >   AutoPrune = yes
> > >   Volume Retention = 6 months
> > > # Label Format = BAC
> > > }
> > >
> > > tonight full backups to DiskBackup pool completed without problems.
> > > today i've manually started a CopyDiskToTape job
> > >
> > > Job {
> > >   Name = "CopyDiskToTape"
> > >   Type = Copy
> > >   Client = troll-fd
> > >   Level = Full                  # must be defined, but is ignored
> > >   FileSet = "Full Set"          # must be defined, but is ignored
> > >   Pool = DiskBackup
> > >   Storage = File
> > >   Messages = Standard
> > >   Selection Type = PoolTime
> > >   Maximum Concurrent Jobs = 10
> > >   SpoolData = yes
> > > }
> > >
> > > copy jobs run fine and all full DiskBackup are copied to pool
> > > DiskCopy (Tape).
> > > as you can see "Selection Type = PoolTime",  but i did not specify
> > > "Migration Time = ..." in DiskBackup.
> > >
> > > i would like to run the "CopyDiskToTape" job every morning after
> > > nightly DiskBackup jobs completed.
> > >
> > > how can i simply select only "new DiskBackup" jobs which have no Copy
> > > ??
> > >
> > > do i need "Selection Type = SQLQuery" where sql selects
> > > jobs which have StartTime > LastCopyDiskToTape.StarTime ??
> > >
> > >
> > >
> > > Thanks



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Bacula-devel mailing list
Bacula-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/bacula-devel


This mailing list archive is a service of Copilotco.