[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bacula-devel] MaximumBlockSize Problem and Question


On Thu, 2008-12-04 at 20:49 +0200, Pasi Kärkkäinen wrote:
> On Thu, Dec 04, 2008 at 07:39:44PM +0100, Ulrich Leodolter wrote:
> > 
> > On Thu, 2008-12-04 at 20:03 +0200, Pasi Kärkkäinen wrote:
> > > On Wed, Nov 05, 2008 at 02:48:20PM +0100, Ulrich Leodolter wrote:
> > > > On Wed, 2008-11-05 at 13:34 +0200, Pasi Kärkkäinen wrote:
> > > > > On Wed, Nov 05, 2008 at 11:12:01AM +0100, Ulrich Leodolter wrote:
> > > > > > Hello,
> > > > > > 
> > > > > > Problem:
> > > > > > 
> > > > > > Settings like (documented in the Manual)
> > > > > > 
> > > > > >      Minimum block size = 64K
> > > > > >      Maximum block size = 200K
> > > > > > 
> > > > > > do not work. Both are defined as size_pint32.
> > > > > > multipliers like K M are not allowed.
> > > > > > don't know if this is intended, 
> > > > > > maybe documentation just a little outdated.
> > > > > > 
> > > > > > 
> > > > > > Question:
> > > > > > 
> > > > > > Can i expect performance improvements (especially for
> > > > > > Copy/Migrate Disk to Tape jobs) by by increasing
> > > > > > "Maximum block size" on File devices in bacula-sd.conf ???
> > > > > > 
> > > > > > Device {
> > > > > >   Name = FileStorage
> > > > > >   Media Type = File
> > > > > >   Archive Device = /disk0/bacula/files
> > > > > >   LabelMedia = yes;  
> > > > > >   Random Access = Yes;
> > > > > >   Requires Mount = No;
> > > > > >   AutomaticMount = yes;
> > > > > >   RemovableMedia = no;
> > > > > >   AlwaysOpen = no;
> > > > > >   MaximumBlockSize = 1048576;         # 1M
> > > > > > }
> > > > > > 
> > > > > 
> > > > > I think it should make difference at least with tape drives.. 
> > > > > 
> > > > > You can measure the difference with dd:
> > > > > 
> > > > > dd if=/dev/null of=/dev/tape bs=<try_different_block_sizes_here>
> > > > > 
> > > > > 4k, 64k, 256k, 1MB, 4MB etc.
> > > > 
> > > > Hello,
> > > > 
> > > > I dont think raw performance is bad, as u can see below
> > > > (exeption is 4k block size on tape,
> > > >  hopfully bacula does not use it :-)
> > > > 
> > > > Maybe i should set
> > > > 
> > > > 	Minimum Block Size = 4194304
> > > > 
> > > > on both devices (Tape, File)
> > > > This should reduce the number of block read/writes communication
> > > > by a factor of 64.
> > > > 
> > > > Will try it tonight,  unless u give me a very good hint how to
> > > > optimize ;-)
> > > > 
> > > 
> > > Hi!
> > > 
> > > Did you figure out 'best' performance settings for these disk-to-tape copy jobs? 
> > > 
> > 
> > Hi,
> > 
> > still using default tape block size 63k
> > 
> 
> Ok. So you don't have any min/max block sizes defined in Bacula config? 

No,   now think its better to stay at default values,
otherwise you may get in trouble when it comes to restore.


> 
> > disk volumes are limited to 4G
> > 
> > concurrent disk backup jobs use spooling, now full backup jobs
> > are spread almost continuous over 4G volumes
> > 
> > copy disk to tape runs without spooling at rates up to 50000 kbytes/s
> > 
> 
> Ok. That's not bad. 
> 
> > 
> > this is acceptable, but emc networker does the same job (copy disk
> > backup jobs to tape) at rates up to 75000 kbytes/s
> > 
> 
> So still some room for improvement.. 
> 
> > 
> > but there is still an open bug which affects performance
> > http://bugs.bacula.org/view.php?id=1190
> > especially when concurrent disk backup dont use spooling.
> > 
> 
> Ok. Thanks.
> 
> And I guess "multiple threading multiple buffering" would help aswell:
> http://www.mail-archive.com/bacula-devel@xxxxxxxxxxxxxxxxxxxxx/msg01246.html
> 

right,  would be nice in bcopy too,
i'am using bcopy to make offsite clones (copy tape to tape)
it runs 5 hours to copy LTO4 tape.

emc networker does the same job (using 2 processes read/writer) in
3:20 hours

i tried also some raw tape copy programs (tcopy, mag) but did not
find a fast (multiprocess/threaded/bufferd) reliable tool.


i tried also bcopy through fifo storage device:

bcopy -c bacula-sd.conf /dev/nst0 /tmp/fifo & 
bcopy -c bacula-sd.conf /dev/fifo /dev/nst1 

but didnt work, needs some special fifo device handling in bcopy.


Regards
Ulrich

PS: do u make offsite clones?




-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-devel mailing list
Bacula-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/bacula-devel


This mailing list archive is a service of Copilotco.