[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bacula-devel] MaximumBlockSize Problem and Question

On Wed, 2008-11-05 at 18:11 +0100, Kjetil Torgrim Homme wrote:
> Arno Lehmann <al@xxxxxxxxxxxxxx> writes:
> > And I strongly recommend to not measure throughput using
> > /dev/zero...  if you use compression, you're almost only measuring
> > bus throughput as all the zeros will be compressed away... it's
> > better to prepare a big file with random data for it. Unfortunately,
> > that brings the disk system into the environment again. A big
> > RAM-disk can help, but you'd need lots of RAM for this to be useful.
> you can make a suitably large file from /dev/urandom and cat it
> endlessly into dd
>    dd if=/dev/urandom of=/tmp/random-data bs=1M count=16
>    cat /tmp/random-data{,,,,,,,,,,,,,,,} |
>        dd of=/dev/nst0 bs=256k
> (the {,,,} syntax requires csh, bash or zsh.  use a for loop or
> similar in POSIX sh.)


here are my results

[root@troll tmp]# cat /disk0/tmp/random-data{,,,,,,,,,,,,,,,} | dd
of=/dev/nst1 bs=256k
1006+28 records in
1006+28 records out

[root@troll tmp]# cat /disk0/tmp/random-data{,,,,,,,,,,,,,,,} | dd
of=/dev/nst1 bs=256k
64854+1048 records in
64854+1048 records out
17179869184 bytes (17 GB) copied, 162.675 seconds, 106 MB/s

As u can see performance is good,  and it would be ok if bacula
writes at  50 MB/s.

But bacula writes only at  5-15 MB/s for Copy/Migrate Disk to Tape
jobs.   A job size like this (17 GB) is split into 282949 63K buffers.
There must be performance bottleneck in bacula's buffer handling.
Maybe because this runs inside one storage daemon process.
Does bacula use multithreaded parallel read/write for jobs like this?
Using top i can see only one active process (bacula-sd)

EMC Legato Networker (on the same hardware) is able to Clone
a set of Full Backup Jobs (800 GB) from Disk To Tape in 5 Hours
This gives an average Speed of 50 MB/s.

Just need a simple Performance Tip from Bacula Experts :-)


Ulrich Leodolter <ulrich.leodolter@xxxxxxxx>

This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
Bacula-devel mailing list

This mailing list archive is a service of Copilot Consulting.