[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bacula-devel] MaximumBlockSize Problem and Question


Hi,

05.11.2008 16:12, Kern Sibbald wrote:
> For writing to tape (providing it is LTO-n) I strongly recommend a block size 
> not to exceed 256K.

And I strongly recommend to not measure throughput using /dev/zero... 
if you use compression, you're almost only measuring bus throughput as 
all the zeros will be compressed away... it's better to prepare a big 
file with random data for it. Unfortunately, that brings the disk 
system into the environment again. A big RAM-disk can help, but you'd 
need lots of RAM for this to be useful.

Unfortunately, reading from /dev/urandom directly is usually rather slow.

Arno

> On Wednesday 05 November 2008 14:48:20 Ulrich Leodolter wrote:
>> On Wed, 2008-11-05 at 13:34 +0200, Pasi Kärkkäinen wrote:
>>> On Wed, Nov 05, 2008 at 11:12:01AM +0100, Ulrich Leodolter wrote:
>>>> Hello,
>>>>
>>>> Problem:
>>>>
>>>> Settings like (documented in the Manual)
>>>>
>>>>      Minimum block size = 64K
>>>>      Maximum block size = 200K
>>>>
>>>> do not work. Both are defined as size_pint32.
>>>> multipliers like K M are not allowed.
>>>> don't know if this is intended,
>>>> maybe documentation just a little outdated.
>>>>
>>>>
>>>> Question:
>>>>
>>>> Can i expect performance improvements (especially for
>>>> Copy/Migrate Disk to Tape jobs) by by increasing
>>>> "Maximum block size" on File devices in bacula-sd.conf ???
>>>>
>>>> Device {
>>>>   Name = FileStorage
>>>>   Media Type = File
>>>>   Archive Device = /disk0/bacula/files
>>>>   LabelMedia = yes;
>>>>   Random Access = Yes;
>>>>   Requires Mount = No;
>>>>   AutomaticMount = yes;
>>>>   RemovableMedia = no;
>>>>   AlwaysOpen = no;
>>>>   MaximumBlockSize = 1048576;         # 1M
>>>> }
>>> I think it should make difference at least with tape drives..
>>>
>>> You can measure the difference with dd:
>>>
>>> dd if=/dev/null of=/dev/tape bs=<try_different_block_sizes_here>
>>>
>>> 4k, 64k, 256k, 1MB, 4MB etc.
>> Hello,
>>
>> I dont think raw performance is bad, as u can see below
>> (exeption is 4k block size on tape,
>>  hopfully bacula does not use it :-)
>>
>> Maybe i should set
>>
>> 	Minimum Block Size = 4194304
>>
>> on both devices (Tape, File)
>> This should reduce the number of block read/writes communication
>> by a factor of 64.
>>
>> Will try it tonight,  unless u give me a very good hint how to
>> optimize ;-)
>>
>> BR
>> Ulrich
>>
>> ---++ Raw Tape Performance
>>
>> [root@troll ~]# dd if=/dev/zero of=/dev/nst1 bs=4k count=256k
>> 262144+0 records in
>> 262144+0 records out
>> 1073741824 bytes (1.1 GB) copied, 48.2057 seconds, 22.3 MB/s
>>
>> [root@troll ~]# dd if=/dev/zero of=/dev/nst1 bs=64k count=64k
>> 65536+0 records in
>> 65536+0 records out
>> 4294967296 bytes (4.3 GB) copied, 41.4204 seconds, 104 MB/s
>>
>> [root@troll ~]# dd if=/dev/zero of=/dev/nst1 bs=256k count=16k
>> 16384+0 records in
>> 16384+0 records out
>> 4294967296 bytes (4.3 GB) copied, 34.1308 seconds, 126 MB/s
>>
>> [root@troll ~]# dd if=/dev/zero of=/dev/nst1 bs=1024k count=4k
>> 4096+0 records in
>> 4096+0 records out
>> 4294967296 bytes (4.3 GB) copied, 32.3195 seconds, 133 MB
>>
>> [root@troll ~]# dd if=/dev/zero of=/dev/nst1 bs=4096k count=4k
>> 4096+0 records in
>> 4096+0 records out
>> 17179869184 bytes (17 GB) copied, 119.205 seconds, 144 MB/s
>>
>> ---++ Raw SATA Raid Performance
>>
>> [root@troll ~]# dd if=/dev/zero of=/disk0/test.img bs=4k count=4096k
>> 4194304+0 records in
>> 4194304+0 records out
>> 17179869184 bytes (17 GB) copied, 83.0424 seconds, 207 MB/s
>>
>> [root@troll ~]# dd if=/dev/zero of=/disk0/test.img bs=64k count=256k
>> 262144+0 records in
>> 262144+0 records out
>> 17179869184 bytes (17 GB) copied, 81.8973 seconds, 210 MB/s
>>
>>> -- Pasi
> 
> 
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Bacula-devel mailing list
> Bacula-devel@xxxxxxxxxxxxxxxxxxxxx
> https://lists.sourceforge.net/lists/listinfo/bacula-devel

-- 
Arno Lehmann
IT-Service Lehmann
Sandstr. 6, 49080 Osnabrück
www.its-lehmann.de

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-devel mailing list
Bacula-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/bacula-devel


This mailing list archive is a service of Copilot Consulting.