[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bacula-devel] Tokyo Cabinet DBM candidate for the accurate project


On Friday 28 March 2008 00:16:11 Kern Sibbald wrote:
> I ran a small test of Tokyo Cabinet DBM compared to our htable routines.
> Inputting 5 million records to htable and then reading them all back takes
> 7.7 seconds, and uses up 240MB .
> Inputting 5 million records to TCDBM (using the same records as above) and
> reading them back takes 1 minute 33 seconds.

In our case, 1 or 2 minutes in a 5 million files backup is not such a too big 
cost :) But during this time, the director have to lock the db connection... 
so many things can be frozen.

Have you run this test with valid filename ? 

> Using 1 million records, it runs in 8.8 seconds.
> So, it is a bit slower for at a million records, and quite a bit at 5
> million, but that can probably be tuned.  In those tests, I did tune it to
> use something like 40MB of memory.  In addition, it mallocs and frees each
> record returned.  He has calls to allow the records to be returned in our
> own buffers, so this would probably reduce the time a lot.

Yes, we can also probably skip some bucket re-allocation if we know in advance
how many files we have.


Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
Bacula-devel mailing list

This mailing list archive is a service of Copilotco.