[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bacula-devel] Marking Files for Restore

Dan Langille wrote on 11/18/2008 10:43 PM:
> On Nov 18, 2008, at 3:06 PM, Tim Frank wrote:
>> I will not pretend to understand all of the complexities of the system
>> for multiple jobID's and file versions, or which queries are or are not
>> compatible across PostgreSQL, MySQL and SQLite, but it would seem that
>> some optimization may be possible.
> At first glance, I agree.  I'd have to look deeper into this to know more.
> We have traditionally sought to keep the SQL the same whereever we
> can.  But I'm thinking there are some thing we can do to speed this up.
> I have only but glanced at your comments and don't have the time to
> investigate now.

I have dug a little deeper into the issue and I believe that the large 
number of queries is related to hardlinks. I noticed that when marking 
various directories that sometimes there would be no hits to the 
database when marking 12,000 or 27,000 files. Other times there would be 
multiple hits to the database when marking 100 files.

The most simple example are the files "/bin/gzip", "/bin/gunzip", and 
"/bin/zcat", which are all hardlinks. When issuing "mark bin" in 
bconsole, there are two database hits for "/bin/gunzip" and "/bin/zcat".

The large number of database hits when marking the /usr directory come 
from the fact that I have 44,479 hardlinks in that directory. So, there 
was not quite a query for each of the 79,979 files, but one or more for 
each of the 22,639 unique hardlinks. (A few kernel source packages are 

The only thing I could think of to speed this up would be to keep a list 
of the hardlinks and execute a single query or a smaller number of 
queries to mark these entries.

Thanks again for any insight.


This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
Bacula-devel mailing list

This mailing list archive is a service of Copilot Consulting.