[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bacula-devel] small mysql patch


On Friday 05 September 2008 20:32:32 Yuri Timofeev wrote:
> 2008/9/5 Kern Sibbald <kern@xxxxxxxxxxx>:
> >From what I understand, it *does* work, but just slowly. It is not correct
> > to say that nothing is done.  It has been working and doing its job it
> > may have even been removing orphaned records in this time.
>
> No. This is not a work.
>
> Mysql all the time (3 days) performs the same request:
>
> SELECT File.FileId, Job.JobId FROM File LEFT OUTER JOIN Job ON
>        (File.JobId = Job.JobId) WHERE Job.JobId IS NULL LIMIT 300000;
>
> Orphaned records are not deleted.
> There was a difference between 3 days and 3 minutes.

How big is your database?  How many file records?  It sounds to me like you 
have something very badly tuned or you have a badly broken MySQL.

>
> dbcheck does not solve the problem of disk space and you as a
> developer, it should be well known.
>
> > I don't see that there was any need for a reply.  You know the solution
> > for you.  Create the index, or write a script that
>
> creates it, runs dbcheck and then removes it.
>
> When I first encountered a problem, I do not wrote.
> And the problems not only to me. See bacula-users.
>
> > If someone submits a patch to dbcheck that improves it and makes it
> > automatic, all the better.
>
> So I made the desired patch.
>
> And you asked: whether to embed the code dbcheck.c code, which depends
> on a specific database.
>
> I do not ask you to do the work. I ask the council. Maybe later I can
> still do some useful patch.
> But I need to start something.

We explained our general philosophy -- it is a generality not anything 
absolute.  If you want a specific answer for specific code, you will need to 
send the code.

>
> > This is an Open Source program.  You know your solution -- create an
> > index.
>
> I made the desired patch.
>
> And I asked you: whether to embed the code dbcheck.c code, which
> depends on a specific database?
> Since in the other database - (probably?) all OK.
>
> I do not ask you to do the work. I ask the council. 

> Maybe later I can still do some another useful patch for bacula. 

That would be nice.

> But I need to start something.

That is good.

>
> We can go for the easiest way: include an appropriate comment in the
> file src/cats/make_mysql_tables.in after rows
> CREATE TABLE File... They already have a similar comment.

Yes, I have your patch for that and will include it in the near future.

>
> Such pacht I have already sent.
> It is hard to do?
>
> 2008/9/5 Kern Sibbald <kern@xxxxxxxxxxx>:
> > Hello,
> >
> > On Thursday 04 September 2008 20:24:29 Yuri Timofeev wrote:
> >> Hi
> >>
> >> Some time ago I received a problem - dbcheck does not work with options:
> >
> > From what I understand, it *does* work, but just slowly.
> >
> >> 8) Check for orphaned File records
> >> 9) Check for orphaned Path records
> >>
> >> dbcheck worked 3 days and nothing is done. I stopped dbcheck (ctrl+c)
> >> and restart MySQL.
> >
> > It is not correct to say that nothing is done.  It has been working and
> > doing its job it may have even been removing orphaned records in this
> > time.
> >
> >> I explain what I need dbcheck:
> >>
> >> If you do not have enough space on the disk, the bacula (all versions)
> >> *very* bad handles this  situation.
> >
> > If you do not have enough disk space on your system, practically every
> > program that exists will have problems, and Bacula does nee a lot of
> > storage.
> >
> >> And this is another big problem. With the shortage of disk space
> >> bacula could damageв Volumes.
> >> I saw it many times.
> >
> > Running dbcheck is not going to resolve a disk shortage problem. It may
> > free up space in your catalog, but that space will remain unless you
> > explicitly compact your catalog as documented in the manual.
> >
> >> So I had to check the database Catalog for orphaned File records and
> >> orphaned Path records.
> >
> > In my opinion, there are better ways of freeing up space, including
> > compacting your database.
> >
> >> The time by pass and in maillist bacula-users described the same
> >> situation:
> >>
> >> [Bacula-users]
> >> From: Ryan Novosielski <novosirj@xxxxxxxxx>
> >> Date: 18/08/08 17:18
> >> Subj: Bacula 2.2.8, dbcheck never completes
> >>
> >> And I and Ryan Novosielski use DBMS MySQL. I empirically determined
> >> that the need to create an index
> >> CREATE INDEX idxFJ  ON File (FileId, JobId);
> >>
> >> After that dbcheck works VERY quickly.
> >> I described the decision [Bacula-users] and [Bacula-devels].
> >>
> >> My proposals were as follows:
> >> *** include an appropriate comment in the file
> >> src/cats/make_mysql_tables.in after rows
> >> CREATE TABLE File...
> >> (They already have a similar comment.)
> >>
> >> I received a reply - no.
> >
> > I don't see that there was any need for a reply.  You know the solution
> > for you.  Create the index, or write a script that creates it, runs
> > dbcheck and then removes it.
> >
> > As we previously explained, items such as indexes are up to the user to
> > manage in general.  If someone submits a patch to dbcheck that improves
> > it and makes it automatic, all the better.
> >
> >> *** Describe the problem and solution in the documentation, for
> >> example, in section Troubleshooting.
> >>
> >> At this suggestion, I also did not receive the normal response.
> >>
> >> *** I studied the source code dbcheck.c and concluded that there can
> >> not insert the code, which will depend on the particular DBMS (in this
> >> case - MySQL).
> >>
> >> Understand ?
> >
> > No.
> >
> >> Conclusion: there is a problem, but to eliminate it, no one will be?
> >> Let the problem exists and further? Or how?
> >
> > This is an Open Source program.  You know your solution -- create an
> > index.
> >
> > Most of the users of Bacula are hard working sysadmins with many of their
> > own priorities. They are not there to actively work on Bacula but do help
> > users find solutions to problems.
> >
> > Some of us like myself work full time for *free* on projects that the
> > users have voted to be the most important.  Most like myself are
> > generally totally overloaded.  We are not complaining, but for programs
> > such as dbcheck that are not at all critical and should not be needed to
> > run more than once a year, we can only offer advice, which is what we
> > have done.
> >
> > At least it is free and you have the source code, and you have received a
> > good deal of feedback.
> >
> > Best regards,
> >
> > Kern
> >
> >> Thanks.
> >>
> >> 2008/9/3 Kern Sibbald <kern@xxxxxxxxxxx>:
> >> > On Wednesday 03 September 2008 08:22:31 Yuri Timofeev wrote:
> >> >> Hi
> >> >>
> >> >> 2008/8/31 Eric Bollengier <eric@xxxxxxxxxxxxxxxx>:
> >> >> > If you want to use new indexes with dbcheck, you can modify
> >> >> > dbcheck.c in this way :
> >> >> >  - ask to the user if he wants to add them (think about disk space)
> >> >> >  - add them + run analyse
> >> >> >  - run the cleanup operation
> >> >> >  - remove them
> >> >> >
> >> >> > FYI, postgresql can use composite indexes, so it will require less
> >> >> > indexes than mysql.
> >> >>
> >> >> It seems the problem appears only when working with MYSQL.
> >> >>
> >> >> I made a small program in order to make temporary index
> >> >> CREATE INDEX idxFJ  ON File (FileId, JobId);
> >> >> and then removes him.
> >> >>
> >> >> Now I little understand a  Bacula API. And there is one problem:
> >> >> my source code depends on the DBMS (MySQL).
> >> >> And I still do not know how it can be inserted in dbcheck.c
> >> >>
> >> >>
> >> >> If I write (in dbcheck.c) as follows:
> >> >>
> >> >> # ifdef HAVE_MYSQL
> >> >> / * My code uses bacula api * /
> >> >> # endif
> >> >>
> >> >> So it would be wrong, imho, or is this possible?
> >> >> Can you suggest any solution?
> >> >
> >> > Please be specific about the exact MySQL dependencies.



-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-devel mailing list
Bacula-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/bacula-devel


This mailing list archive is a service of Copilotco.