[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bacula-devel] small mysql patch


On Sunday 31 August 2008 23:17:05 Yuri Timofeev wrote:
> 2008/8/31 Eric Bollengier <eric@xxxxxxxxxxxxxxxx>:
> > Le Sunday 31 August 2008 21:32:16 Yuri Timofeev, vous avez écrit :
> >> Hi
> >>
> >> 2008/8/31 Eric Bollengier <eric@xxxxxxxxxxxxxxxx>:
> >> > Adding indexes will probably speed up dbcheck, but it will slow down
> >> > the attribute insertion process and grow up the database.
> >>
> >> I wrote earlier (see reference to the discussion), that dbcheck can't
> >> complete the job for 3 days!
> >> Once the necessary indexes were created, all work was completed
> >> dbcheck for 3-5 minutes!
> >> Therefore, I say NOT to increase the speed of dbcheck (though this
> >> too), I say about NOT normal work dbcheck.
> >
> > it's why i suggest you to add them to dbcheck.c, if we add this to the
> > default creation script, we can be sure that users will let them in place
> > forever. And after, they will complain about performance problem.
>
> If the problem only in dbcheck (and no problem with Verifies Jobs)
> then have to change dbcheck.
> Maybe I can do it later, I - beginner C programmer ;)

Ok, this is not a problem, you can help with many other things :)

> So I asked here (bacula-devel):
> "What IDE, editors, plugins, utilities, etc. You use in developing the
> bacula? In other words, are interested in your working environment. "
> ;)

I'm using emacs

> >> > So, we can say that speed of attribute insertion is much more
> >> > important than dbcheck performance.
> >>
> >> MySQL performs INSERT operations with a very high speed, thus slowing
> >> the work should not be noticeable, IMHO.
> >
> > Performance are much higher with few indexes, i'm not sure that mysql
> > will insert million of rows at the same speed if you add 5 indexes.
>
> Million files in a single Jobs? Maybe this is indeed a rare case.

Not, look at the users list, one of them ask question about a 27 million files 
fileset...

> >> > The dbcheck operation have to be run once per month, maybe twice per
> >> > year.
> >>
> >> I do understand (see comments in src/cats/make_mysql_tables.in:46,
> >> revision 7535) that there are problems with Verifies Jobs.
> >
> > You can use them if your verify jobs are too slow. Many users don't use
> > this feature. In bacula, you are free to modify indexes as you want.
>
> I agree with you, but all these questions, imho, political:
> whether bacula function normally "out of the box" (decision for a
> large number of users / system administrators), or bacula will work
> for hackers (who will explore SQL queries, create / delete indexes,
> etc.).
> For example, I often do not have time to do so.

No, don't say hackers. If you have a large installation with a big database,
you need advices from a DBA, not from a hacker.

> No problem will not happen, imho, if Job will be 5-10 minutes longer,
> and if the database will be at 5-10 Gb more.

Not a problem for you, it's a real problem for users with large backups.

> >> > If you want to use new indexes with dbcheck, you can modify dbcheck.c
> >> > in this way :
>
> 1. check that used mysql (?)
>
> 2. to verify the existence of indices, if not there, then :
> >> >  - ask to the user if he wants to add them (think about disk space)
> >> >  - add them + run analyse
> >> >  - run the cleanup operation
> >> >  - remove them

IMHO, It's a good start for a new small project.

> >> > I'm not sure that all your indexes are useful, for example (FileId,
> >> > JobId) is probably used to see if you have orphan jobids in File
> >> > table. The Job table is very small compare to others, so the database
> >> > engine won't use it.
> >> >
> >> > Here, it's not the type of your index that will be important, it's
> >> > much more the size of the File table (hundred of million in my case)
> >> > that will change the cost of your index.
> >>
> >> I think the problem (dbcheck) and its solution can be described, for
> >> example, in the section "Troubleshooting" in the documentation.
> >> And then not need to change the source code.
> >
> > This solution will not decrease support request about this subject, too
> > few users are reading this document. Modify dbcheck is a better idea, if
> > you can't work on it, perhaps we can add this to the project list.
>
> I do not know.
> On the other hand, it is really not the biggest problem.
>
> For example, I am more concerned about progress on the problem from
> the file projects:
> "Item 8: Implement Copy pools" ;)
> I'll do a separate message on this issue.

Ok

Bye

> >> >> I believe that the results of this discussion concerns
> >> >>
> >> >> "[Bacula-users] Bacula 2.2.8, dbcheck never completes"
> >> >> http://sourceforge.net/mailarchive/forum.php?thread_name=48A984BF.405
> >> >>020 6%4 0umdnj.edu&forum_name=bacula-users
> >> >>
> >> >> requires a small patch, see attached file.
> >> >>
> >> >> Since all fields (which fall in the indices) type INTEGER  the size
> >> >> of the indices will be low and increase speed dbcheck will be very
> >> >> substantial.



-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-devel mailing list
Bacula-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/bacula-devel


This mailing list archive is a service of Copilotco.