[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bacula-devel] Feature request: "minimum spool size" setting

John Stoffel wrote:
> Frank> Item: Add a "minimum spool size" setting, defining a minimum
> Frank> level of free space to spool new jobs
> Great idea.  Lousy name.  How about "min free spool size" instead?  

Works for me - I'm more attached to the behavior change than the label
associated with it =)

Or, if you really wanted to be accurate, you could change "maximum spool size"
to "spool high watermark" and add "spool low watermark", since that's really
the behavior it would have then.

> Frank> Date: 20 March 2008
> Frank> Origin: Frank Sweetser <fs@xxxxxxx>
> Frank> What: Add a new SD directive, "minimum spool size" (or
> Frank> similar).  This directive would specify a minimum level of free
> Frank> space available for spooling.  If the unused spool space is
> Frank> less than this level, any new spooling requests would be
> Frank> blocked as if the "maximum spool size" threshold had bee
> Frank> reached.  Already spooling jobs would be unaffected by this
> Frank> directive.
> Frank> Why: I've been bitten by this scenario a couple of times:
> Frank> Assume a maximum spool size of 100M.  Two concurrent jobs, A
> Frank> and B, are both running.  Due to timing quirks and previously
> Frank> running jobs, job A has used 99.9M of space in the spool
> Frank> directory.  While A is busy despooling to disk, B is happily
> Frank> using the remaining 0.1M of spool space.  This ends up in a
> Frank> spool/despool sequence every 0.1M of data.  In addition to
> Frank> fragmenting the data on the volume far more than was necessary,
> Frank> in larger data sets (ie, tens or hundreds of gigabytes) it can
> Frank> easily produce multi-megabyte report emails!
> Would it make more sense to have directives called "spool chunk size"
> and "minimum free spool chunks" instead?  So you could have a chunk
> size of 10mb, with a minimum free spool chunk of 2 (20mb).  
> On my home machine, it's not unusual for me to end up with my entire
> staging disk filled to the rim with a single staged saveset in one
> large file.  
> Breaking it into smaller chunks (10% of max spool size each?)  might
> help.
> Dunno... I do like your idea though.

I've thought about making suggestions along those lines (having a spool chunk
size, where each chunk is independently spooled and despooled, preventing a
single large job from having to pause spooling while despooling is going on),
but never really thought about it enough to nail down exactly what the
semantics ought to be.  This idea should fix at least this problem, and also
(I would imagine) be a lot simpler to implement.

Frank Sweetser fs at wpi.edu  |  For every problem, there is a solution that
WPI Senior Network Engineer   |  is simple, elegant, and wrong. - HL Mencken
    GPG fingerprint = 6174 1257 129E 0D21 D8D4  E8A3 8E39 29E3 E2E8 8CEC

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
Bacula-devel mailing list

This mailing list archive is a service of Copilotco.