[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bacula-devel] Backing up MS Exchange


On Wednesday 10 September 2008 02:18:08 James Harper wrote:
> > > > 2. As an option be able to do brick level backup or at least brick
> > > > level restore.
> > >
> > > I am lead to believe that Backup Exec can do a brick level (or at
>
> least
>
> > > mailbox level) restore from a full backup, so maybe there are API's
>
> to
>
> > > do that, or maybe just under Exchange 2007.
> > >
> > > Either way, IMHO a brick level backup is a completely different
>
> project
>
> > > to an Exchange Database backup. There is almost nothing in common
>
> with
>
> > > the two other than that they ultimately get data from the same
>
> Exchange
>
> > > database, but at the Windows API level they couldn't be more
>
> different.
>
> > > Make of that what you will in terms of your spec, I'll be in a
>
> better
>
> > > position to comment when I see it.
> >
> > Yes, you are right, it is an entirely different beast.  Some vendors
>
> do it
>
> > and
> > it is often requested.  It appears that it requires the use the MAPI
> > interface as is used by Outlook, and from what I read from Microsoft,
>
> it
>
> > is
> > first very slow, and second subject to a considerable number of
>
> problems.
>
> > I suspect that the way to get brick level restore (mailbox or message
> > level)
> > is to restore the whole database to another server then use ExMerge to
> > pull
> > out the parts you want -- at least that is what it seems is generally
> > recommened by Microsoft and other users.
>
> Under Exchange 2003 and above (*) you can create a special 'Recovery
> Storage Group' which you can restore your database to, and then just
> copy mailboxes from that to your normal database. Obviously you need
> enough disk space free to hold a complete second copy of your mail store
> in the Recovery Storage Group, but that is one way to do a mailbox or
> item restore. I have tested this using my existing agent and it works
> well.

OK, that sounds like a good solution that we need to document.

>
> (*) Another question... I have only ever worked with Exchange 2003.
> Exchange 2007 is out now and I believe the API is compatible. Is there
> any requirement to support Exchange 2000, or 5.5?

I would say to design and implement it for Exchange 2003 and Exchange 2007 and 
if there is any demand for Exchange 2000 or 5.5, we can see later if there 
are any changes.

>
> > > > 3. If possible be able to implement differential and incremental
> > >
> > > backups.
> > >
> > > Based on the work I've already done:
> > >
> > > . a 'full' Exchange backup involves backing up one or more databases
>
> in
>
> > > the Information Store, backing up the logs, and then truncating the
>
> logs
>
> > > afterwards (I put the option in to not truncate too I think via a
>
> dummy
>
> > > 'include' directive in the file set).
> > > . a 'differential' Exchange backup involves backing up the logs only
>
> but
>
> > > not truncating them each time
> > > . an 'incremental' Exchange backup involves backing up the logs only
>
> and
>
> > > then truncating them each time.
> >
> > Yes, these are the options we need.  It is a lot easier to restore an
> > incremental backup rather than the whole database.  I am sure we can
> > handle a
> > differential backup, but I am not quite sure how Bacula will treat the
> > incrementals, since from what I understand, you must restore all the
> > Exchange
> > incrementals to have a valid restore, but then I am kind of ignorant
>
> on
>
> > this.
>
> Yes. The restore operation involves restoring the database and then
> replaying the log files. The log files can be ones you have just
> restored or ones already on disk. If you had a situation where you did a
> backup 24 hours ago and then your database went corrupt (eg your raid
> array blew up), you can restore the database and Exchange will
> automatically replay the logs already on the system (assuming they
> weren't on the array that blew up) and your restore will be effectively
> as at the moment your system died. You need every single log file
> between the last backup and now for this to work though. In many ways
> it's a very nice system.

Thanks.  I understand it better now.  We will need to carefully think about 
how we handle such situations in Bacula regarding Bacula's Differential and 
Incremental backups.  Perhaps we need one more type of backup that is a sort 
of incremental with the difference that all such special incrementals are 
needed not just the last one.  I suppose that the first step is coming up 
with a term for this.

In case, this is not clear, if Bacula has the following records for a single 
file:

Full,  Inc1, Inc2, Inc3.  

on a restore it will put back:

Full, Inc3.

while if I understand correctly, for Exchange we will want to restore:

Full,  Inc1, Inc2, Inc3

>
> > > I'm not sure how much wiggle room we have in terms of controlling
>
> the
>
> > > truncating of the log files. When the backup is done, using the
>
> Exchange
>
> > > API's, you either truncate the logs or you don't. So if you do a
>
> full
>
> > > backup, then an incremental backup, you can no longer do a
>
> differential
>
> > > backup based on the last full backup. You're probably the best
>
> person to
>
> > > comment on how that will interact with any assumptions the existing
> > > Bacula code makes about such things.
> >
> > I think we should give the user the option of truncating the logs or
>
> not.
>
> Agreed.
>
> > > I'll review the plugin API you have developed again and let you
>
> know. C
>
> > > or C++ should be fine. Do you have a preference?
> >
> > I prefer C++, just because it makes cleaner code (checks the source
> > better)
>
> C++ it will be then!

Thanks.

>
> > The only other item I can think of, that I think is automatic in the
>
> API
>
> > you are planning to use is to ensure that Calendar and Contact items
>
> are
>
> > also backed up, and that the plugin works for both Exchange 2003 and
> > Exchange 2007.
>
> Correct. The API backs up complete Information Stores, which contain all
> the mail items, calendar items, contact items, task items, journal
> items, etc

Great.  I was hoping that was the case. Thanks for confirming it.

>
> One other thing, I noticed that the default VSS backup also backs up the
> exchange databases, although not in a way that makes them nice to
> restore and replay the logs. I think it would have to be manual but it
> would be nice for the agent to be able to tell the file daemon not to do
> the vss backup of the exchange databases...

Yes, I haven't figured out exactly how we will handle this.  From what I read 
on the web, if you are doing a VSS backup (Bacula is backing up the whole 
system), Exchange is sort of blocked or locked, so doing the normal API calls 
to Exchange either fail or won't work correctly.  As a consequence, we need 
to ensure that VSS is not enabled at the same time that the plugin is 
running.   

The current Bacula plugin code runs during the normal backup, so if the user 
enabled VSS, it will be active when the plugin runs.  For the moment, we can 
make sure the plugin is run in a job by itself.  Once it works, we can test 
to see if it functions correctly at the same time that VSS is running 
(perhaps Microsoft fixed these problems).  If not,we can possibly delay 
calling the plugin until after all the other files have been backed up and 
VSS has been shutdown.  Many plugins will want to run while VSS is enabled, 
so we will need to figure out something here ...

Best regards,

Kern

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-devel mailing list
Bacula-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/bacula-devel


This mailing list archive is a service of Copilot Consulting.