[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bacula-devel] Feature request: Amazon S3 integration

> Bacula, as far as I know, is meant to work across different Unices. If
> you insist on putting S3 support into the operating system's kernel,
> you're effectively limiting your S3 support to the few operating
> which provide this.

Not at all what I'm doing. I'm doing a user-space API to a managed
storage system. The applications are responsible for opening a channel
to the managed storage system, asking to store an object, and receiving
a unique object id in return. The application keeps track of the object
by storing that unique object ID. The remainder of the details of how
the object is stored and managed are *not the application's problem*. 

> Tapes, file systems, and S3 are three very different interfaces to
> storage. Any sort of common emulation layer for all three is bound to
> either lossy, leaky, or both.

Not emulation. I want the entire details of how data is stored and
managed to be *opaque and invisible*. This is how DFSMS works on z/OS,
and it's *extremely* effective. I have almost 2 EB managed in this form
right now. It works.

> Don't get me wrong. I'm all for abstraction layers. I've worked on a
> myself.  In this case, however, the appropriate abstration layer is
> Bacula's storage daemon.

The appropriate interface is to teach the Bacula SD to speak to the
managed storage system, and never have to deal with Bacula storage
management again. You then teach the managed storage system how to work
with a new storage medium, and *every* application that uses it can then
use the new medium transparently. 

Not OS specific, not platform specific, not application specific. You
need a C compiler and a TCP socket capability. I have that even on z/OS
and Windows. 

The API provides a connect, create object, store data into object,
retrieve object, expire object, delete object, disconnect set of
functions. The objects are opaque to the storage system and can contain
anything the client likes. 

The rest of the data storage functions are part of the managed storage
system; the client doesn't know or care how the data is stored as long
as it can retrieve it when it wants it. 

This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
Bacula-devel mailing list

This mailing list archive is a service of Copilotco.