[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Bacula-devel] patch for presence of file daemon


Maxime Rousseau has created a new feature for bacula. This patch has been created in order to optimize the communications between the File daemon and the Director daemon. It has been written for 2.4.0. All regression tests and function tests have been passed for 2.4.0. Patch is ready for 2.4.0. Maxime is making patch for 2.4.2 and trunk before sending it. Here are some explanations:

With the new features, Bacula can backup clients who change their IP like laptops. There are less error messages when a job is canceled because of the absence of the File Daemon. The communication between FD and DIR become bidirectional so connections are more frequent.

New features for the DIR:
	- when the DIR start, he tries to connect to the FD. If the connection is successful, a presence parameter in the Client ressource change to "yes". Else the presence parameter keep his value "no".
	- when the DIR is going to start a new job, he checks the presence parameter. If the client is present, the DIR starts the job, else he waits for him during a time specified in the Client ressource in the bacula-dir.conf (this parameter is named "WaitTimer"). He checks if the client is connected at each interval of a time (attribute "PresenceTimer" in bacula-dir.conf). If the client never connect himself during the "WaitTimer" time, the job is marked as "JSAutomaticallyCanceled" in the Catalog. "JSAutomaticallyCanceled" is a new parameter defined in jcr.h and it means that the job is canceled because the File daemon has never been connected.
	- I have created a new file named fd_server.c. It allow the DIR to listen to the File Daemon connections (the default port is 9104, parameter DIRportFD in Director ressource of bacula.dir.conf). The parameter MaxClientsPresence defined in Director ressource in bacula-dir.conf decide how many File Daemons the DIR can listen simultaneously.
	- Authentifications fonctions are also implemented in authenticate.c in src/dird and src/filed.
New features for the FD:
	- the FD must know the address of the Director which is stocked in the Director ressource in bacula-fd.conf. Also, he knows on which port he is able to contact the DIR (default 9104).
	- when the FD start, he tries to connect to the DIR. If the connection is successful, a presence parameter in the Client ressource of the Director daemon changes to "yes". Else the presence parameter keep his default value "no". For the authentification he uses the existing password between the File Daemon and the Director. The File Daemon gives his new address to the DIR so if the client is a laptop, jobs can be run with any IP.
	- when the File Daemon stops, he warns the DIR he is going away. After this warning, presence_parameter = 0 : the DIR knows the client is absent. This feature doesn't work on Windows system. Perhaps the FD not finished in the same way as it stop on Linux. At least, on Windows, bacula does not go in the fonction "terminate_filed" in filed.c so the presence parameter keep his value at 1. ----> Perhaps there is a possible upgrade to do.
For the connections at the start of the two Daemons, there is a retry_interval defined at 10 seconds (if connection fail, retry after 10 seconds) and a max_retry_time defined at 20 seconds (abandon connection after 20 seconds).

Normally, the old configurations works fine even though files are patched.
If configuration files not exist when we apply the patch, they are created with a new configuration (Presence parameter, PresenceTimer, WaitTimer, Address of the Director...). Else you must modify the configuration files: if the Presence parameter in Client ressource in bacula-dir.conf and the address attribute in Director ressource in bacula-fd.conf not exist, bacula will run like an old configuration. 

Exemple of a new configuration:

1/ In "bacula-dir.conf"

Director {                            # define myself
  Name = localhost-dir
  DIRport = 9101            # where we listen for UA connections
  DIRportFD = 9104  # where we listen for FD connections -----------------> NEW
  QueryFile = "/home/rousseaum/bacula/bin/query.sql"
  WorkingDirectory = "/home/rousseaum/bacula/working"
  PidDirectory = "/home/rousseaum/bacula/working"
  Maximum Concurrent Jobs = 1
  Password = "6V2ghmC6A0YUfncxiF5wJJ1x+WAT2BpUD55l1tfaOury"         # Console password
  Messages = Daemon
  MaxClientsPresence = 20  #How many client the DIR can listen simultaneously
  																			-----------------> NEW

Client {
  Name = localhost-fd
  Address = localhost
  FDPort = 9102
  Catalog = MyCatalog
  Password = "VfCC+e5Lp87mlgdW58PqkxLRvyM2jcwhGCkBMNOOuzXz"          # password for FileDaemon
  File Retention = 30 days            # 30 days
  Job Retention = 6 months            # six months
  AutoPrune = yes                     # Prune expired Jobs/Files
  Presence = yes	# The presence parameter exist -------------------------> NEW
  PresenceTimer = 15 # Maximum time to verify the client presence --------> NEW
  WaitTimer = 60 minutes  # Maximum time to wait the client --------------> NEW
  # PresenceTimer and WaitTimer are defined in second by default. We can use minutes, hours, days... like the other
  # temporal parameter in Bacula.

2/ In "bacula-fd.conf"

Director {
  Name = localhost-dir
  Address = localhost
  DIRport = 9104 ---------------------------------------------------------> NEW
  Password = "VfCC+e5Lp87mlgdW58PqkxLRvyM2jcwhGCkBMNOOuzXz"

Exemple of a typical communication between the FD and the DIR:

1/ Starting daemons:

1.1/ DIR starts before FD (most frequent situations)

DIR starts;
DIR tries to connect to FD;
if (FD connected) {
	presence_parameter = 1;
FD starts;
FD tries to connect to DIR;
if (DIR connected) {
	presence_parameter = 1;
	FD give his new address to DIR;

1.2/ FD starts before DIR

FD starts;
FD tries to connect to DIR;
if (DIR connected) {
	presence_parameter = 1;
	FD give his new address to DIR;
DIR starts;
DIR tries to connect to FD;
if (FD connected) {
	presence_parameter = 1;

1/ Starting job (Backup, Restore):

DIR check FD presence;
if (FD hasn't got presence_parameter) {                  ----> old configuration
	run job like old configuration;
else {																	----> new configuration
	if (FD present) {
		run job;
	else {		
		while (WaitTimer isn't terminate) {
			check FD connection all the PresenceTimer interval;
			if (FD connect) {
				run job;
		Job mark at JSAutomaticallyCanceled;

Any remarks are welcome. We hope this feature to be included in bacula, so we made it with existing clients configuration in mind in order not to disturb existing configurations.


Jean-Sébastien Hederer - Gérant ASPerience
Conseil, intégration, hébergement de systèmes informatiques 
spécialisé en solutions logiciel libre de gestion d'entreprise
Mob.: 06 69 56 21 49
Tél.: 09 80 08 25 41 (nouveau)
Fax.: 09 80 08 25 49 (nouveau)
XMPP/Jabber: hedererjs@xxxxxxxxxxxx
fn;quoted-printable:Jean-S=C3=A9bastien Hederer
adr:CS 50743;;9, rue Alfred Kastler;Nantes Cedex 3;;44307;FRANCE
note;quoted-printable:Conseil, int=C3=A9gration, h=C3=A9bergement de syst=C3=A8mes informatique=
	s sp=C3=A9cialis=C3=A9 en=0D=0A=
	solutions logiciel libre de gestion d'entreprise 

This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
Bacula-devel mailing list

This mailing list archive is a service of Copilotco.