2010 Jan 29 - Fri
Migrating Bacula 2.x on Debian Etch to 3.x on Squeeze
Debian Etch, which is the current release, has the Bacula 2.x packages.
I needed to upgrade to the Bacula 3.x packages, which are located in debian/testing,
also known as the forthcoming Debian Squeeze release. In addition, since PostgreSQL 8.3 is
packaged in Etch, and PostgreSQL 8.4 is packaged in Squeeze/testing, a database
migration is also required.
I had attempted updating my sources.list file to testing and then running the
apt-get dist-upgrade process. This broke some dependences, and also broke on a
udev migration. I guess testing has more testing to do on the distribution
upgrade process.
In the end, I built a new Bacula service on a freshly installed Debian testing server.
The special consideration for this configuration is that it needs to handle backing
up servers across a WAN. As such, backups may travel through one or more firewalls. Through
such a configuration, it is very difficult to get the firewall ports opened for the various
necessary Bacula service ports. The better way to tackle this is through the use of
ssh' port local and remote port forwarding capability. Port 22 is becomes the only necessary
port to open on a firewall. The ssh-tunnel.sh script helps make this happen.
To build the server, when it came to package selection, I unselected all packages, and then chose
just the database package which installed PostgreSQL.
After the basic server finished installing and rebooted, I manually installed the following packages:
apt-get install bacula-common-pgsql
apt-get install bacula-client
apt-get install bacula-director-common
apt-get install bacula-director-pgsql
apt-get install bacula-sd-pgsql
apt-get install bacula-server
If starting with a new database, then dbconfig-common can be used. If migrating an older database, don't use
dbconfig-common, and use the manual methods I'll describe further on. There is further documentation in
/usr/share/doc/bacula-director-pgsql.
During installation of the bacula packages, a new user of 'bacula' is created, as well as a group called 'tape'.
The 'bacula' user has a home directory of /var/lib/bacula.
Into that directory, create a .ssh directory for any authorized_keys and known_hosts required. I also created a keys
subdirectory to hold the public/private keys for ssh'ing into other servers for processing backups. I called the
two files 'bacula' and 'bacula.pub'. These will be referenced in my customized ssh-tunnel.sh script.
Run
dpkg-reconfigure exim4-config
to reconfigure the mail system to allow outbound mail delivery.
My backups go onto a remote file share. I created an entry in /etc/fstab along the lines of:
10.1.1.1:/bu /mnt/nas nfs rw,hard,intr,async,nodev,nosuid 0 0
Ensure that the NFS client is installed through:
apt-get install nfs-common
In /etc/postgresql/8.4/main/pg_hba.conf, I have lines along:
host bacula bacula 127.0.0.1/32 trust
host bacula sysadmin 127.0.0.1/32 trust
local bacula bacula trust
local bacula sysadmin trust
As an aside, a useful command to find out database information is through the use of:
psql -l
When migrating the database to 8.4, there are modifcations to the pg_dump command required (which are
required to prevent import errors along the lines of 'ERROR: invalid byte sequence for encoding "UTF8"',
basically resolving the UTF-8 to SQL_ASCII issues in Bacula):
pg_dump -E SQL_ASCII -U bacula bacula > /var/lib/bacula/bacula.sql
On the new server, use the following to import the database:
dropdb bacula
su - postgres
psql
create role bacula;
create database bacula owner=bacula encoding='SQL_ASCII' template=template0;
/q
psql bacula </var/lib/bacula/bacula.sql
Basic instructions for updating the database from Bacula table version 10 to Bacula
table version 11 is found in /usr/share/bacula-director/update_postgresql_tables:
BEGIN;
ALTER TABLE file ALTER fileid TYPE bigint ;
ALTER TABLE basefiles ALTER fileid TYPE bigint;
ALTER TABLE job ADD COLUMN readbytes bigint default 0;
ALTER TABLE media ADD COLUMN ActionOnPurge smallint default 0;
ALTER TABLE pool ADD COLUMN ActionOnPurge smallint default 0;
-- Create a table like Job for long term statistics
CREATE TABLE JobHisto (LIKE Job);
CREATE INDEX jobhisto_idx ON JobHisto ( starttime );
UPDATE Version SET VersionId=11;
COMMIT;
Once the configuration files for the director, storage manager, and file manager are ready, bacula
can be managed through 'bconsole'.
My modified /etc/bacula/scripts/ssh-tunnel.sh looks like:
#!/bin/sh
# script for creating / stopping a ssh-tunnel to a backupclient
# Stephan Holl sholl@gmx.net
# Modified by Joshua Kugler joshua.kugler@uaf.edu
# Modified by Ray Burkholder ray@oneunified.net
#
#
# variables
USER=bacula
CLIENTADDR=$2
# CLIENTPORT is local end
CLIENTPORT=$3
#LOCAL=your.backup.server.host.name
# local is a local address and uses ssh's remote/local port forwarding
LOCAL=127.0.0.1
SSH=/usr/bin/ssh
SSHOPTIONS=-vfnCN2
LOG1=/var/lib/bacula/log1.log
LOG2=/var/lib/bacula/log2.log
#LOG1=/dev/null
#LOG2=/dev/null
# location of the public/private keys used with ssh to gain access to remote servers
KEY=/etc/bacula/keys/bacula
case "$1" in
start)
# create ssh-tunnel
echo "Starting SSH-tunnel to $CLIENTADDR..."
$SSH $SSHOPTIONS -o PreferredAuthentications=publickey -i $KEY -l $USER \
-R 9101:$LOCAL:9101 -R 9103:$LOCAL:9103 -L $CLIENTPORT:$LOCAL:9102 $CLIENTADDR \
>> $LOG1 2>> $LOG2
exit $?
;;
stop)
# remove tunnel
echo "Stopping SSH-tunnel to $CLIENTADDR..."
# find PID killem
PID=`ps ax | grep "$SSH $SSHOPTIONS -o PreferredAuthentications=publickey -i $KEY" \
| grep "$CLIENTADDR" | awk '{ print $1 }'`
kill $PID
exit $?
;;
*)
# usage:
echo " "
echo " Start SSH-tunnel to client-host"
echo " to bacula-director and storage-daemon"
echo " "
echo " USAGE:"
echo " ssh-tunnel.sh {start|stop} client.fqdn"
echo ""
exit 1
;;
esac
The links I used for getting started with ssh-tunnels are found at:
In /etc/hosts file, 127.0.0.1 should be the only line referring to the local server. The exteral port
ip address should be commented out:
127.0.0.1 localhost bu.example.com bu
#10.10.10.1 bu.example.com bu
In the bacula-dir.conf configuration file, a typical client configuration will look similar to:
Client {
Name = mail-fd
Address = 127.0.0.1
FDPort = 9130 # specific port for this client, allows multiple simultaneous backups
Catalog = MyCatalog
Password = "xxxxxx" # password for FileDaemon
File Retention = 120 days
Job Retention = 4 months
AutoPrune = yes # Prune expired Jobs/Files
}
The special characteristic of the above configuration is the use of a unique port number for FDPort.
Each client in the bacula-dir.conf should have a unique port number. This allows bacula to
tunnel via ssh to remote clients and redirect them to the storage manager on the
local server.
The definition of the storage device in bacula-dir.conf will have Address=127.0.0.1 and SDPort=9103.
The job description for each client should have something similar to:
Job {
Name = "mail-fd"
Client = mail-fd
JobDefs = "DefaultJob"
FileSet = "FileSet_mail"
Storage = storageSshClients
Write Bootstrap = "/var/lib/bacula/mail.bsr"
Priority = 12
Run Before Job = "/etc/bacula/scripts/ssh-tunnel.sh start mail.example.com 9130"
Run After Job = "/etc/bacula/scripts/ssh-tunnel.sh stop mail.example.com 9130"
}
When using Bacula in console mode, a useful command to find out the meaning of the
backup status codes:
*sqlquery
Entering SQL query mode.
Terminate each query with a semicolon.
Terminate query mode with a blank line.
Enter SQL query: select * from status;
+-----------+---------------------------------+
| jobstatus | jobstatuslong |
+-----------+---------------------------------+
| C | Created, not yet running |
| R | Running |
| B | Blocked |
| T | Completed successfully |
| E | Terminated with errors |
| e | Non-fatal error |
| f | Fatal error |
| D | Verify found differences |
| A | Canceled by user |
| F | Waiting for Client |
| S | Waiting for Storage daemon |
| m | Waiting for new media |
| M | Waiting for media mount |
| s | Waiting for storage resource |
| j | Waiting for job resource |
| c | Waiting for client resource |
| d | Waiting on maximum jobs |
| t | Waiting on start time |
| p | Waiting on higher priority jobs |
+-----------+---------------------------------+
Enter SQL query:
End query mode.
For the bacula entry in /etc/passwd, change /bin/false to be /bin/sh.
For each server to which will be connected via ssh, within the
context of the bacula user, use the following command to update ~/.ssh/known_hosts:
ssh -l bacula -i /etc/bacula/keys/bacula -v server.example.com
|
