Incrementell – Linux Als Server

Incrementelle Sicherung

Übersicht:

Aufgabe
Verzeichnisse
Backup- Configuration
Backup- Scripte
Tape- Rotation und Mails an Backup- Admin
Cronjobs
Test/ Erfolgskontrolle

Downloads:

backup-0.1-2.noarch.rpm (6111 Bytes – RPM Paket zum incrementellen Backup auf 2 Streamern)
backup-0.1-2.src.rpm (6902 Bytes – SRPM Paket zum incrementellen Backup auf 2 Streamern)
GnuPG Public Key und Fingerprint oder

Aufgabe

Zusätzlich zu den unter Backup genannten Punkten gelten folgende Rahmenbedingungen:

Es werden die beiden vorhandenen Streamer verwendet: auf den einen Streamer wird die wöchentliche Vollsicherung vorgenommen, auf dem anderen die täglichen incrementeller Backups. Somit bestehen 2 identisch benannte Tape- Sätze A …N . Die Tapes müssen nur noch wöchentlich gewechselt werden.

Verzeichnisse

Folgende Verzeichnisse sind in host:/backup vorhanden:

host:/
/backup/inc		target directory for incremental backup
/backup/weekly		target dirextory for weekly full backup
/bin			backup scripte
/etc/backup		backup configuration files
/etc/cron.d		crontab file for autom. backup run
/var/log/backup		log- and status files

Backup- Konfiguration

Die Konfigurationsdateien zum Backup befinden sich im Verzeichnis “./etc”:

/etc/backup/backup.conf
############################################################################
#
# $Id: server-config-backup-inc.php,v 2.1 2003/09/01 19:57:49 love Exp $
#
# central configuration file for Olaf Segger backup configuration
#
# supports incremental backup daily, relativ to last weekly backup
# supports backup to tape drives
# supports mail notification for change tape in streamer
#
############################################################################
# Contents:
# I   General settings
# II  Weekly settings
# III Daily settings
# IV  Mail notification
# V   Functions
############################################################################
#
# I   General settings
#
############################################################################
#
# MAINDIR
# the root of the backup sub system
# MAINDIR="/"
MAINDIR="/"
#
# CONFIGDIR
# the directory for the configuration
# CONFIGDIR="/etc/backup"
CONFIGDIR="/etc/backup"
#
# LOGDIR
# the directory for the log- and statusfiles of the jobs
# and the tape device
# LOGDIR="/var/log/backup"
LOGDIR="/var/log/backup"
#
# TARGETDIR
# the directory for the backup files
# TARGETDIR="/backup"
TARGETDIR="/backup"
#
# TMPFILE
# file for tempory save of configuration, uses the bash env. $RANDOM
# TMPFILE="/tmp/$RANDOM.tmp"
TMPFILE="/tmp/$RANDOM.tmp"
#
# BACKUPDIRS
# get directories and description from seperate config file
# BACKUPDIRS=`cat $CONFIGDIR/backup-dirs.conf | grep -v "#"`
BACKUPDIRS=`cat $CONFIGDIR/backup-dirs.conf | grep -v "#"`
#
# WEEKLYCHANGE
# the day of the week for change the weekly backup set
# depends off system settings, Sunday or Monday is 0 ("date" and "cron")
# On Linux default is 0 = sunday, 
# The default day for weekly change is Monday = 1
# WEEKLYCHANGE="1"
WEEKLYCHANGE="1"
# 
# TAPECHANGE
# day of week to change the tape. In current concep the tape has to
# changed after the last incremental backup and before next weekly
# full backup: on sunday (day 0) has the tape to changed.
# TAPECHANGE="0"
TAPECHANGE="0"
#
# TAPENAMEFILE
# file with name of the tape, only descriptional for mailbody
# TAPENAMEFILE="$LOGDIR/weekly-tape"
TAPENAMEFILE="$LOGDIR/weekly-tape"
#
# DAYOFWEEK
# get the current day of week
# DAYOFWEEK=`date +%w`
DAYOFWEEK=`date +%w`
#
############################################################################
#
# II  Weekly settings
#
############################################################################
#
# WEEKLYDIR
# dirextory for the weekly backup files
# WEEKLYDIR="$TARGETDIR/weekly"
WEEKLYDIR="$TARGETDIR/weekly"
#
# WEEKLYLOG
# logfile for the weekly backup
# WEEKLYLOG="$LOGDIR/weekly.log"
WEEKLYLOG="$LOGDIR/weekly.log"
#
# WEEKLYSTREAMER
# the tape- device on local machine e.g. "/dev/st0" for weekly backup, 
# remote machines are possile, e.g. "tapehost:/dev/st0"
# STREAMER="/dev/st0"
WEEKLYSTREAMER="/dev/st0"
#
# WEEKLYTAPETYPE
# type definition for weekly tape to change, only descrptional for mailbody
# WEEKLYTAPETYPE="DDS2/120m/4-8GB"
WEEKLYTAPETYPE="DDS2/120m/4-8GB"
#
# WEEKLYSTREAMERDESC
# description of the weeklystreamer drive, only descriptional for mailbody
# STREAMERDESC="grauen"
WEEKLYSTREAMERDESC="grauen"
#
############################################################################
#
# III Daily settings
#
############################################################################
#
# DAILYDIR
# dirextory for the daily backup files
# DAILYDIR="$TARGETDIR/inc"
DAILYDIR="$TARGETDIR/inc"
#
# DAILYLOG
# logfile for the daily backup
# DAILYLOG="$LOGDIR/daily.log"
DAILYLOG="$LOGDIR/daily.log"
#
# DAILYPREFIX
# filename prefix for daily backup files, builded with the date command
# DAILYPREFIX=`date +%a-%Y-%m-%d`
DAILYPREFIX=`date +%a-%Y-%m-%d`
#
# RELATEFILE
# the relation file for the daily incremental backups, needed by find
# RELATEFILE=$LOGDIR/start-weekly.log
RELATEFILE=$LOGDIR/start-weekly.log
#
# DAILYSTREAMER
# the tape- device on local machine e.g. "/dev/st0" for daily backup,
# remote machines are possile, e.g. "tapehost:/dev/st0"
# DAILYSTREAMER="/dev/st1"
DAILYSTREAMER="/dev/st1"
#
# DAILYTAPETYPE
# type definition for daily tape to change, only descrptional for mailbody
# DAILYTAPETYPE="DDS1/90m/2-4GB"
DAILYTAPETYPE="DDS1/90m/2-4GB"
#
# DAILYSTREAMERDESC
# description of the daily streamer drive, only descriptional for mailbody
# DAILYSTREAMERDESC="schwarzen"
DAILYSTREAMERDESC="schwarzen"
#
############################################################################
#
# IV  Mail notification
#
############################################################################
#
# SENDMAILBIN
# sendmail binary with params to use to send a mail, input with pipe
# SENDMAILBIN="/usr/lib/sendmail -t"
SENDMAILBIN="/usr/lib/sendmail -t"
#
# SENDMAILTO
# sendmail syntax with addresses to send a mail for tape change
# SENDMAILTO="root@localhost"
SENDMAILTO="root@localhost"
#
# MAILSCRIPT
# filename of the script that's send a mail to admin for tape change
# MAILSCRIPT="$MAINDIR/bin.NEW/mail.sh"
MAILSCRIPT="/bin/backup-mail"
#
############################################################################
#
# V   Functions
#
############################################################################
#
# create the tar files, used by daily and weekly
createBackupFiles(){
	SUBTOUSE=$1
	for BACKSET in $BACKUPDIRS; do
		DESC=`echo $BACKSET | awk -F ":" '{ print $1 }' `
		SOURCE=`echo $BACKSET | awk -F ":" '{ print $2 }'`
		$SUBTOUSE $SOURCE $DESC
	done
}
# create a log-/statusfile from current tape, used by daily and weekly
createLOG(){
	LOGFILE=$1
	STREAMER=$2
	echo "Backup and tape status:" > $LOGFILE
	date >> $LOGFILE
	tar -tvf $STREAMER >> $LOGFILE
	date >> $LOGFILE
	$MAILSCRIPT
}

/etc/backup/backup-dirs.conf
####################################################################
#
# $Id: server-config-backup-inc.php,v 2.1 2003/09/01 19:57:49 love Exp $
#
# list of directory's to backup with filename for tar- file
#
# :<dir[,dir]>
#
# sample:
# backup;/backup/config,/backup/etc,/backup/var
####################################################################
# Server
mysql:/var/lib/mysql
cvs:/var/lib/cvs
etc:/etc
root:/root
var-spool-mail:/var/spool/mail
opt:/opt
f-prot:/usr/local/f-prot
# Data
home:/export/home
httpd:/export/share/httpd
sfirm:/export/share/develop/SFIRM
develop:/export/share/develop
domain-logon:/export/share/domain-logon
linux-app:/export/share/linux-app
pictures:/export/share/pictures.segger.net
public:/export/share/public
# backup configuration
backup:/backup/bin,/backup/etc,/backup/log
# End of file
</dir[,dir]>

Backup- Scripte

/bin/daily.sh
#! /bin/sh
#
# $Id: server-config-backup-inc.php,v 2.1 2003/09/01 19:57:49 love Exp $
#
# this script creates the daily backup
# as incremental from last weekly backup
#

# source the config file
. /backup/etc/backup.conf

# my sub for daily incremental backup
createDaily (){
	ALLSOURCE=$1
	FILENAME=$2
	SOURCEDIR=`echo $ALLSOURCE | sed s/,/\ /g`
	find $SOURCEDIR -cnewer $RELATEFILE -type f > $TMPFILE
	RECORDCOUNT=` wc -l $TMPFILE | awk '{ print $1 }' `
	if [ "$RECORDCOUNT" == "0" ]; then
		 echo "no files for backup found in $SOURCEDIR" \
			 > $DAILYDIR/$DAILYPREFIX-$FILENAME.txt
	else
		 tar -czf $DAILYDIR/$DAILYPREFIX-$FILENAME.tar.gz \
			-T $TMPFILE -Pv \
			> $DAILYDIR/$DAILYPREFIX-$FILENAME.txt
	fi
	rm -f $TMPFILE
}

# MAIN

if [ "$DAYOFWEEK" == "$WEEKLYCHANGE" ]; then
	# only at day of weekly backup no incremental backup and delete
	# the files from last old incremental backup week
	# clean the target directory
	rm -f $DAILYDIR/*
else
	# create the backup files
	createBackupFiles "createDaily"
	# write to streamer, overwrite backup from yesterday
	# temporary append to weekly backup
	tar -cf $WEEKLYSTREAMER $DAILYDIR/$DAILYPREFIX*
	# create a status/log file
	createLOG "$DAILYLOG" "$WEEKLYSTREAMER"
fi


/bin/weekly.sh
#! /bin/sh
#
# $Id: server-config-backup-inc.php,v 2.1 2003/09/01 19:57:49 love Exp $
#
# this script creates the weekly backup
#

# source the config file
. /backup/etc/backup.conf

# my sub for weekly full backup
createWeekly (){
	ALLSOURCE=$1
	FILENAME=$2
	SOURCEDIR=` echo $ALLSOURCE | sed s/,/\ /g`
	tar -czf $WEEKLYDIR/$FILENAME.tar.gz $SOURCEDIR -Pv \
		> $WEEKLYDIR/$FILENAME.txt
}

### MAIN

if [ "$DAYOFWEEK" == "$WEEKLYCHANGE" ]; then
	# clean the target directory
	rm -f $WEEKLYDIR/*
	# create the relativ file for incremental backups
	echo "Start the weekly backup" > $RELATEFILE
	# create tar files
	createBackupFiles "createWeekly"
	# write to streamer
	tar -cf $WEEKLYSTREAMER $WEEKLYDIR
	# create status/logfile
	createLOG "$WEEKLYLOG" "$WEEKLYSTREAMER"
fi

/bin/rpm-updates.sh
#! /bin/sh
#
# $Id: server-config-backup-inc.php,v 2.1 2003/09/01 19:57:49 love Exp $
#
# this script creates tar files from downloaded rpm updates
#

# some settigs
TARGETDIR="/backup/rpm-updates"
LOGFILE="/backup/log/rpm-updates"

# my sub
createArchive (){
	SOURCEDIR=$1
	FILENAME=$2
	tar -cf $TARGETDIR/$FILENAME.tar $1 -Pv > $TARGETDIR/$FILENAME.txt
}

fileBackup(){
	#  save the updates of ncftpgeted rpm files
	createArchive "/export/share/ftp/pub/rh73/updates" "rh73-updates"
	createArchive "/export/share/ftp/pub/rh90/updates" "rh90-updates"
}

### MAIN

# clean the target directory
rm -f $TARGETDIR/*

# create tar files
fileBackup

# create a log file
echo "Backup of $TARGETDIR:" > $LOGFILE
date >> $LOGFILE
cd $TARGETDIR
ls -la  >> $LOGFILE

Tape- Rotation und Mails an Backup- Admin

/bin/mail.sh
#! /bin/sh
#############################################################################
#
# $Id: server-config-backup-inc.php,v 2.1 2003/09/01 19:57:49 love Exp $
#
#############################################################################
#
# ths script send mail to backup- Admin to rotate/change the tapes in
# the streamer
#
#############################################################################

# source the config file
. /backup/etc/backup.conf

# MAIN

if [ "$DAYOFWEEK" == "$TAPECHANGE" ]; then
	# we do it only only on tape change day
	CURRENTTAPENAME=`cat $TAPENAMEFILE`
	case $CURRENTTAPENAME in 
		A)
			NEXTTAPENAME="B"
			;;
		B)
			NEXTTAPENAME="C"
			;;
		C)
			NEXTTAPENAME="D"
			;;
		D)
			NEXTTAPENAME="E"
			;;
		E)
			NEXTTAPENAME="F"
			;;
		F)
			NEXTTAPENAME="G"
			;;
		G)
			NEXTTAPENAME="H"
			;;
		H)
			NEXTTAPENAME="A"
			;;
	esac
	echo "$NEXTTAPENAME" > $TAPENAMEFILE
	# create contentfile of tape
	cp -f $WEEKLYLOG $LOGDIR/$CURRENTTAPENAME-weekly-contents.txt
	cp -f $DAILYLOG $LOGDIR/$CURRENTTAPENAME-daily-contents.txt
	# eject the tape for change 
	mt -f $WEEKLYSTREAMER eject
	mt -f $DAILYSTREAMER eject
	# create mailmessage
	echo "To: $SENDMAILTO " > $TMPFILE
	echo "Subject: Bandwechsel in den Streamern" >> $TMPFILE
	echo "Es ist das Band vom Typ '$WEEKLYTAPETYPE' im $WEEKLYSTREAMERDESC Streamer zu wechseln" >> $TMPFILE
	echo "Es ist das Band vom Typ '$DAILYTAPETYPE' im $DAILYSTREAMERDESC Streamer zu wechseln" >> $TMPFILE
	echo "Es ist jeweils Band '$CURRENTTAPENAME' zu entnehmen." >> $TMPFILE
	echo "Es ist jeweils Band '$NEXTTAPENAME' einlegen." >> $TMPFILE
	echo "" >> $TMPFILE
	echo "Gruss Olaf" >> $TMPFILE
	cat $TMPFILE | $SENDMAILBIN
	rm -f $TMPFILE 
fi

Cronjobs

/etc/cron.d/backup
# backup jobs with incremental concept
00 03  * * * /bin/backup-daily
30 05  * * * /bin/backup-weekly

Test/ Erfolgskontrolle
Nach dem Abarbeiten der einzelnen Jobs stehen entsprechende Status- Files zur Verfü,gung:

# Status des letzten taeglichen Backups: 
> cat /var/log/backup/daily
Backup and tape status:
Tue Jan  7 02:13:35 CET 2003
drwxr-xr-x root/root         0 2003-01-07 01:28:40 backup/daily/
-rw-r--r-- root/root  55403482 2003-01-07 01:16:16 backup/daily/var-spool-mail.tar.gz
-rw-r--r-- root/root      6407 2003-01-07 01:28:40 backup/daily/backup-config.tar.gz
-rw-r--r-- root/root    206933 2003-01-07 01:15:40 backup/daily/cvs.txt
-rw-r--r-- root/root   3037603 2003-01-07 01:28:29 backup/daily/httpd.txt
-rw-r--r-- root/root  17755030 2003-01-07 01:15:52 backup/daily/root.tar.gz
-rw-r--r-- root/root        10 2003-01-07 01:15:52 backup/daily/var-mail.txt
-rw-r--r-- root/root   4152080 2003-01-07 01:15:44 backup/daily/etc.tar.gz
-rw-r--r-- root/root       116 2003-01-07 01:15:52 backup/daily/var-mail.tar.gz
-rw-r--r-- root/root     42447 2003-01-07 01:15:44 backup/daily/etc.txt
-rw-r--r-- root/root   1688332 2003-01-07 01:25:13 backup/daily/home.txt
-rw-r--r-- root/root 291048254 2003-01-07 01:28:29 backup/daily/httpd.tar.gz
-rw-r--r-- root/root     36678 2003-01-07 01:28:40 backup/daily/sfirm.txt
-rw-r--r-- root/root      2334 2003-01-07 01:16:15 backup/daily/var-spool-mail.txt
-rw-r--r-- root/root    115462 2003-01-07 01:15:52 backup/daily/root.txt
-rw-r--r-- root/root  52218523 2003-01-07 01:15:40 backup/daily/cvs.tar.gz
-rw-r--r-- root/root       696 2003-01-07 01:28:40 backup/daily/backup-config.txt
-rw-r--r-- root/root  14224211 2003-01-07 01:28:40 backup/daily/sfirm.tar.gz
-rw-r--r-- root/root    296015 2003-01-07 01:16:16 backup/daily/opt.tar.gz
-rw-r--r-- root/root 1428418665 2003-01-07 01:25:13 backup/daily/home.tar.gz
Tue Jan  7 03:03:45 CET 2003

# Status des letzten woechentlichen Backups: 
> cat /vgar/log/backup/weekly
Backup and tape status:
Mon Jan  6 03:26:55 CET 2003
drwxr-xr-x root/root         0 2003-01-06 01:38:55 backup/weekly/
-rw-r--r-- root/root  55393885 2003-01-06 01:16:21 backup/weekly/var-spool-mail.tar.gz
-rw-r--r-- root/root      6064 2003-01-06 01:38:56 backup/weekly/backup-config.tar.gz
-rw-r--r-- root/root 371547575 2003-01-06 01:36:28 backup/weekly/pictures.tar.gz
-rw-r--r-- root/root     25289 2003-01-06 01:32:44 backup/weekly/brennen.txt
-rw-r--r-- root/root    206933 2003-01-06 01:15:44 backup/weekly/cvs.txt
-rw-r--r-- root/root   2994894 2003-01-06 01:30:32 backup/weekly/httpd.txt
-rw-r--r-- root/root  17755029 2003-01-06 01:15:56 backup/weekly/root.tar.gz
-rw-r--r-- root/root        10 2003-01-06 01:15:56 backup/weekly/var-mail.txt
-rw-r--r-- root/root   4152070 2003-01-06 01:15:48 backup/weekly/etc.tar.gz
-rw-r--r-- root/root     17463 2003-01-06 01:38:55 backup/weekly/temp.txt
-rw-r--r-- root/root 120880513 2003-01-06 01:33:57 backup/weekly/linux-app.tar.gz
-rw-r--r-- root/root       116 2003-01-06 01:15:56 backup/weekly/var-mail.tar.gz
-rw-r--r-- root/root     42447 2003-01-06 01:15:48 backup/weekly/etc.txt
-rw-r--r-- root/root   1688621 2003-01-06 01:25:17 backup/weekly/home.txt
-rw-r--r-- root/root 287873726 2003-01-06 01:30:32 backup/weekly/httpd.tar.gz
-rw-r--r-- root/root     36640 2003-01-06 01:28:43 backup/weekly/sfirm.txt
-rw-r--r-- root/root    313851 2003-01-06 01:32:21 backup/weekly/develop.txt
-rw-r--r-- root/root      2334 2003-01-06 01:16:20 backup/weekly/var-spool-mail.txt
-rw-r--r-- root/root    296742 2003-01-06 01:33:55 backup/weekly/linux-app.txt
-rw-r--r-- root/root       531 2003-01-06 01:32:44 backup/weekly/domain-logon.tar.gz
-rw-r--r-- root/root    115462 2003-01-06 01:15:56 backup/weekly/root.txt
-rw-r--r-- root/root  52210746 2003-01-06 01:15:44 backup/weekly/cvs.tar.gz
-rw-r--r-- root/root 219765848 2003-01-06 01:32:21 backup/weekly/develop.tar.gz
-rw-r--r-- root/root       669 2003-01-06 01:38:56 backup/weekly/backup-config.txt
-rw-r--r-- root/root  67020269 2003-01-06 01:38:55 backup/weekly/temp.tar.gz
-rw-r--r-- root/root 123907364 2003-01-06 01:38:32 backup/weekly/public.tar.gz
-rw-r--r-- root/root  14223792 2003-01-06 01:28:43 backup/weekly/sfirm.tar.gz
-rw-r--r-- root/root    295998 2003-01-06 01:16:22 backup/weekly/opt.tar.gz
-rw-r--r-- root/root 1428360286 2003-01-06 01:25:17 backup/weekly/home.tar.gz
-rw-r--r-- root/root    2375000 2003-01-06 01:36:28 backup/weekly/pictures.txt
-rw-r--r-- root/root        244 2003-01-06 01:32:44 backup/weekly/domain-logon.txt
-rw-r--r-- root/root       2337 2003-01-06 01:16:22 backup/weekly/opt.txt
-rw-r--r-- root/root    2328283 2003-01-06 01:15:08 backup/weekly/mysql.tar.gz
-rw-r--r-- root/root       5531 2003-01-06 01:15:08 backup/weekly/mysql.txt
-rw-r--r-- root/root      10944 2003-01-06 01:38:32 backup/weekly/public.txt
-rw-r--r-- root/root   63084407 2003-01-06 01:32:44 backup/weekly/brennen.tar.gz
Mon Jan  6 05:19:00 CET 2003


# Status des letzten RPM- Updates Backups: 
> cat /var/log/backup/rpm-updates
Backup of /backup/rpm-updates:
Mon Jan  6 09:25:09 CET 2003
total 4749437
drwxr-xr-x    2 root     root          368 Jan  6 09:22 .
drwxr-xr-x   11 root     root          296 Dec 15 21:03 ..
-rw-r--r--    1 root     root     1170575360 Jan  6 09:22 rh73-updates.tar
-rw-r--r--    1 root     root        30835 Jan  6 09:22 rh73-updates.txt
-rw-r--r--    1 root     root     596387840 Jan  6 09:25 rh90-updates.tar
-rw-r--r--    1 root     root        10498 Jan  6 09:25 rh90-updates.txt

Linux Als Server

Incrementelle Sicherung

Übersicht:

Downloads:

Aufgabe

Verzeichnisse

Pages