DokuWiki » Infrastructure » Operating System » CentOS 6

h1. Setup rsnapshot

h2. Requirements

* [[repo_epel|epel]]

h2. Preliminary Note

I want to say first that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take.

You may also check: http://wiki.centos.org/HowTos/RsnapshotBackups

h2. install rsnapshot

<pre><code class="bash">

yum install rsnapshot

</code></pre>

h2. config /etc/rnsapshot.conf

<pre><code class="bash">

vim /etc/rsnapshot.conf

</code></pre>

* set @snapshot_root /var/cache/rsnapshot/@

* if apropriate uncomment @#no_create_root	1@, useful when @snapshot_root@ points to removeable destination (usb or network)

* only other section to touch is: @Backup Points / Scripts@

<pre>

#################################################

# rsnapshot.conf - rsnapshot configuration file #

#################################################

#                                               #

# PLEASE BE AWARE OF THE FOLLOWING RULES:       #

#                                               #

# This file requires tabs between elements      #

#                                               #

# Directories require a trailing slash:         #

#   right: /home/                               #

#   wrong: /home                                #

#                                               #

#################################################

#######################

# CONFIG FILE VERSION #

#######################

config_version	1.2

###########################

# SNAPSHOT ROOT DIRECTORY #

###########################

# All snapshots will be stored under this root directory.

#

snapshot_root	/var/cache/rsnapshot/

# If no_create_root is enabled, rsnapshot will not automatically create the

# snapshot_root directory. This is particularly useful if you are backing

# up to removable media, such as a FireWire or USB drive.

#

#no_create_root	1

#################################

# EXTERNAL PROGRAM DEPENDENCIES #

#################################

# LINUX USERS:   Be sure to uncomment "cmd_cp". This gives you extra features.

# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.

#

# See the README file or the man page for more details.

#

cmd_cp		/bin/cp

# uncomment this to use the rm program instead of the built-in perl routine.

#

cmd_rm		/bin/rm

# rsync must be enabled for anything to work. This is the only command that

# must be enabled.

#

cmd_rsync	/usr/bin/rsync

# Uncomment this to enable remote ssh backups over rsync.

#

#cmd_ssh	/usr/bin/ssh

# Comment this out to disable syslog support.

#

cmd_logger	/usr/bin/logger

# Uncomment this to specify the path to "du" for disk usage checks.

# If you have an older version of "du", you may also want to check the

# "du_args" parameter below.

#

cmd_du		/usr/bin/du

# Uncomment this to specify the path to rsnapshot-diff.

#

#cmd_rsnapshot_diff	/usr/local/bin/rsnapshot-diff

# Specify the path to a script (and any optional arguments) to run right

# before rsnapshot syncs files

#

#cmd_preexec	/path/to/preexec/script

# Specify the path to a script (and any optional arguments) to run right

# after rsnapshot syncs files

#

#cmd_postexec	/path/to/postexec/script

#########################################

#           BACKUP INTERVALS            #

# Must be unique and in ascending order #

# i.e. hourly, daily, weekly, etc.      #

#########################################

interval	hourly	24

interval	daily	7

interval	weekly	4

#interval	monthly	3

############################################

#              GLOBAL OPTIONS              #

# All are optional, with sensible defaults #

############################################

# Verbose level, 1 through 5.

# 1     Quiet           Print fatal errors only

# 2     Default         Print errors and warnings only

# 3     Verbose         Show equivalent shell commands being executed

# 4     Extra Verbose   Show extra verbose information

# 5     Debug mode      Everything

#

verbose		2

# Same as "verbose" above, but controls the amount of data sent to the

# logfile, if one is being used. The default is 3.

#

loglevel	4

# If you enable this, data will be written to the file you specify. The

# amount of data written is controlled by the "loglevel" parameter.

#

logfile	/var/log/rsnapshot

# If enabled, rsnapshot will write a lockfile to prevent two instances

# from running simultaneously (and messing up the snapshot_root).

# If you enable this, make sure the lockfile directory is not world

# writable. Otherwise anyone can prevent the program from running.

#

lockfile	/var/run/rsnapshot.pid

# Default rsync args. All rsync commands have at least these options set.

#

#rsync_short_args	-a

#rsync_long_args	--delete --numeric-ids --relative --delete-excluded

# ssh has no args passed by default, but you can specify some here.

#

#ssh_args	-p 22

# Default arguments for the "du" program (for disk space reporting).

# The GNU version of "du" is preferred. See the man page for more details.

# If your version of "du" doesn't support the -h flag, try -k flag instead.

#

#du_args	-csh

# If this is enabled, rsync won't span filesystem partitions within a

# backup point. This essentially passes the -x option to rsync.

# The default is 0 (off).

#

#one_fs		0

# The include and exclude parameters, if enabled, simply get passed directly

# to rsync. If you have multiple include/exclude patterns, put each one on a

# separate line. Please look up the --include and --exclude options in the

# rsync man page for more details on how to specify file name patterns. 

# 

#include	???

#include	???

#exclude	???

#exclude	???

# The include_file and exclude_file parameters, if enabled, simply get

# passed directly to rsync. Please look up the --include-from and

# --exclude-from options in the rsync man page for more details.

#

#include_file	/path/to/include/file

#exclude_file	/path/to/exclude/file

# If your version of rsync supports --link-dest, consider enable this.

# This is the best way to support special files (FIFOs, etc) cross-platform.

# The default is 0 (off).

#

#link_dest	0

# When sync_first is enabled, it changes the default behaviour of rsnapshot.

# Normally, when rsnapshot is called with its lowest interval

# (i.e.: "rsnapshot hourly"), it will sync files AND rotate the lowest

# intervals. With sync_first enabled, "rsnapshot sync" handles the file sync,

# and all interval calls simply rotate files. See the man page for more

# details. The default is 0 (off).

#

#sync_first	0

# If enabled, rsnapshot will move the oldest directory for each interval

# to [interval_name].delete, then it will remove the lockfile and delete

# that directory just before it exits. The default is 0 (off).

#

#use_lazy_deletes	0

# Number of rsync re-tries. If you experience any network problems or

# network card issues that tend to cause ssh to crap-out with

# "Corrupted MAC on input" errors, for example, set this to a non-zero

# value to have the rsync operation re-tried

#

#rsync_numtries 0

###############################

### BACKUP POINTS / SCRIPTS ###

###############################

# LOCALHOST

#backup	/home/		localhost/

backup	/etc/		localhost/

#backup	/usr/local/	localhost/

#backup	/var/log/rsnapshot		localhost/

#backup	/etc/passwd	localhost/

#backup	/home/foo/My Documents/		localhost/

#backup	/foo/bar/	localhost/	one_fs=1, rsync_short_args=-urltvpog

backup_script	/usr/local/bin/backup_pgsql.sh	localhost/postgres/

# EXAMPLE.COM

#backup_script	/bin/date "+ backup of example.com started at %c"	unused1

#backup	root@example.com:/home/	example.com/	+rsync_long_args=--bwlimit=16,exclude=core

#backup	root@example.com:/etc/	example.com/	exclude=mtab,exclude=core

#backup_script	ssh root@example.com "mysqldump -A > /var/db/dump/mysql.sql"	unused2

#backup	root@example.com:/var/db/dump/	example.com/

#backup_script	/bin/date	"+ backup of example.com ended at %c"	unused9

# CVS.SOURCEFORGE.NET

#backup_script	/usr/local/bin/backup_rsnapshot_cvsroot.sh	rsnapshot.cvs.sourceforge.net/

# RSYNC.SAMBA.ORG

#backup	rsync://rsync.samba.org/rsyncftp/	rsync.samba.org/rsyncftp/

</pre>

After doing changes to @rsnapshot.conf@ you maybe want to check if the config is still valid by

<pre><code class="bash">

rsnapshot configtest 

</code></pre>

h2. add /etc/cron.d/rsnapshot

<pre><code class="bash">

vim /etc/cron.d/rsnapshot

</code></pre>

basic example:

<pre>

# JKE 2011-05-01

#

# minute hour day month dayofweek

# +---------min (0-59)

# | +-------hour (0-23)

# | | +-----day of month (1-31)

# | | | +---month (1-12)

# | | | | +-day of week (0-7) sunday=0 or 7

# | | | | |

# - - - - -

# m h d m d

# 

	0	*	*	*	*	root	/usr/bin/rsnapshot hourly

	55	11	*	*	*	root	/usr/bin/rsnapshot daily

	50	11	*	*	5	root	/usr/bin/rsnapshot weekly

#		root	/usr/bin/rsnapshot monthly

</pre>

h2. add /usr/local/bin/backup_pgsql.sh

<pre><code class="bash">

vim /usr/local/bin/backup_pgsql.sh

</code></pre>

basic example:

<pre>

##############################################################################

# backup_pgsql.sh

#

# by Nathan Rosenquist <nathan@rsnapshot.org>

# http://www.rsnapshot.org/

#

# This is a simple shell script to backup a PostgreSQL database with rsnapshot.

#

# The assumption is that this will be invoked from rsnapshot. Also, since it

# will run unattended, the user that runs rsnapshot (probably root) should have

# a .pgpass file in their home directory that contains the password for the

# postgres user. For example:

#

# /root/.pgpass (chmod 0600)

# *:*:*:postgres:thepassword

#

# This script simply needs to dump a file into the current working directory.

# rsnapshot handles everything else.

##############################################################################

# $Id: backup_pgsql.sh,v 1.6 2007/03/22 02:50:21 drhyde Exp $

umask 0077

# backup the database

#sudo -u postgres /usr/bin/pg_dumpall -Upostgres > pg_dumpall.sql

/usr/bin/pg_dumpall -Upostgres > pg_dumpall.sql

cat /var/lib/pgsql/9.2/data/pg_hba.conf > pg_hba.conf

cat /var/lib/pgsql/9.2/data/pg_ident.conf > pg_ident.conf

cat /var/lib/pgsql/9.2/data/postgresql.conf > postgresql.conf

#/usr/bin/pg_dumpall -Upeter > pg_dumpall.sql

#/usr/bin/pg_dumpall -Upostgres > pg_dumpall.sql

#/usr/bin/pg_dumpall -h localhost -p 5432 -U janus_admin -v > pg_dumpall.sql

# make the backup readable only by root

/bin/chmod 600 pg_dumpall.sql

/bin/chmod 600 pg_hba.conf

/bin/chmod 600 pg_ident.conf

/bin/chmod 600 postgresql.conf

</pre>

set shell-script being executeable

<pre><code class="bash">

chmod +x /usr/local/bin/backup_pgsql.sh

</code></pre>

h2. Check rsnapshot

goto @/var/cache/rsnapshot/...@ and check if the desired content is properly backed up.

h2. Report rsnapshot

If you're into noise, then rsnapshot has a nifty little reporting script that's easily setup. This will send you a small e-mail with a few details as to what occurred during the backup. It's a very simple report. First, copy the script over to somewhere such as /usr/local/bin and make it executable.

<pre><code class="bash">

cp /usr/share/doc/rsnapshot-1.3.1/utils/rsnapreport.pl /usr/local/bin

chmod +x /usr/local/bin/rsnapreport.pl

</code></pre>

Next, add --stats to the rsync's long arguments section in your host rsnapshot file. Remember the spaces are not spaces but tabs.

<pre><code class="bash">

vim /etc/rsnapshot.conf

</code></pre>

<pre>

rsync_long_args --stats --delete        --numeric-ids   --delete-excluded

</pre>

Last, edit the crontab entries that were setup earlier to pass the results of the rsnapshot run through the rsnapreport.pl script. 

<pre><code class="bash">

crontab -e

</code></pre>

<pre>

#MAILTO=""

##########################################################

#minute (0-59),                                          #

#|  hour (0-23),                                         #

#|  |  day of the month (1-31),                          #

#|  |  |  month of the year (1-12),                      #

#|  |  |  |  day of the week (0-6 with 0=Sunday)         #

#|  |  |  |  |       commands                            #

##########################################################

15 02***     /usr/bin/rsnapshot -c /etc/rsnapshot/laptop.rsnapshot.conf daily 2&>1 | \

/usr/local/bin/rsnapreport.pl | mail -s "laptop daily" me@myemail.com

15 03**  Sun   /usr/bin/rsnapshot -c /etc/rsnapshot/laptop.rsnapshot.conf weekly 2&>1 | \

/usr/local/bin/rsnapreport.pl | mail -s "laptop weekly" me@myemail.com

15 04  1**     /usr/bin/rsnapshot -c /etc/rsnapshot/laptop.rsnapshot.conf monthly 2&>1 | \

/usr/local/bin/rsnapreport.pl | mail -s "laptop monthly" me@myemail.com

</pre>

Don't forget the redirection of 2>&1 (standard errors), or they will be lost to stderr. This means redirect standard errors to the same place as standard output. That's it for reporting. You should now get a report to your e-mail address the next rsnapshot run. It will look like this output: 

<pre><code class="bash">

SOURCE           TOTAL FILES   FILES TRANS      TOTAL MB     MB TRANS   LIST GEN TIME  FILE XFER TIME

-----------------------------------------------------------------------------------------------------

laptop:/              59076          1353      17279.45      7169.38   20.361 second   0.000 seconds

</code></pre>

h2. Logwatch integration

this is basically taken from [[http://benjaminjchapman.wordpress.com/2011/03/02/customizing-logwatch-to-get-error-messages-from-rsnapshot/]]

By default, RSnapshot generates a single logfile at “@/var/log/rsnapshot@”. We want to parse this file and make sure that logwatch notifies us if there are any lines in the file that contain the word “ERROR:”, as we know from looking at the log file that RSnapshot flags all errors this way:

  ERROR: Errors were found in /etc/rsnapshot.conf,

  ERROR: rsnapshot can not continue. If you think an entry looks right, make

  ERROR: sure you don’t have spaces where only tabs should be.

To tackle this problem, you will create three new files:

  - /etc/logwatch/conf/logfiles/rsnapshot.conf

  - /etc/logwatch/conf/services/rsnapshot.conf

  - /etc/logwatch/scripts/services/rsnapshot

h2. Create a new rsnaphot.conf under/etc/logwatch/conf/logfiles/

The contents should look something like this. Basically, you’re telling Logwatch to watch /var/log/rsnapshot and you’re telling it to use some built-in date routines.

<pre><code class="bash">

vim /etc/logwatch/conf/logfiles/rsnapshot.conf

</code></pre>

<pre>

# First cut at monitoring rsnapshot errors

# rsnapshot log file is /var/log/rsnapshot

# There is no archive file

#

LogFile = rsnapshot

*ApplyHttpDate

</pre>

h3. Create a new rsnapshot.conf in /etc/logwatch/conf/services/

Again, a two-line file. This tells logwatch to look at the log group that you defined in @/etc/logwatch/conf/logfiles@ and it tells it to use “RSNAPSHOT” as the title when it sends you reports.

<pre><code class="bash">

vim /etc/logwatch/conf/services/rsnapshot.conf

</code></pre>

<pre>

LogFile = rsnapshot

Title = "RSNAPSHOT"

</pre>

h3. Create the actual program that will search through the logfile looking for information to send you

This file is @/etc/logwatch/scripts/services/rsnapshot@ and it’s an actual script that takes the logfiles identified in the configuration files above and searches through the file looking for matching patterns. The script can be written in any language. I copied another script, which happened to be written in Perl. Here’s my very basic script:

<pre><code class="bash">

vim /etc/logwatch/scripts/services/rsnapshot

</code></pre>

<pre>

#!/usr/bin/perl

##########################################################################

# $Id: rsnapshot Wed Mar  2 09:10:43 EST 2011

##########################################################################

########################################################

# Logwatch was written and is maintained by:

#    Kirk Bauer

########################################################

my $Debug = $ENV{'LOGWATCH_DEBUG'};

my $Detail = $ENV{'LOGWATCH_DETAIL_LEVEL'};

my $Error = 0;

my $Warning = 0;

my @ErrorLines = ();

my @WarningLines = ();

if ( $Debug >= 5 ) {

  print STDERR "nnDEBUG nn";

}

while (defined($ThisLine = <STDIN> )) {

  if ($ThisLine =~ /ERROR: /) {

     @ErrorLines[$Error] = $ThisLine;

     $Error++;

  }

  if ($ThisLine =~ /WARNING: /) {

     @WarningLines[$Warning] = $ThisLine;

     $Warning++;

  }

}

################################################

if ($Error > 0) {

  print "WARNING: There are a total of $Error RSnapshot error(s).\n";

  print "Please review immediately!\n";

}

if ($Warning > 0) {

  print "WARNING: There are a total of $Warning RSnapshot warning(s).\n";

  print "Please review immediately!\n";

}

if ( ( $Detail >= 0 ) and (scalar(@ErrorLines) > 0) ) {

  print     "\n[RSNAPSHOT] Errors:".

            "\n=========================\n";

  for ($count = 0; $count <= scalar(@ErrorLines); $count++) {

    print "@ErrorLines[$count]";

  }

}

if ( ( $Detail >= 0 ) and (scalar(@WarningLines) > 0) ) {

  print     "\n[RSNAPSHOT] Warnings:".

            "\n=========================\n";

  for ($count = 0; $count <= scalar(@WarningLines); $count++) {

    print "@WarningLines[$count]";

  }

}

exit(0);

</pre>