Projekt

Allgemein

Profil

Setup rsnapshot » Historie » Version 1

Jeremias Keihsler, 06.02.2019 09:28

1 1 Jeremias Keihsler
h1. Setup rsnapshot
2
3
h2. Preliminary note
4
5
I want to say first that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take.
6
7
You may also check: http://wiki.centos.org/HowTos/RsnapshotBackups
8
9
h2. install rsnapshot
10
11
<pre><code class="bash">
12
yum install rsnapshot
13
</code></pre>
14
15
h3. config /etc/rnsapshot.conf
16
17
<pre><code class="bash">
18
vim /etc/rsnapshot.conf
19
</code></pre>
20
21
* set @snapshot_root /var/cache/rsnapshot/@
22
* if appropriate uncomment @#no_create_root	1@, useful when @snapshot_root@ points to removeable destination (usb or network)
23
* only other section to touch is: @Backup Points / Scripts@
24
<pre>
25
#################################################
26
# rsnapshot.conf - rsnapshot configuration file #
27
#################################################
28
#                                               #
29
# PLEASE BE AWARE OF THE FOLLOWING RULES:       #
30
#                                               #
31
# This file requires tabs between elements      #
32
#                                               #
33
# Directories require a trailing slash:         #
34
#   right: /home/                               #
35
#   wrong: /home                                #
36
#                                               #
37
#################################################
38
39
#######################
40
# CONFIG FILE VERSION #
41
#######################
42
43
config_version	1.2
44
45
###########################
46
# SNAPSHOT ROOT DIRECTORY #
47
###########################
48
49
# All snapshots will be stored under this root directory.
50
#
51
snapshot_root	/var/cache/rsnapshot/
52
53
# If no_create_root is enabled, rsnapshot will not automatically create the
54
# snapshot_root directory. This is particularly useful if you are backing
55
# up to removable media, such as a FireWire or USB drive.
56
#
57
#no_create_root	1
58
59
#################################
60
# EXTERNAL PROGRAM DEPENDENCIES #
61
#################################
62
63
# LINUX USERS:   Be sure to uncomment "cmd_cp". This gives you extra features.
64
# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.
65
#
66
# See the README file or the man page for more details.
67
#
68
cmd_cp		/bin/cp
69
70
# uncomment this to use the rm program instead of the built-in perl routine.
71
#
72
cmd_rm		/bin/rm
73
74
# rsync must be enabled for anything to work. This is the only command that
75
# must be enabled.
76
#
77
cmd_rsync	/usr/bin/rsync
78
79
# Uncomment this to enable remote ssh backups over rsync.
80
#
81
#cmd_ssh	/usr/bin/ssh
82
83
# Comment this out to disable syslog support.
84
#
85
cmd_logger	/usr/bin/logger
86
87
# Uncomment this to specify the path to "du" for disk usage checks.
88
# If you have an older version of "du", you may also want to check the
89
# "du_args" parameter below.
90
#
91
cmd_du		/usr/bin/du
92
93
# Uncomment this to specify the path to rsnapshot-diff.
94
#
95
#cmd_rsnapshot_diff	/usr/local/bin/rsnapshot-diff
96
97
# Specify the path to a script (and any optional arguments) to run right
98
# before rsnapshot syncs files
99
#
100
#cmd_preexec	/path/to/preexec/script
101
102
# Specify the path to a script (and any optional arguments) to run right
103
# after rsnapshot syncs files
104
#
105
#cmd_postexec	/path/to/postexec/script
106
107
#########################################
108
#           BACKUP INTERVALS            #
109
# Must be unique and in ascending order #
110
# i.e. hourly, daily, weekly, etc.      #
111
#########################################
112
113
interval	hourly	24
114
interval	daily	7
115
interval	weekly	4
116
#interval	monthly	3
117
118
############################################
119
#              GLOBAL OPTIONS              #
120
# All are optional, with sensible defaults #
121
############################################
122
123
# Verbose level, 1 through 5.
124
# 1     Quiet           Print fatal errors only
125
# 2     Default         Print errors and warnings only
126
# 3     Verbose         Show equivalent shell commands being executed
127
# 4     Extra Verbose   Show extra verbose information
128
# 5     Debug mode      Everything
129
#
130
verbose		2
131
132
# Same as "verbose" above, but controls the amount of data sent to the
133
# logfile, if one is being used. The default is 3.
134
#
135
loglevel	4
136
137
# If you enable this, data will be written to the file you specify. The
138
# amount of data written is controlled by the "loglevel" parameter.
139
#
140
logfile	/var/log/rsnapshot
141
142
# If enabled, rsnapshot will write a lockfile to prevent two instances
143
# from running simultaneously (and messing up the snapshot_root).
144
# If you enable this, make sure the lockfile directory is not world
145
# writable. Otherwise anyone can prevent the program from running.
146
#
147
lockfile	/var/run/rsnapshot.pid
148
149
# Default rsync args. All rsync commands have at least these options set.
150
#
151
#rsync_short_args	-a
152
#rsync_long_args	--delete --numeric-ids --relative --delete-excluded
153
154
# ssh has no args passed by default, but you can specify some here.
155
#
156
#ssh_args	-p 22
157
158
# Default arguments for the "du" program (for disk space reporting).
159
# The GNU version of "du" is preferred. See the man page for more details.
160
# If your version of "du" doesn't support the -h flag, try -k flag instead.
161
#
162
#du_args	-csh
163
164
# If this is enabled, rsync won't span filesystem partitions within a
165
# backup point. This essentially passes the -x option to rsync.
166
# The default is 0 (off).
167
#
168
#one_fs		0
169
170
# The include and exclude parameters, if enabled, simply get passed directly
171
# to rsync. If you have multiple include/exclude patterns, put each one on a
172
# separate line. Please look up the --include and --exclude options in the
173
# rsync man page for more details on how to specify file name patterns. 
174
# 
175
#include	???
176
#include	???
177
#exclude	???
178
#exclude	???
179
180
# The include_file and exclude_file parameters, if enabled, simply get
181
# passed directly to rsync. Please look up the --include-from and
182
# --exclude-from options in the rsync man page for more details.
183
#
184
#include_file	/path/to/include/file
185
#exclude_file	/path/to/exclude/file
186
187
# If your version of rsync supports --link-dest, consider enable this.
188
# This is the best way to support special files (FIFOs, etc) cross-platform.
189
# The default is 0 (off).
190
#
191
#link_dest	0
192
193
# When sync_first is enabled, it changes the default behaviour of rsnapshot.
194
# Normally, when rsnapshot is called with its lowest interval
195
# (i.e.: "rsnapshot hourly"), it will sync files AND rotate the lowest
196
# intervals. With sync_first enabled, "rsnapshot sync" handles the file sync,
197
# and all interval calls simply rotate files. See the man page for more
198
# details. The default is 0 (off).
199
#
200
#sync_first	0
201
202
# If enabled, rsnapshot will move the oldest directory for each interval
203
# to [interval_name].delete, then it will remove the lockfile and delete
204
# that directory just before it exits. The default is 0 (off).
205
#
206
#use_lazy_deletes	0
207
208
# Number of rsync re-tries. If you experience any network problems or
209
# network card issues that tend to cause ssh to crap-out with
210
# "Corrupted MAC on input" errors, for example, set this to a non-zero
211
# value to have the rsync operation re-tried
212
#
213
#rsync_numtries 0
214
215
###############################
216
### BACKUP POINTS / SCRIPTS ###
217
###############################
218
219
# LOCALHOST
220
#backup	/home/		localhost/
221
backup	/etc/		localhost/
222
#backup	/usr/local/	localhost/
223
#backup	/var/log/rsnapshot		localhost/
224
#backup	/etc/passwd	localhost/
225
#backup	/home/foo/My Documents/		localhost/
226
#backup	/foo/bar/	localhost/	one_fs=1, rsync_short_args=-urltvpog
227
backup_script	/usr/local/bin/backup_pgsql.sh	localhost/postgres/
228
229
# EXAMPLE.COM
230
#backup_script	/bin/date "+ backup of example.com started at %c"	unused1
231
#backup	root@example.com:/home/	example.com/	+rsync_long_args=--bwlimit=16,exclude=core
232
#backup	root@example.com:/etc/	example.com/	exclude=mtab,exclude=core
233
#backup_script	ssh root@example.com "mysqldump -A > /var/db/dump/mysql.sql"	unused2
234
#backup	root@example.com:/var/db/dump/	example.com/
235
#backup_script	/bin/date	"+ backup of example.com ended at %c"	unused9
236
237
# CVS.SOURCEFORGE.NET
238
#backup_script	/usr/local/bin/backup_rsnapshot_cvsroot.sh	rsnapshot.cvs.sourceforge.net/
239
240
# RSYNC.SAMBA.ORG
241
#backup	rsync://rsync.samba.org/rsyncftp/	rsync.samba.org/rsyncftp/
242
</pre>
243
After doing changes to @rsnapshot.conf@ you maybe want to check if the config is still valid by
244
<pre><code class="bash">
245
rsnapshot configtest 
246
</code></pre>
247
248
h3. add /etc/cron.d/rsnapshot
249
250
<pre><code class="bash">
251
vim /etc/cron.d/rsnapshot
252
</code></pre>
253
254
basic example:
255
<pre>
256
# JKE 2017-03-17
257
#
258
# minute hour day month dayofweek
259
# +---------min (0-59)
260
# | +-------hour (0-23)
261
# | | +-----day of month (1-31)
262
# | | | +---month (1-12)
263
# | | | | +-day of week (0-7) sunday=0 or 7
264
# | | | | |
265
# - - - - -
266
# m h d m d
267
# 
268
	0	*	*	*	*	root	/usr/bin/rsnapshot alpha
269
	55	11	*	*	*	root	/usr/bin/rsnapshot beta
270
	50	11	*	*	5	root	/usr/bin/rsnapshot gamma
271
#		root	/usr/bin/rsnapshot monthly
272
</pre>
273
274
h3. add /usr/local/bin/backup_pgsql.sh
275
276
<pre><code class="bash">
277
vim /usr/local/bin/backup_pgsql.sh
278
</code></pre>
279
280
basic example:
281
<pre>
282
##############################################################################
283
# backup_pgsql.sh
284
#
285
# by Nathan Rosenquist <nathan@rsnapshot.org>
286
# http://www.rsnapshot.org/
287
#
288
# This is a simple shell script to backup a PostgreSQL database with rsnapshot.
289
#
290
# The assumption is that this will be invoked from rsnapshot. Also, since it
291
# will run unattended, the user that runs rsnapshot (probably root) should have
292
# a .pgpass file in their home directory that contains the password for the
293
# postgres user. For example:
294
#
295
# /root/.pgpass (chmod 0600)
296
# *:*:*:postgres:thepassword
297
#
298
# This script simply needs to dump a file into the current working directory.
299
# rsnapshot handles everything else.
300
##############################################################################
301
302
# $Id: backup_pgsql.sh,v 1.6 2007/03/22 02:50:21 drhyde Exp $
303
304
umask 0077
305
306
# backup the database
307
#sudo -u postgres /usr/bin/pg_dumpall -Upostgres > pg_dumpall.sql
308
/usr/bin/pg_dumpall -Upostgres > pg_dumpall.sql
309
cat /var/lib/pgsql/9.2/data/pg_hba.conf > pg_hba.conf
310
cat /var/lib/pgsql/9.2/data/pg_ident.conf > pg_ident.conf
311
cat /var/lib/pgsql/9.2/data/postgresql.conf > postgresql.conf
312
313
#/usr/bin/pg_dumpall -Upeter > pg_dumpall.sql
314
#/usr/bin/pg_dumpall -Upostgres > pg_dumpall.sql
315
#/usr/bin/pg_dumpall -h localhost -p 5432 -U janus_admin -v > pg_dumpall.sql
316
317
# make the backup readable only by root
318
/bin/chmod 600 pg_dumpall.sql
319
/bin/chmod 600 pg_hba.conf
320
/bin/chmod 600 pg_ident.conf
321
/bin/chmod 600 postgresql.conf
322
</pre>
323
324
set shell-script being executeable
325
<pre><code class="bash">
326
chmod +x /usr/local/bin/backup_pgsql.sh
327
</code></pre>
328
329
h2. Check rsnapshot
330
331
goto @/var/cache/rsnapshot/...@ and check if the desired content is properly backed up.
332
333
h2. Report rsnapshot
334
335
If you're into noise, then rsnapshot has a nifty little reporting script that's easily setup. This will send you a small e-mail with a few details as to what occurred during the backup. It's a very simple report. First, copy the script over to somewhere such as /usr/local/bin and make it executable.
336
337
<pre><code class="bash">
338
cp /usr/share/doc/rsnapshot-1.3.1/utils/rsnapreport.pl /usr/local/bin
339
chmod +x /usr/local/bin/rsnapreport.pl
340
</code></pre>
341
342
Next, add --stats to the rsync's long arguments section in your host rsnapshot file. Remember the spaces are not spaces but tabs.
343
344
<pre><code class="bash">
345
vim /etc/rsnapshot.conf
346
</code></pre>
347
<pre>
348
rsync_long_args --stats --delete        --numeric-ids   --delete-excluded
349
</pre>
350
351
Last, edit the crontab entries that were setup earlier to pass the results of the rsnapshot run through the rsnapreport.pl script. 
352
353
<pre><code class="bash">
354
crontab -e
355
</code></pre>
356
357
<pre>
358
#MAILTO=""
359
##########################################################
360
#minute (0-59),                                          #
361
#|  hour (0-23),                                         #
362
#|  |  day of the month (1-31),                          #
363
#|  |  |  month of the year (1-12),                      #
364
#|  |  |  |  day of the week (0-6 with 0=Sunday)         #
365
#|  |  |  |  |       commands                            #
366
##########################################################
367
15 02***     /usr/bin/rsnapshot -c /etc/rsnapshot/laptop.rsnapshot.conf daily 2&>1 | \
368
/usr/local/bin/rsnapreport.pl | mail -s "laptop daily" me@myemail.com
369
15 03**  Sun   /usr/bin/rsnapshot -c /etc/rsnapshot/laptop.rsnapshot.conf weekly 2&>1 | \
370
/usr/local/bin/rsnapreport.pl | mail -s "laptop weekly" me@myemail.com
371
15 04  1**     /usr/bin/rsnapshot -c /etc/rsnapshot/laptop.rsnapshot.conf monthly 2&>1 | \
372
/usr/local/bin/rsnapreport.pl | mail -s "laptop monthly" me@myemail.com
373
</pre>
374
375
Don't forget the redirection of 2>&1 (standard errors), or they will be lost to stderr. This means redirect standard errors to the same place as standard output. That's it for reporting. You should now get a report to your e-mail address the next rsnapshot run. It will look like this output: 
376
377
<pre><code class="bash">
378
SOURCE           TOTAL FILES   FILES TRANS      TOTAL MB     MB TRANS   LIST GEN TIME  FILE XFER TIME
379
-----------------------------------------------------------------------------------------------------
380
laptop:/              59076          1353      17279.45      7169.38   20.361 second   0.000 seconds
381
</code></pre>
382
383
h1. Logwatch integration
384
385
this is basically taken from [[http://benjaminjchapman.wordpress.com/2011/03/02/customizing-logwatch-to-get-error-messages-from-rsnapshot/]]
386
387
By default, RSnapshot generates a single logfile at “@/var/log/rsnapshot@”. We want to parse this file and make sure that logwatch notifies us if there are any lines in the file that contain the word “ERROR:”, as we know from looking at the log file that RSnapshot flags all errors this way:
388
389
  ERROR: Errors were found in /etc/rsnapshot.conf,
390
  ERROR: rsnapshot can not continue. If you think an entry looks right, make
391
  ERROR: sure you don’t have spaces where only tabs should be.
392
393
To tackle this problem, you will create three new files:
394
395
* /etc/logwatch/conf/logfiles/rsnapshot.conf
396
* /etc/logwatch/conf/services/rsnapshot.conf
397
* /etc/logwatch/scripts/services/rsnapshot
398
399
h3. Create a new rsnaphot.conf under/etc/logwatch/conf/logfiles/
400
401
The contents should look something like this. Basically, you’re telling Logwatch to watch /var/log/rsnapshot and you’re telling it to use some built-in date routines.
402
403
<pre><code class="bash">
404
vim /etc/logwatch/conf/logfiles/rsnapshot.conf
405
</code></pre>
406
407
<pre>
408
# First cut at monitoring rsnapshot errors
409
# rsnapshot log file is /var/log/rsnapshot
410
# There is no archive file
411
#
412
LogFile = rsnapshot
413
*ApplyHttpDate
414
</pre>
415
416
h3. Create a new rsnapshot.conf in /etc/logwatch/conf/services/
417
418
Again, a two-line file. This tells logwatch to look at the log group that you defined in @/etc/logwatch/conf/logfiles@ and it tells it to use “RSNAPSHOT” as the title when it sends you reports.
419
420
<pre><code class="bash">
421
vim /etc/logwatch/conf/services/rsnapshot.conf
422
</code></pre>
423
424
<pre>
425
LogFile = rsnapshot
426
Title = "RSNAPSHOT"
427
</pre>
428
429
h3. Create the actual program that will search through the logfile looking for information to send you
430
431
This file is @/etc/logwatch/scripts/services/rsnapshot@ and it’s an actual script that takes the logfiles identified in the configuration files above and searches through the file looking for matching patterns. The script can be written in any language. I copied another script, which happened to be written in Perl. Here’s my very basic script:
432
433
<pre><code class="bash">
434
vim /etc/logwatch/scripts/services/rsnapshot
435
</code></pre>
436
437
<pre><code class="perl">
438
#!/usr/bin/perl
439
##########################################################################
440
# $Id: rsnapshot Wed Mar  2 09:10:43 EST 2011
441
##########################################################################
442
443
########################################################
444
# Logwatch was written and is maintained by:
445
#    Kirk Bauer
446
########################################################
447
448
my $Debug = $ENV{'LOGWATCH_DEBUG'};
449
my $Detail = $ENV{'LOGWATCH_DETAIL_LEVEL'};
450
my $Error = 0;
451
my $Warning = 0;
452
my @ErrorLines = ();
453
my @WarningLines = ();
454
455
if ( $Debug >= 5 ) {
456
  print STDERR "nnDEBUG nn";
457
}
458
459
while (defined($ThisLine = <STDIN> )) {
460
  if ($ThisLine =~ /ERROR: /) {
461
     @ErrorLines[$Error] = $ThisLine;
462
     $Error++;
463
  }
464
  if ($ThisLine =~ /WARNING: /) {
465
     @WarningLines[$Warning] = $ThisLine;
466
     $Warning++;
467
  }
468
}
469
470
################################################
471
472
if ($Error > 0) {
473
  print "WARNING: There are a total of $Error RSnapshot error(s).\n";
474
  print "Please review immediately!\n";
475
}
476
if ($Warning > 0) {
477
  print "WARNING: There are a total of $Warning RSnapshot warning(s).\n";
478
  print "Please review immediately!\n";
479
}
480
481
if ( ( $Detail >= 0 ) and (scalar(@ErrorLines) > 0) ) {
482
  print     "\n[RSNAPSHOT] Errors:".
483
            "\n=========================\n";
484
  for ($count = 0; $count <= scalar(@ErrorLines); $count++) {
485
    print "@ErrorLines[$count]";
486
  }
487
}
488
if ( ( $Detail >= 0 ) and (scalar(@WarningLines) > 0) ) {
489
  print     "\n[RSNAPSHOT] Warnings:".
490
            "\n=========================\n";
491
  for ($count = 0; $count <= scalar(@WarningLines); $count++) {
492
    print "@WarningLines[$count]";
493
  }
494
}
495
496
exit(0);
497
</code></pre>