Projekt

Allgemein

Profil

Setup zabbix-server22 » Historie » Version 1

Jeremias Keihsler, 13.01.2017 09:17

1 1 Jeremias Keihsler
h1. Install Zabbix Server 2.2.x
2
3
h2. preliminary note
4
5
this procedure is based on [[vbip:software:operatingsystem:centos6:setup:zabbix-server2.2|Install Zabbix Server 2.2.x]]
6
7
you may also check 
8
https://www.zabbix.com/documentation/2.2/manual/installation/install_from_packages
9
10
**this procedure has not yet been used or tested, use with caution**
11
remove this flag as soon as the first installation has succeeded.
12
13
* [[repo_epel|EPEL]]
14
* [[setup_apache| Apache]]
15
* [[setup_mariadb| MariaDB]]
16
17
h2. Server-Install
18
19
<pre><code class="bash">
20
yum update
21
yum install httpd httpd-devel
22
yum install zabbix22-server-mysql zabbix22-agent zabbix22-web-mysql
23
yum install mariadb-server
24
systemctl start httpd.service
25
systemctl enable httpd.service
26
systemctl start mariadb.service
27
systemctl enable mariadb.service
28
/usr/bin/mysql_secure_installation
29
mysql -u root -p
30
</code></pre>
31
<pre><code class="bash">
32
mysql> create database zabbix character set utf8 collate utf8_bin;
33
mysql> grant all privileges on zabbix.* to zabbix@localhost identified by 'password';
34
mysql> exit
35
</code></pre>
36
<pre><code class="bash">
37
cd /usr/share/zabbix-mysql/
38
mysql -u root -p zabbix < schema.sql
39
mysql -u root -p zabbix < images.sql
40
mysql -u root -p zabbix < data.sql
41
vim /etc/zabbix/zabbix_server.conf
42
</code></pre>
43
<pre>
44
# line 116: uncomment and add DB password for Zabbix
45
DBPassword=password
46
</pre>
47
<pre><code class="bash">
48
vim /etc/zabbix/zabbix_agentd.conf
49
</code></pre>
50
<pre>
51
# line 133: change to your hostname
52
Hostname=zabbix.server.com
53
</pre>
54
<pre><code class="bash">
55
vim /etc/php.ini
56
</code></pre>
57
<pre>
58
# line 384: change to Zabbix recommended
59
max_execution_time = 300
60
 
61
# line 394: change to Zabbix recommended
62
max_input_time = 300
63
 
64
# line 405: change to Zabbix recommended
65
memory_limit = 128M
66
 
67
# line 672: change to Zabbix recommended
68
post_max_size = 16M
69
 
70
# line 800: change to Zabbix recommended
71
upload_max_filesize = 2M
72
 
73
# line 879: uncomment and add your timezone
74
date.timezone = Europe/Vienna
75
</pre>
76
<pre><code class="bash">
77
systemctl start zabbix-server
78
systemctl enable zabbix-server-mysql
79
systemctl start zabbix-agent
80
systemctl enable zabbix-agent
81
systemctl restart httpd
82
</code></pre>
83
84
h3. Firewall
85
86
if you are going to monitor other systems than the @localhost@ then you have to open ports @10050@ and @10051@
87
88
<pre><code class="bash">
89
firewall-cmd --permanent --add-port=10050/tcp
90
firewall-cmd --permanent --add-port=10051/tcp
91
systemctl restart firewalld
92
</code></pre>
93
94
h3. SELinux
95
96
as SELinux prevents the web-client to establish a connection to @localhost:10050@ you will get a wrong indication on Server-Status. Allow access to the server via
97
<pre><code class="bash">
98
setsebool -P httpd_can_connect_zabbix 1
99
</code></pre>
100
101
h1. Backup Zabbix config
102
103
is taken from https://github.com/maxhq/zabbix-backup/
104
105
This script is a simple way to backup all configuration tables (eg. templates, hostgroups, hosts, triggers…) without the history data.
106
107
As the result is very small, is possible run this backup many times per day.
108
109
The backup-script may be invoked by cron
110
111
@/etc/cron.d/zabbix-backup@
112
<pre>
113
# JKE 2016-04-10
114
#
115
# minute hour day month dayofweek
116
# m h d m d
117
# - - - - -
118
# | | | | |
119
# | | | | +-day of week (0-7) sunday=0 or 7
120
# | | | +---month (1-12)
121
# | | +-----day of month (1-31)
122
# | +-------hour (0-23)
123
# +---------min (0-59)
124
# 
125
	5	1	*	*	*	root	/usr/local/bin/zabbix-mysql-backupconf.sh -p password -o /var/cache/myzabbixdump -r 5 -q
126
</pre>
127
128
@zabbix-mysql-backupconf.sh@
129
<pre>
130
#!/usr/bin/env bash
131
#
132
# NAME
133
#     zabbix-mysql-dump - Configuration Backup for Zabbix with MySQL
134
#
135
# SYNOPSIS
136
#     This is a MySQL configuration backup script for Zabbix 1.x and 2.x.
137
#     It does a full backup of all configuration tables, but only a schema
138
#     backup of large data tables.
139
#
140
#     The script is based on a script by Ricardo Santos
141
#     (http://zabbixzone.com/zabbix/backuping-only-the-zabbix-configuration/)
142
#
143
# CONTRIBUTORS
144
#      - Ricardo Santos
145
#      - Jens Berthold (maxhq)
146
#      - Oleksiy Zagorskyi (zalex)
147
#      - Petr Jendrejovsky
148
#      - Jonathan Bayer
149
#      - Andreas Niedermann (dre-)
150
#      - Mi?u Moldovan (dumol)
151
#      - Daniel Schneller (dschneller)
152
#
153
# HISTORY
154
#     0.8.0 (2016-01-22)
155
#         FIX: Only invoke `dig` if available
156
#         ENH: Option -c to use a MySQL config ("options") file (suggested by Daniel Schneller)
157
#         ENH: Option -r to rotate backup files (Daniel Schneller)
158
#         ENH: Add database version to filename if available
159
#         ENH: Add quiet mode. IP reverse lookup optional (Daniel Schneller)
160
#         ENH: Bash related fixes (Misu Moldovan)
161
#         CHG: Default output directory is now $PWD instead of script dir
162
#
163
#     0.7.1 (2015-01-27)
164
#         NEW: Parsing of commandline arguments implemented
165
#         ENH: Try reverse lookup of IPs and include hostname/IP in filename
166
#         REV: Stop if database password is wrong
167
# 
168
#     0.7.0 (2014-10-02)
169
#         ENH: Complete overhaul to make script work with lots of Zabbix versions
170
#
171
#     0.6.0 (2014-09-15)
172
#         REV: Updated the table list for use with zabbix v2.2.3
173
#
174
#     0.5.0 (2013-05-13)
175
#         NEW: Added table list comparison between database and script
176
#
177
#     0.4.0 (2012-03-02)
178
#         REV: Incorporated mysqldump options (suggested by Jonathan Bayer)
179
#
180
#     0.3.0 (2012-02-06)
181
#         ENH: Backup of Zabbix 1.9.x / 2.0.0, removed unnecessary use of
182
#              variables (DATEBIN etc) for commands that use to be in $PATH
183
#
184
#     0.2.0 (2011-11-05)
185
#
186
# AUTHOR
187
#     Jens Berthold (maxhq), 2016
188
189
190
#
191
# DEFAULT VALUES
192
# 
193
# DO NOT EDIT THESE VALUES!
194
# Instead, use command line parameters or a config file to specify options.
195
#
196
DUMPDIR="$PWD"
197
DBHOST="127.0.0.1"
198
DBNAME="zabbix"
199
DBUSER="zabbix"
200
DBPASS=""
201
QUIET="no"
202
REVERSELOOKUP="yes"
203
GENERATIONSTOKEEP=0
204
205
#
206
# SHOW HELP
207
#
208
if [ -z "$1" ]; then
209
	cat <<EOF
210
USAGE
211
	$(basename $BASH_SOURCE) [options]
212
	
213
OPTIONS
214
	-h HOST
215
		Hostname/IP of MySQL server.
216
		Default: $DBHOST
217
218
	-d DATABASE
219
		Zabbix database name.
220
		Default: $DBNAME
221
222
	-u USER
223
		MySQL user to access Zabbix database.
224
		Default: $DBUSER
225
226
	-p PASSWORD
227
		MySQL user password (specify "-" for a prompt).
228
		Default: no password
229
230
	-o DIR
231
		Save Zabbix MySQL dumps to DIR.
232
		Default: $DUMPDIR
233
234
	-c FILE
235
		Use FILE for MySQL options (passed via --defaults-extra-file).
236
		PLEASE NOTE:
237
		mysqldump needs the database to be specified via command line.
238
		So the first "database" options found in the config file is
239
		used for mysqldump.
240
241
	-r NUM
242
		Rotate backups while keeping up to NUM generations.
243
		Uses filename to match.
244
		Default: keep all backups
245
246
	-n
247
		Skip reverse lookup of IP address for host.
248
249
	-q
250
		Quiet mode: no output except for errors (for batch/crontab use).
251
252
EXAMPLES
253
	$(basename $BASH_SOURCE) -h 1.2.3.4 -d zabbixdb -u zabbix -p test
254
	$(basename $BASH_SOURCE) -u zabbix -p - -o /tmp
255
	$(basename $BASH_SOURCE) -c /etc/mysql/mysql.cnf
256
	$(basename $BASH_SOURCE) -c /etc/mysql/mysql.cnf -d zabbixdb
257
258
EOF
259
	exit 1
260
fi	
261
262
#
263
# PARSE COMMAND LINE ARGUMENTS
264
#
265
DB_GIVEN=0
266
while getopts ":h:d:u:p:o:r:c:qn" opt; do
267
	case $opt in
268
		h)  DBHOST="$OPTARG" ;;
269
		d)  DBNAME="$OPTARG"; DB_GIVEN=1 ;;
270
		u)  DBUSER="$OPTARG" ;;
271
		p)  DBPASS="$OPTARG" ;;
272
		c)  CNFFILE="$OPTARG" ;;
273
		o)  DUMPDIR="$OPTARG" ;;
274
		r)  GENERATIONSTOKEEP=$(printf '%.0f' "$OPTARG") ;;
275
		n)  REVERSELOOKUP="no" ;;
276
		q)  QUIET="yes" ;;
277
		\?) echo "Invalid option: -$OPTARG" >&2; exit 1 ;;
278
		:)  echo "Option -$OPTARG requires an argument" >&2; exit 1 ;;
279
	esac
280
done
281
282
# Password prompt
283
if [ "$DBPASS" = "-" ]; then
284
	read -s -p "Enter MySQL password for user '$DBUSER' (input will be hidden): " DBPASS
285
	echo ""
286
fi
287
288
# Config file validations
289
if [ ! -z "$CNFFILE" ]; then
290
	if [ ! -r "$CNFFILE" ]; then
291
		echo "ERROR: Cannot read configuration file $CNFFILE" >&2
292
		exit 1
293
	fi
294
	# Database name needs special treatment:
295
	# For mysqldump it has to be specified on the command line!
296
	# Therefore we need to get it from the config file
297
	if [ $DB_GIVEN -eq 0 ]; then
298
		DBNAME=$(grep -m 1 ^database= "$CNFFILE" | cut -d= -f2)
299
	fi
300
fi
301
302
#
303
# CONSTANTS
304
#
305
MYSQL_OPTS=()
306
[ ! -z "$CNFFILE" ] && MYSQL_OPTS=("${MYSQL_OPTS[@]}" --defaults-extra-file="$CNFFILE")
307
[ ! -z "$DBHOST" ] && MYSQL_OPTS=("${MYSQL_OPTS[@]}" -h $DBHOST)
308
[ ! -z "$DBUSER" ] && MYSQL_OPTS=("${MYSQL_OPTS[@]}" -u $DBUSER)
309
[ ! -z "$DBPASS" ] && MYSQL_OPTS=("${MYSQL_OPTS[@]}" -p"$DBPASS")
310
311
MYSQL_OPTS_BATCH=("${MYSQL_OPTS[@]}" --batch --silent)
312
[ ! -z "$DBNAME" ] && MYSQL_OPTS_BATCH=("${MYSQL_OPTS_BATCH[@]}" -D $DBNAME)
313
314
# Host name: try reverse lookup if IP is given
315
DBHOSTNAME="$DBHOST"
316
command -v dig >/dev/null 2>&1
317
FIND_DIG=$?
318
if [ "$REVERSELOOKUP" == "yes" -a $FIND_DIG -eq 0 ]; then
319
	# Try resolving a given host ip
320
	newHostname=$(dig +noall +answer -x $DBHOST | sed -r 's/((\S+)\s+)+([^\.]+)\..*/\3/')
321
	test \! -z "$newHostname" && DBHOSTNAME="$newHostname"
322
fi
323
324
#
325
# CONFIG DUMP
326
#
327
if [ "$QUIET" == "no" ]; then
328
	cat <<-EOF
329
	Configuration:
330
	 - host:     $DBHOST ($DBHOSTNAME)
331
	 - database: $DBNAME
332
	 - user:     $DBUSER
333
	 - output:   $DUMPDIR
334
335
	EOF
336
fi
337
338
#
339
# FUNCTIONS
340
#
341
342
# Returns TRUE if argument 1 is part of the given array (remaining arguments)
343
elementIn () {
344
	local e
345
	for e in "${@:2}"; do [[ "$e" == "$1" ]] && return 0; done
346
	return 1
347
}
348
349
#
350
# CHECKS
351
#
352
if [ ! -x /usr/bin/mysqldump ]; then
353
	echo "mysqldump not found." >&2
354
	echo "(with Debian, \"apt-get install mysql-client\" will help)" >&2
355
	exit 1
356
fi
357
358
#
359
# READ TABLE LIST from __DATA__ section at the end of this script
360
# (http://stackoverflow.com/a/3477269/2983301) 
361
#
362
DATA_TABLES=()
363
while read line; do
364
	table=$(echo "$line" | cut -d" " -f1)
365
	echo "$line" | cut -d" " -f5 | grep -qi "DATA"
366
	test $? -eq 0 && DATA_TABLES+=($table)
367
done < <(sed '0,/^__DATA__$/d' "$BASH_SOURCE" | tr -s " ")
368
369
# paranoid check
370
if [ ${#DATA_TABLES[@]} -lt 5 ]; then
371
	echo "ERROR: The number of large data tables configured in this script is less than 5." >&2
372
	exit 1
373
fi
374
375
#
376
# BACKUP
377
#
378
# Read table list from database
379
[ "$QUIET" == "no" ] && echo "Fetching list of existing tables..."
380
DB_TABLES=$(mysql "${MYSQL_OPTS_BATCH[@]}" -e "SELECT table_name FROM information_schema.tables WHERE table_schema = '$DBNAME'" 2>&1)
381
if [ $? -ne 0 ]; then echo -e "ERROR while trying to access database:\n$DB_TABLES" 2>&1; exit 1; fi
382
DB_TABLES=$(echo "$DB_TABLES" | sort)
383
DB_TABLE_NUM=$(echo "$DB_TABLES" | wc -l)
384
385
# Query Zabbix database version
386
VERSION=""
387
DB_VER=$(mysql "${MYSQL_OPTS_BATCH[@]}" -N -e "select optional from dbversion;" 2>&1)
388
if [ $? -eq 0 ]; then
389
	# version string is like: 02030015
390
	re='(.*)([0-9]{2})([0-9]{4})'
391
	if [[ $DB_VER =~ $re ]]; then
392
		VERSION="_db-${BASH_REMATCH[1]}.$(( ${BASH_REMATCH[2]} + 0 )).$(( ${BASH_REMATCH[3]} + 0 ))"
393
	fi
394
fi
395
396
# Assemble file name
397
DUMPFILENAME_PREFIX="zabbix_cfg_${DBHOSTNAME}"
398
DUMPFILEBASE="${DUMPFILENAME_PREFIX}_$(date +%Y%m%d-%H%M)${VERSION}.sql"
399
DUMPFILE="$DUMPDIR/$DUMPFILEBASE"
400
401
PROCESSED_DATA_TABLES=()
402
i=0
403
404
mkdir -p "${DUMPDIR}"
405
406
[ "$QUIET" == "no" ] && echo "Starting table backups..."
407
while read table; do
408
	# large data tables: only store schema
409
	if elementIn "$table" "${DATA_TABLES[@]}"; then
410
		dump_opt="--no-data"
411
		PROCESSED_DATA_TABLES+=($table)
412
	# configuration tables: full dump
413
	else
414
		dump_opt="--extended-insert=FALSE"
415
	fi
416
417
	mysqldump "${MYSQL_OPTS[@]}" \
418
		--routines --opt --single-transaction --skip-lock-tables \
419
		$dump_opt \
420
		$DBNAME --tables ${table} >> "$DUMPFILE"
421
422
	if [ "$QUIET" == "no" ]; then
423
		# show percentage
424
		i=$((i+1)); i_percent=$(($i * 100 / $DB_TABLE_NUM))
425
		if [ $(($i_percent % 12)) -eq 0 ]; then
426
			echo -n "${i_percent}%"
427
		else
428
			if [ $(($i_percent % 2)) -eq 0 ]; then echo -n "."; fi
429
		fi
430
	fi
431
done <<<"$DB_TABLES" 
432
433
#
434
# COMPRESS BACKUP
435
#
436
if [ "$QUIET" == "no" ]; then
437
	echo -e "\n"
438
	echo "For the following large tables only the schema (without data) was stored:"
439
	for table in "${PROCESSED_DATA_TABLES[@]}"; do echo " - $table"; done
440
	
441
	echo 
442
	echo "Compressing backup file..."
443
fi
444
gzip -f "$DUMPFILE"
445
if [ $? -ne 0 ]; then
446
	echo -e "\nERROR: Could not compress backup file, see previous messages" >&2
447
	exit 1
448
fi
449
450
[ "$QUIET" == "no" ] && echo -e "\nBackup Completed:\n${DUMPFILE}.gz"
451
452
#
453
# ROTATE OLD BACKUPS
454
#
455
if [ $GENERATIONSTOKEEP -gt 0 ]; then
456
	[ "$QUIET" == "no" ] && echo "Removing old backups, keeping up to $GENERATIONSTOKEEP"
457
	REMOVE_OLD_CMD="cd \"$DUMPDIR\" && ls -t \"${DUMPFILENAME_PREFIX}\"* | /usr/bin/awk \"NR>${GENERATIONSTOKEEP}\" | xargs rm -f "
458
	eval ${REMOVE_OLD_CMD}
459
	if [ $? -ne 0 ]; then
460
		echo "ERROR: Could not rotate old backups" >&2
461
		exit 1
462
	fi
463
fi
464
465
exit 0
466
467
################################################################################
468
# List of all known table names and a flag indicating data (=large) tables
469
#
470
471
__DATA__
472
acknowledges              1.3.1    - 2.4.0  DATA
473
actions                   1.3.1    - 2.4.0
474
alerts                    1.3.1    - 2.4.0  DATA
475
application_template      2.1.0    - 2.4.0
476
applications              1.3.1    - 2.4.0
477
auditlog                  1.3.1    - 2.4.0  DATA
478
auditlog_details          1.7      - 2.4.0  DATA
479
autoreg                   1.3.1    - 1.3.4
480
autoreg_host              1.7      - 2.4.0
481
conditions                1.3.1    - 2.4.0
482
config                    1.3.1    - 2.4.0
483
dbversion                 2.1.0    - 2.4.0
484
dchecks                   1.3.4    - 2.4.0
485
dhosts                    1.3.4    - 2.4.0
486
drules                    1.3.4    - 2.4.0
487
dservices                 1.3.4    - 2.4.0
488
escalations               1.5.3    - 2.4.0
489
events                    1.3.1    - 2.4.0  DATA
490
expressions               1.7      - 2.4.0
491
functions                 1.3.1    - 2.4.0
492
globalmacro               1.7      - 2.4.0
493
globalvars                1.9.6    - 2.4.0
494
graph_discovery           1.9.0    - 2.4.0
495
graph_theme               1.7      - 2.4.0
496
graphs                    1.3.1    - 2.4.0
497
graphs_items              1.3.1    - 2.4.0
498
group_discovery           2.1.4    - 2.4.0
499
group_prototype           2.1.4    - 2.4.0
500
groups                    1.3.1    - 2.4.0
501
help_items                1.3.1    - 2.1.8
502
history                   1.3.1    - 2.4.0  DATA
503
history_log               1.3.1    - 2.4.0  DATA
504
history_str               1.3.1    - 2.4.0  DATA
505
history_str_sync          1.3.1    - 2.2.6  DATA
506
history_sync              1.3.1    - 2.2.6  DATA
507
history_text              1.3.1    - 2.4.0  DATA
508
history_uint              1.3.1    - 2.4.0  DATA
509
history_uint_sync         1.3.1    - 2.2.6  DATA
510
host_discovery            2.1.4    - 2.4.0
511
host_inventory            1.9.6    - 2.4.0
512
host_profile              1.9.3    - 1.9.5
513
hostmacro                 1.7      - 2.4.0
514
hosts                     1.3.1    - 2.4.0
515
hosts_groups              1.3.1    - 2.4.0
516
hosts_profiles            1.3.1    - 1.9.2
517
hosts_profiles_ext        1.6      - 1.9.2
518
hosts_templates           1.3.1    - 2.4.0
519
housekeeper               1.3.1    - 2.4.0
520
httpstep                  1.3.3    - 2.4.0
521
httpstepitem              1.3.3    - 2.4.0
522
httptest                  1.3.3    - 2.4.0
523
httptestitem              1.3.3    - 2.4.0
524
icon_map                  1.9.6    - 2.4.0
525
icon_mapping              1.9.6    - 2.4.0
526
ids                       1.3.3    - 2.4.0
527
images                    1.3.1    - 2.4.0
528
interface                 1.9.1    - 2.4.0
529
interface_discovery       2.1.4    - 2.4.0
530
item_condition            2.3.0    - 2.4.0
531
item_discovery            1.9.0    - 2.4.0
532
items                     1.3.1    - 2.4.0
533
items_applications        1.3.1    - 2.4.0
534
maintenances              1.7      - 2.4.0
535
maintenances_groups       1.7      - 2.4.0
536
maintenances_hosts        1.7      - 2.4.0
537
maintenances_windows      1.7      - 2.4.0
538
mappings                  1.3.1    - 2.4.0
539
media                     1.3.1    - 2.4.0
540
media_type                1.3.1    - 2.4.0
541
node_cksum                1.3.1    - 2.2.6
542
node_configlog            1.3.1    - 1.4.7
543
nodes                     1.3.1    - 2.2.6
544
opcommand                 1.9.4    - 2.4.0
545
opcommand_grp             1.9.2    - 2.4.0
546
opcommand_hst             1.9.2    - 2.4.0
547
opconditions              1.5.3    - 2.4.0
548
operations                1.3.4    - 2.4.0
549
opgroup                   1.9.2    - 2.4.0
550
opmediatypes              1.7      - 1.8.21
551
opmessage                 1.9.2    - 2.4.0
552
opmessage_grp             1.9.2    - 2.4.0
553
opmessage_usr             1.9.2    - 2.4.0
554
optemplate                1.9.2    - 2.4.0
555
profiles                  1.3.1    - 2.4.0
556
proxy_autoreg_host        1.7      - 2.4.0
557
proxy_dhistory            1.5      - 2.4.0
558
proxy_history             1.5.1    - 2.4.0
559
regexps                   1.7      - 2.4.0
560
rights                    1.3.1    - 2.4.0
561
screens                   1.3.1    - 2.4.0
562
screens_items             1.3.1    - 2.4.0
563
scripts                   1.5      - 2.4.0
564
service_alarms            1.3.1    - 2.4.0
565
services                  1.3.1    - 2.4.0
566
services_links            1.3.1    - 2.4.0
567
services_times            1.3.1    - 2.4.0
568
sessions                  1.3.1    - 2.4.0
569
slides                    1.3.4    - 2.4.0
570
slideshows                1.3.4    - 2.4.0
571
sysmap_element_url        1.9.0    - 2.4.0
572
sysmap_url                1.9.0    - 2.4.0
573
sysmaps                   1.3.1    - 2.4.0
574
sysmaps_elements          1.3.1    - 2.4.0
575
sysmaps_link_triggers     1.5      - 2.4.0
576
sysmaps_links             1.3.1    - 2.4.0
577
timeperiods               1.7      - 2.4.0
578
trends                    1.3.1    - 2.4.0  DATA
579
trends_uint               1.5      - 2.4.0  DATA
580
trigger_depends           1.3.1    - 2.4.0
581
trigger_discovery         1.9.0    - 2.4.0
582
triggers                  1.3.1    - 2.4.0
583
user_history              1.7      - 2.4.0
584
users                     1.3.1    - 2.4.0
585
users_groups              1.3.1    - 2.4.0
586
usrgrp                    1.3.1    - 2.4.0
587
valuemaps                 1.3.1    - 2.4.0
588
589
</pre>