Projekt

Allgemein

Profil

Setup kvm » Historie » Version 6

Jeremias Keihsler, 18.01.2021 12:10

1 1 Jeremias Keihsler
h1. KVM
2
3
this is for a vanilla CentOS 7 minimal installation,
4 2 Jeremias Keihsler
largely based on @kvm_virtualization_in_rhel_7_made_easy.pdf@
5 1 Jeremias Keihsler
6
good information is also found at http://virtuallyhyper.com/2013/06/migrate-from-libvirt-kvm-to-virtualbox/
7
8
h2. basic updates/installs
9
10
<pre><code class="bash">
11
yum update
12
yum install wget
13
yum install vim
14
reboot
15
</code></pre>
16
17
h2. check machine capability
18
19
<pre><code class="bash">
20
grep -E 'svm|vmx' /proc/cpuinfo
21
</code></pre>
22
23
vmx ... Intel
24
svm ... AMD
25
26
h2. install KVM on CentOS minimal
27
28
<pre><code class="bash">
29
yum install yum install qemu-kvm libvirt libvirt-python libguestfs-tools virt-install
30
systemctl enable libvirtd && systemctl start libvirtd
31
</code></pre>
32
33
verify the following kernel modules are loaded
34
<pre><code class="bash">
35
lsmod | grep kvm
36
</code></pre>
37
38
<pre><code class="bash">
39
kvm
40
kvm_intel
41
</code></pre>
42
h2. setup networking
43
44
add to the network controller configuration file @/etc/sysconfig/network-scripts/ifcfg-em1@
45 3 Jeremias Keihsler
<pre>
46 1 Jeremias Keihsler
...
47
BRIDGE=br0
48
</pre>
49
50
add following new file @/etc/sysconfig/network-scripts/ifcfg-br0@
51 3 Jeremias Keihsler
<pre>
52 1 Jeremias Keihsler
DEVICE="br0"
53
# BOOTPROTO is up to you. If you prefer “static”, you will need to
54
# specify the IP address, netmask, gateway and DNS information.
55
BOOTPROTO="dhcp"
56
IPV6INIT="yes"
57
IPV6_AUTOCONF="yes"
58
ONBOOT="yes"
59
TYPE="Bridge"
60
DELAY="0"
61
</pre>
62
63
enable network forwarding @/etc/sysctl.conf@
64 3 Jeremias Keihsler
<pre>
65 1 Jeremias Keihsler
...
66
net.ipv4.ip_forward = 1
67
</pre>
68
69
read the file and restart NetworkManager
70
<pre><code class="bash">
71
sysctl -p /etc/sysctl.conf
72
systemctl restart NetworkManager
73
</code></pre>
74
75
h2. can KVM and Virtualbox coexist
76
77
http://www.dedoimedo.com/computers/kvm-virtualbox.html
78
79
h2. convert Virtualbox to KVM
80
81
h3. uninstall Virtualbox-guest-additions
82
83
<pre><code class="bash">
84
opt/[VboxAddonsFolder]/uninstall.sh
85
</code></pre>
86
87
some people had to remove @/etc/X11/xorg.conf@
88
89
h3. convert image from Virtualbox to KWM
90
91
<pre><code class="bash">
92
VBoxManage clonehd --format RAW Virt_Image.vdi Virt_Image.img
93
</code></pre>
94
95
RAW-Datei nach qcow konvertieren
96
<pre><code class="bash">
97
qemu-img convert -f raw Virt_Image.img -O qcow2 Virt_Image.qcow
98
</code></pre>
99
100
h2. automatic start/shutdown of VMs with Host
101
102
taken from https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/sub-sect-Shutting_down_rebooting_and_force_shutdown_of_a_guest_virtual_machine-Manipulating_the_libvirt_guests_configuration_settings.html
103
104
h3. enable libvirt-guests service
105
<pre><code class="bash">
106
systemctl enable libvirt-guests
107
systemctl start libvirt-guests
108
</code></pre>
109
110
all settings are to be done in @/etc/sysconfig/libvirt-guests@
111
112
h2. install
113
114
115
<pre><code class="bash">
116
yum install virt-manager
117
</code></pre>
118
119
<pre><code class="bash">
120
usermod -a -G libvirt username
121
</code></pre>
122
123
h2. rename KVM-guest
124
125
taken from http://www.taitclarridge.com/techlog/2011/01/rename-kvm-virtual-machine-with-virsh.html
126
127
Power off the virtual machine and export the machine's XML configuration file:
128
129
<pre><code class="bash">
130
virsh dumpxml name_of_vm > name_of_vm.xml
131
</code></pre>
132
133
Next, edit the XML file and change the name between the <name></name> tags (should be right near the top). As an added step you could also rename the disk file to reflect the change of the name and change the name of it in the <devices> section under <source file='/path/to/name_of_vm.img'>.
134
135
Save the XML file and undefine the old VM name with:
136
137
<pre><code class="bash">
138
virsh undefine name_of_vm
139
</code></pre>
140
141
Now just import the edited XML file to define the VM:
142
143
<pre><code class="bash">
144
virsh define name_of_vm.xml
145
</code></pre>
146
147
And that should be it! You can now start up your vm either in the Virtual Machine Manager or with virsh using:
148
149
<pre><code class="bash">
150
virsh start name_of_vm
151
</code></pre>
152
153
h2. set fixed IP-adr via DHCP (default-network)
154
155
taken from https://wiki.libvirt.org/page/Networking
156
157
<pre><code class="bash">
158
virsh edit <guest>
159
</code></pre>
160
161
where <guest> is the name or uuid of the guest. Add the following snippet of XML to the config file: 
162
163
<pre><code class="bash">
164
<interface type='network'>
165
  <source network='default'/>
166
  <mac address='00:16:3e:1a:b3:4a'/>
167
</interface>
168
</code></pre>
169
170
Applying modifications to the network
171
172
Sometimes, one needs to edit the network definition and apply the changes on the fly. The most common scenario for this is adding new static MAC+IP mappings for the network's DHCP server. If you edit the network with "virsh net-edit", any changes you make won't take effect until the network is destroyed and re-started, which unfortunately will cause a all guests to lose network connectivity with the host until their network interfaces are explicitly re-attached.
173
virsh net-update
174
175
Fortunately, many changes to the network configuration (including the aforementioned addition of a static MAC+IP mapping for DHCP) can be done with "virsh net-update", which can be told to enact the changes immediately. For example, to add a DHCP static host entry to the network named "default" mapping MAC address 53:54:00:00:01 to IP address 192.168.122.45 and hostname "bob", you could use this command: 
176
177
<pre><code class="bash">
178
virsh net-update default add ip-dhcp-host \
179
          "<host mac='52:54:00:00:00:01' \
180
           name='bob' ip='192.168.122.45' />" \
181
           --live --config
182
</code></pre>
183
184
h2. forwarding incoming connections
185
186
taken from https://wiki.libvirt.org/page/Networking
187
188
By default, guests that are connected via a virtual network with <forward mode='nat'/> can make any outgoing network connection they like. Incoming connections are allowed from the host, and from other guests connected to the same libvirt network, but all other incoming connections are blocked by iptables rules.
189
190
If you would like to make a service that is on a guest behind a NATed virtual network publicly available, you can setup libvirt's "hook" script for qemu to install the necessary iptables rules to forward incoming connections to the host on any given port HP to port GP on the guest GNAME:
191
192
1) Determine a) the name of the guest "G" (as defined in the libvirt domain XML), b) the IP address of the guest "I", c) the port on the guest that will receive the connections "GP", and d) the port on the host that will be forwarded to the guest "HP".
193
194
(To assure that the guest's IP address remains unchanged, you can either configure the guest OS with static ip information, or add a <host> element inside the <dhcp> element of the network that is used by your guest. See the libvirt network XML documentation address section for defails and an example.)
195
196
2) Stop the guest if it's running.
197
198
3) Create the file /etc/libvirt/hooks/qemu (or add the following to an already existing hook script), with contents similar to the following (replace GNAME, IP, GP, and HP appropriately for your setup):
199
200
Use the basic script below or see an "advanced" version, which can handle several different machines and port mappings here (improvements are welcome) or here's a python script which does a similar thing and is easy to understand and configure (improvements are welcome): 
201
202
<pre>
203
#!/bin/bash
204
# used some from advanced script to have multiple ports: use an equal number of guest and host ports
205
206
# Update the following variables to fit your setup
207
Guest_name=GUEST_NAME
208
Guest_ipaddr=GUEST_IP
209
Host_ipaddr=HOST_IP
210
Host_port=(  'HOST_PORT1' 'HOST_PORT2' )
211
Guest_port=( 'GUEST_PORT1' 'GUEST_PORT2' )
212
213
length=$(( ${#Host_port[@]} - 1 ))
214
if [ "${1}" = "${Guest_name}" ]; then
215
   if [ "${2}" = "stopped" ] || [ "${2}" = "reconnect" ]; then
216
       for i in `seq 0 $length`; do
217
               iptables -t nat -D PREROUTING -d ${Host_ipaddr} -p tcp --dport ${Host_port[$i]} -j DNAT --to ${Guest_ipaddr}:${Guest_port[$i]}
218
               iptables -D FORWARD -d ${Guest_ipaddr}/32 -p tcp -m state --state NEW -m tcp --dport ${Guest_port[$i]} -j ACCEPT
219
       done
220
   fi
221
   if [ "${2}" = "start" ] || [ "${2}" = "reconnect" ]; then
222
       for i in `seq 0 $length`; do
223
               iptables -t nat -A PREROUTING -d ${Host_ipaddr} -p tcp --dport ${Host_port[$i]} -j DNAT --to ${Guest_ipaddr}:${Guest_port[$i]}
224
               iptables -I FORWARD -d ${Guest_ipaddr}/32 -p tcp -m state --state NEW -m tcp --dport ${Guest_port[$i]} -j ACCEPT
225
       done
226
   fi
227
fi
228
</pre>
229
4) chmod +x /etc/libvirt/hooks/qemu
230
231
5) Restart the libvirtd service.
232
233
6) Start the guest.
234
235
(NB: This method is a hack, and has one annoying flaw in versions of libvirt prior to 0.9.13 - if libvirtd is restarted while the guest is running, all of the standard iptables rules to support virtual networks that were added by libvirtd will be reloaded, thus changing the order of the above FORWARD rule relative to a reject rule for the network, hence rendering this setup non-working until the guest is stopped and restarted. Thanks to the new "reconnect" hook in libvirt-0.9.13 and newer (which is used by the above script if available), this flaw is not present in newer versions of libvirt (however, this hook script should still be considered a hack). 
236
237
h2. wrapper script for virsh
238
239 2 Jeremias Keihsler
<pre>
240 1 Jeremias Keihsler
#! /bin/sh
241
# kvm_control   Startup script for KVM Virtual Machines
242
#
243
# description: Manages KVM VMs
244
# processname: kvm_control.sh
245
#
246
# pidfile: /var/run/kvm_control/kvm_control.pid
247
#
248
### BEGIN INIT INFO
249
#
250
### END INIT INFO
251
#
252 4 Jeremias Keihsler
# Version 20171103 by Jeremias Keihsler added ionice prio 'idle'
253 1 Jeremias Keihsler
# Version 20161228 by Jeremias Keihsler based on:
254
# virsh-specific parts are taken from:
255
#  https://github.com/kumina/shutdown-kvm-guests/blob/master/shutdown-kvm-guests.sh
256
# Version 20110509 by Jeremias Keihsler (vboxcontrol) based on:
257
# Version 20090301 by Kevin Swanson <kswan.info> based on:
258
# Version 2008051100 by Jochem Kossen <jochem.kossen@gmail.com>
259
# http://farfewertoes.com
260
#
261
# Released in the public domain
262
#
263
# This file came with a README file containing the instructions on how
264
# to use this script.
265
# 
266
# this is no more to be used as an init.d-script (vboxcontrol was an init.d-script)
267
#
268
269
################################################################################
270
# INITIAL CONFIGURATION
271
272
export PATH="${PATH:+$PATH:}/bin:/usr/bin:/usr/sbin:/sbin"
273
274
VIRSH=/usr/bin/virsh
275
TIMEOUT=300
276
277
declare -i VM_isrunning
278
279
################################################################################
280
# FUNCTIONS
281
282
log_failure_msg() {
283
echo $1
284
}
285
286
log_action_msg() {
287
echo $1
288
}
289
290
# list running domains
291
list_running_domains() {
292
  $VIRSH list | grep running | awk '{ print $2}'
293
}
294
295
# Check for running machines every few seconds; return when all machines are
296
# down
297
wait_for_closing_machines() {
298
RUNNING_MACHINES=`list_running_domains | wc -l`
299
if [ $RUNNING_MACHINES != 0 ]; then
300
  log_action_msg "machines running: "$RUNNING_MACHINES
301
  sleep 2
302
303
  wait_for_closing_machines
304
fi
305
}
306
307
################################################################################
308
# RUN
309
case "$1" in
310
  start)
311
    if [ -f /etc/kvm_box/machines_enabled_start ]; then
312
313
      cat /etc/kvm_box/machines_enabled_start | while read VM; do
314
        log_action_msg "Starting VM: $VM ..."
315
        $VIRSH start $VM
316
        sleep 20
317
        RETVAL=$?
318
      done
319
      touch /tmp/kvm_control
320
    fi
321
  ;;
322
  stop)
323
    # NOTE: this stops first the listed VMs in the given order
324
    # and later all running VM's. 
325
    # After the defined timeout all remaining VMs are killed
326
327
    # Create some sort of semaphore.
328
    touch /tmp/shutdown-kvm-guests
329
330
    echo "Try to cleanly shut down all listed KVM domains..."
331
    # Try to shutdown each listed domain, one by one.
332
    if [ -f /etc/kvm_box/machines_enabled_stop ]; then
333
      cat /etc/kvm_box/machines_enabled_stop | while read VM; do
334
        log_action_msg "Shutting down VM: $VM ..."
335
        $VIRSH shutdown $VM --mode acpi
336
        sleep 10
337
        RETVAL=$?
338
      done
339
    fi
340
    sleep 10
341 4 Jeremias Keihsler
342
    echo "give still running machines some more time..."
343
    # wait 20s per still running machine
344
    list_running_domains | while read VM; do
345
      log_action_msg "waiting 20s ... for: $VM ..."
346
      sleep 20
347
    done
348
349 1 Jeremias Keihsler
    echo "Try to cleanly shut down all running KVM domains..."
350
    # Try to shutdown each remaining domain, one by one.
351
    list_running_domains | while read VM; do
352
      log_action_msg "Shutting down VM: $VM ..."
353
      $VIRSH shutdown $VM --mode acpi
354
      sleep 10
355
    done
356
357
    # Wait until all domains are shut down or timeout has reached.
358
    END_TIME=$(date -d "$TIMEOUT seconds" +%s)
359
360
    while [ $(date +%s) -lt $END_TIME ]; do
361
      # Break while loop when no domains are left.
362
      test -z "$(list_running_domains)" && break
363
      # Wait a litte, we don't want to DoS libvirt.
364
      sleep 2
365
    done
366
367
    # Clean up left over domains, one by one.
368
    list_running_domains | while read DOMAIN; do
369
      # Try to shutdown given domain.
370
      $VIRSH destroy $DOMAIN
371
      # Give libvirt some time for killing off the domain.
372
      sleep 10
373
    done
374
375
    wait_for_closing_machines
376
    rm -f /tmp/shutdown-kvm-guests
377
    rm -f /tmp/kvm_control
378
  ;;
379
  export)
380
    JKE_DATE=$(date +%F)
381
    if [ -f /etc/kvm_box/machines_enabled_export ]; then
382
      cat /etc/kvm_box/machines_enabled_export  | while read VM; do
383
        rm -f /tmp/kvm_control_VM_isrunning
384
        VM_isrunning=0
385
        list_running_domains | while read RVM; do
386
          #echo "VM list -$VM- : -$RVM-"
387 4 Jeremias Keihsler
          if [[ "$VM" ==  "$RVM" ]]; then
388 1 Jeremias Keihsler
            #echo "VM found running..."
389
            touch /tmp/kvm_control_VM_isrunning
390
            VM_isrunning=1
391
            #echo "$VM_isrunning"
392
            break
393
          fi
394
          #echo "$VM_isrunning"
395 4 Jeremias Keihsler
        done
396 1 Jeremias Keihsler
397
        # took me a while to figure out that the above 'while'-loop 
398
        # runs in a separate process ... let's use the 'file' as a 
399
        # kind of interprocess-communication :-) JKE 20161229
400
        if [ -f /tmp/kvm_control_VM_isrunning ]; then
401
          VM_isrunning=1
402
        fi
403
        rm -f /tmp/kvm_control_VM_isrunning
404 4 Jeremias Keihsler
405 1 Jeremias Keihsler
        #echo "VM status $VM_isrunning"
406 4 Jeremias Keihsler
        if [ "$VM_isrunning" -ne 0 ]; then
407 1 Jeremias Keihsler
          log_failure_msg "Exporting VM: $VM is not possible, it's running ..."
408
        else
409
          log_action_msg "Exporting VM: $VM ..."
410
          VM_BAK_DIR="$VM"_"$JKE_DATE"
411
          mkdir "$VM_BAK_DIR"
412
          $VIRSH dumpxml $VM > ./$VM_BAK_DIR/$VM.xml
413
          $VIRSH -q domblklist $VM | awk '{ print$2}' | while read VMHDD; do
414
            echo "$VM hdd=$VMHDD"
415
            if [ -f "$VMHDD" ]; then
416 4 Jeremias Keihsler
              ionice -c 3 rsync --progress $VMHDD ./$VM_BAK_DIR/`basename $VMHDD`
417 1 Jeremias Keihsler
            else
418
              log_failure_msg "Exporting VM: $VM image-file $VMHDD not found ..."
419
            fi
420
          done
421
        fi
422
      done
423
    else
424
      log_action_msg "export-list not found"
425
    fi
426
  ;;
427
  start-vm)
428
    log_action_msg "Starting VM: $2 ..."
429
    $VIRSH start $2
430
    RETVAL=$?
431
  ;;
432
  stop-vm)
433
    log_action_msg "Stopping VM: $2 ..."
434
    $VIRSH shutdown $2 --mode acpi
435
    RETVAL=$?
436
  ;;
437
  poweroff-vm)
438
    log_action_msg "Powering off VM: $2 ..."
439
    $VIRSH destroy $2
440
    RETVAL=$?
441
  ;;
442
  export-vm)
443
    # NOTE: this exports the given VM
444
    log_action_msg "Exporting VM: $2 ..."
445
    rm -f /tmp/kvm_control_VM_isrunning
446
    VM_isrunning=0
447
    JKE_DATE=$(date +%F)
448
    list_running_domains | while read RVM; do
449
      #echo "VM list -$VM- : -$RVM-"
450 4 Jeremias Keihsler
      if [[ "$2" ==  "$RVM" ]]; then
451 1 Jeremias Keihsler
        #echo "VM found running..."
452
        touch /tmp/kvm_control_VM_isrunning
453
        VM_isrunning=1
454
        #echo "$VM_isrunning"
455
        break
456
      fi
457
      #echo "$VM_isrunning"
458 4 Jeremias Keihsler
    done
459 1 Jeremias Keihsler
460
    # took me a while to figure out that the above 'while'-loop 
461
    # runs in a separate process ... let's use the 'file' as a 
462
    # kind of interprocess-communication :-) JKE 20161229
463
    if [ -f /tmp/kvm_control_VM_isrunning ]; then
464
      VM_isrunning=1
465
    fi
466
    rm -f /tmp/kvm_control_VM_isrunning
467 4 Jeremias Keihsler
468 1 Jeremias Keihsler
    #echo "VM status $VM_isrunning"
469 4 Jeremias Keihsler
    if [ "$VM_isrunning" -ne 0 ]; then
470 1 Jeremias Keihsler
      log_failure_msg "Exporting VM: $VM is not possible, it's running ..."
471
    else
472
      log_action_msg "Exporting VM: $VM ..."
473
      VM_BAK_DIR="$2"_"$JKE_DATE"
474
      mkdir "$VM_BAK_DIR"
475
      $VIRSH dumpxml $2 > ./$VM_BAK_DIR/$2.xml
476
      $VIRSH -q domblklist $2 | awk '{ print$2}' | while read VMHDD; do
477
        echo "$2 hdd=$VMHDD"
478
        if [ -f "$VMHDD" ]; then
479 4 Jeremias Keihsler
          ionice -c 3 rsync --progress $VMHDD ./$VM_BAK_DIR/`basename $VMHDD`
480 1 Jeremias Keihsler
        else
481
          log_failure_msg "Exporting VM: $2 image-file $VMHDD not found ..."
482
        fi
483
      done
484
    fi
485
  ;;
486
  status)
487
    echo "The following virtual machines are currently running:"
488
    list_running_domains | while read VM; do
489
      echo -n "  $VM"
490
      echo " ... is running"
491
    done
492
  ;;
493
494
  *)
495
    echo "Usage: $0 {start|stop|status|export|start-vm <VM name>|stop-vm <VM name>|poweroff-vm <VM name>}|export-vm <VMname>"
496
    echo "  start      start all VMs listed in '/etc/kvm_box/machines_enabled_start'"
497
    echo "  stop       1st step: acpi-shutdown all VMs listed in '/etc/kvm_box/machines_enabled_stop'"
498 4 Jeremias Keihsler
    echo "             2nd step: wait 20s for each still running machine to give a chance to shut-down on their own"
499
    echo "             3rd step: acpi-shutdown all running VMs"
500
    echo "             4th step: wait for all machines shutdown or $TIMEOUT s"
501
    echo "             5th step: destroy all sitting VMs"
502 1 Jeremias Keihsler
    echo "  status     list all running VMs"
503
    echo "  export     export all VMs listed in '/etc/kvm_box/machines_enabled_export' to the current directory"
504
    echo "  start-vm <VM name>     start the given VM"
505
    echo "  stop-vm <VM name>      acpi-shutdown the given VM"
506
    echo "  poweroff-vm <VM name>  poweroff the given VM"
507
    echo "  export-vm <VM name>    export the given VM to the current directory"
508
    exit 3
509
esac
510
511
exit 0
512 4 Jeremias Keihsler
513 1 Jeremias Keihsler
</pre>
514 5 Jeremias Keihsler
515
h2. restore 'exported' kvm-machines
516
517
<pre><code class="shell">
518
tar xvf mach-name_202x-01-01.tar.gz 
519
</code></pre>
520 6 Jeremias Keihsler
521
copy the image-files to @/var/lib/libvirt/images/@
522
523
define the machine by
524
525
<pre><code class="shell">
526
virsh define mach-name.xml
527
</code></pre>