Projekt

Allgemein

Profil

Setup kvm » Historie » Version 10

Jeremias Keihsler, 17.11.2022 16:06

1 1 Jeremias Keihsler
h1. KVM
2
3
this is for a vanilla CentOS 7 minimal installation,
4 2 Jeremias Keihsler
largely based on @kvm_virtualization_in_rhel_7_made_easy.pdf@
5 1 Jeremias Keihsler
6
good information is also found at http://virtuallyhyper.com/2013/06/migrate-from-libvirt-kvm-to-virtualbox/
7
8
h2. basic updates/installs
9
10
<pre><code class="bash">
11
yum update
12
yum install wget
13
yum install vim
14
reboot
15
</code></pre>
16
17
h2. check machine capability
18
19
<pre><code class="bash">
20
grep -E 'svm|vmx' /proc/cpuinfo
21
</code></pre>
22
23
vmx ... Intel
24
svm ... AMD
25
26
h2. install KVM on CentOS minimal
27
28
<pre><code class="bash">
29
yum install yum install qemu-kvm libvirt libvirt-python libguestfs-tools virt-install
30
systemctl enable libvirtd && systemctl start libvirtd
31
</code></pre>
32
33
verify the following kernel modules are loaded
34
<pre><code class="bash">
35
lsmod | grep kvm
36
</code></pre>
37
38
<pre><code class="bash">
39
kvm
40
kvm_intel
41
</code></pre>
42
h2. setup networking
43
44
add to the network controller configuration file @/etc/sysconfig/network-scripts/ifcfg-em1@
45 3 Jeremias Keihsler
<pre>
46 1 Jeremias Keihsler
...
47
BRIDGE=br0
48
</pre>
49
50
add following new file @/etc/sysconfig/network-scripts/ifcfg-br0@
51 3 Jeremias Keihsler
<pre>
52 1 Jeremias Keihsler
DEVICE="br0"
53
# BOOTPROTO is up to you. If you prefer “static”, you will need to
54
# specify the IP address, netmask, gateway and DNS information.
55
BOOTPROTO="dhcp"
56
IPV6INIT="yes"
57
IPV6_AUTOCONF="yes"
58
ONBOOT="yes"
59
TYPE="Bridge"
60
DELAY="0"
61
</pre>
62
63
enable network forwarding @/etc/sysctl.conf@
64 3 Jeremias Keihsler
<pre>
65 1 Jeremias Keihsler
...
66
net.ipv4.ip_forward = 1
67
</pre>
68
69
read the file and restart NetworkManager
70
<pre><code class="bash">
71
sysctl -p /etc/sysctl.conf
72
systemctl restart NetworkManager
73
</code></pre>
74
75
h2. can KVM and Virtualbox coexist
76
77
http://www.dedoimedo.com/computers/kvm-virtualbox.html
78
79
h2. convert Virtualbox to KVM
80
81
h3. uninstall Virtualbox-guest-additions
82
83
<pre><code class="bash">
84
opt/[VboxAddonsFolder]/uninstall.sh
85
</code></pre>
86
87
some people had to remove @/etc/X11/xorg.conf@
88
89 8 Jeremias Keihsler
h3. convert image from Virtualbox to KVM
90 1 Jeremias Keihsler
91
<pre><code class="bash">
92
VBoxManage clonehd --format RAW Virt_Image.vdi Virt_Image.img
93
</code></pre>
94
95
RAW-Datei nach qcow konvertieren
96
<pre><code class="bash">
97
qemu-img convert -f raw Virt_Image.img -O qcow2 Virt_Image.qcow
98
</code></pre>
99
100
h2. automatic start/shutdown of VMs with Host
101
102
taken from https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/sub-sect-Shutting_down_rebooting_and_force_shutdown_of_a_guest_virtual_machine-Manipulating_the_libvirt_guests_configuration_settings.html
103
104
h3. enable libvirt-guests service
105 9 Jeremias Keihsler
106 1 Jeremias Keihsler
<pre><code class="bash">
107
systemctl enable libvirt-guests
108
systemctl start libvirt-guests
109
</code></pre>
110
111
all settings are to be done in @/etc/sysconfig/libvirt-guests@
112
113
h2. install
114
115
116
<pre><code class="bash">
117
yum install virt-manager
118
</code></pre>
119
120
<pre><code class="bash">
121
usermod -a -G libvirt username
122
</code></pre>
123
124
h2. rename KVM-guest
125
126
taken from http://www.taitclarridge.com/techlog/2011/01/rename-kvm-virtual-machine-with-virsh.html
127
128
Power off the virtual machine and export the machine's XML configuration file:
129
130
<pre><code class="bash">
131
virsh dumpxml name_of_vm > name_of_vm.xml
132
</code></pre>
133
134
Next, edit the XML file and change the name between the <name></name> tags (should be right near the top). As an added step you could also rename the disk file to reflect the change of the name and change the name of it in the <devices> section under <source file='/path/to/name_of_vm.img'>.
135
136
Save the XML file and undefine the old VM name with:
137
138
<pre><code class="bash">
139
virsh undefine name_of_vm
140
</code></pre>
141
142
Now just import the edited XML file to define the VM:
143
144
<pre><code class="bash">
145
virsh define name_of_vm.xml
146
</code></pre>
147
148
And that should be it! You can now start up your vm either in the Virtual Machine Manager or with virsh using:
149
150
<pre><code class="bash">
151
virsh start name_of_vm
152
</code></pre>
153
154
h2. set fixed IP-adr via DHCP (default-network)
155
156
taken from https://wiki.libvirt.org/page/Networking
157
158
<pre><code class="bash">
159
virsh edit <guest>
160
</code></pre>
161
162
where <guest> is the name or uuid of the guest. Add the following snippet of XML to the config file: 
163
164
<pre><code class="bash">
165
<interface type='network'>
166
  <source network='default'/>
167
  <mac address='00:16:3e:1a:b3:4a'/>
168
</interface>
169
</code></pre>
170
171
Applying modifications to the network
172
173
Sometimes, one needs to edit the network definition and apply the changes on the fly. The most common scenario for this is adding new static MAC+IP mappings for the network's DHCP server. If you edit the network with "virsh net-edit", any changes you make won't take effect until the network is destroyed and re-started, which unfortunately will cause a all guests to lose network connectivity with the host until their network interfaces are explicitly re-attached.
174
virsh net-update
175
176
Fortunately, many changes to the network configuration (including the aforementioned addition of a static MAC+IP mapping for DHCP) can be done with "virsh net-update", which can be told to enact the changes immediately. For example, to add a DHCP static host entry to the network named "default" mapping MAC address 53:54:00:00:01 to IP address 192.168.122.45 and hostname "bob", you could use this command: 
177
178
<pre><code class="bash">
179
virsh net-update default add ip-dhcp-host \
180
          "<host mac='52:54:00:00:00:01' \
181
           name='bob' ip='192.168.122.45' />" \
182
           --live --config
183
</code></pre>
184
185
h2. forwarding incoming connections
186
187
taken from https://wiki.libvirt.org/page/Networking
188
189
By default, guests that are connected via a virtual network with <forward mode='nat'/> can make any outgoing network connection they like. Incoming connections are allowed from the host, and from other guests connected to the same libvirt network, but all other incoming connections are blocked by iptables rules.
190
191
If you would like to make a service that is on a guest behind a NATed virtual network publicly available, you can setup libvirt's "hook" script for qemu to install the necessary iptables rules to forward incoming connections to the host on any given port HP to port GP on the guest GNAME:
192
193
1) Determine a) the name of the guest "G" (as defined in the libvirt domain XML), b) the IP address of the guest "I", c) the port on the guest that will receive the connections "GP", and d) the port on the host that will be forwarded to the guest "HP".
194
195
(To assure that the guest's IP address remains unchanged, you can either configure the guest OS with static ip information, or add a <host> element inside the <dhcp> element of the network that is used by your guest. See the libvirt network XML documentation address section for defails and an example.)
196
197
2) Stop the guest if it's running.
198
199
3) Create the file /etc/libvirt/hooks/qemu (or add the following to an already existing hook script), with contents similar to the following (replace GNAME, IP, GP, and HP appropriately for your setup):
200
201
Use the basic script below or see an "advanced" version, which can handle several different machines and port mappings here (improvements are welcome) or here's a python script which does a similar thing and is easy to understand and configure (improvements are welcome): 
202
203
<pre>
204
#!/bin/bash
205
# used some from advanced script to have multiple ports: use an equal number of guest and host ports
206
207
# Update the following variables to fit your setup
208
Guest_name=GUEST_NAME
209
Guest_ipaddr=GUEST_IP
210
Host_ipaddr=HOST_IP
211
Host_port=(  'HOST_PORT1' 'HOST_PORT2' )
212
Guest_port=( 'GUEST_PORT1' 'GUEST_PORT2' )
213
214
length=$(( ${#Host_port[@]} - 1 ))
215
if [ "${1}" = "${Guest_name}" ]; then
216
   if [ "${2}" = "stopped" ] || [ "${2}" = "reconnect" ]; then
217
       for i in `seq 0 $length`; do
218
               iptables -t nat -D PREROUTING -d ${Host_ipaddr} -p tcp --dport ${Host_port[$i]} -j DNAT --to ${Guest_ipaddr}:${Guest_port[$i]}
219
               iptables -D FORWARD -d ${Guest_ipaddr}/32 -p tcp -m state --state NEW -m tcp --dport ${Guest_port[$i]} -j ACCEPT
220
       done
221
   fi
222
   if [ "${2}" = "start" ] || [ "${2}" = "reconnect" ]; then
223
       for i in `seq 0 $length`; do
224
               iptables -t nat -A PREROUTING -d ${Host_ipaddr} -p tcp --dport ${Host_port[$i]} -j DNAT --to ${Guest_ipaddr}:${Guest_port[$i]}
225
               iptables -I FORWARD -d ${Guest_ipaddr}/32 -p tcp -m state --state NEW -m tcp --dport ${Guest_port[$i]} -j ACCEPT
226
       done
227
   fi
228
fi
229
</pre>
230
4) chmod +x /etc/libvirt/hooks/qemu
231
232
5) Restart the libvirtd service.
233
234
6) Start the guest.
235
236
(NB: This method is a hack, and has one annoying flaw in versions of libvirt prior to 0.9.13 - if libvirtd is restarted while the guest is running, all of the standard iptables rules to support virtual networks that were added by libvirtd will be reloaded, thus changing the order of the above FORWARD rule relative to a reject rule for the network, hence rendering this setup non-working until the guest is stopped and restarted. Thanks to the new "reconnect" hook in libvirt-0.9.13 and newer (which is used by the above script if available), this flaw is not present in newer versions of libvirt (however, this hook script should still be considered a hack). 
237
238
h2. wrapper script for virsh
239
240 2 Jeremias Keihsler
<pre>
241 1 Jeremias Keihsler
#! /bin/sh
242
# kvm_control   Startup script for KVM Virtual Machines
243
#
244
# description: Manages KVM VMs
245
# processname: kvm_control.sh
246
#
247
# pidfile: /var/run/kvm_control/kvm_control.pid
248
#
249
### BEGIN INIT INFO
250
#
251
### END INIT INFO
252
#
253 4 Jeremias Keihsler
# Version 20171103 by Jeremias Keihsler added ionice prio 'idle'
254 1 Jeremias Keihsler
# Version 20161228 by Jeremias Keihsler based on:
255
# virsh-specific parts are taken from:
256
#  https://github.com/kumina/shutdown-kvm-guests/blob/master/shutdown-kvm-guests.sh
257
# Version 20110509 by Jeremias Keihsler (vboxcontrol) based on:
258
# Version 20090301 by Kevin Swanson <kswan.info> based on:
259
# Version 2008051100 by Jochem Kossen <jochem.kossen@gmail.com>
260
# http://farfewertoes.com
261
#
262
# Released in the public domain
263
#
264
# This file came with a README file containing the instructions on how
265
# to use this script.
266
# 
267
# this is no more to be used as an init.d-script (vboxcontrol was an init.d-script)
268
#
269
270
################################################################################
271
# INITIAL CONFIGURATION
272
273
export PATH="${PATH:+$PATH:}/bin:/usr/bin:/usr/sbin:/sbin"
274
275
VIRSH=/usr/bin/virsh
276
TIMEOUT=300
277
278
declare -i VM_isrunning
279
280
################################################################################
281
# FUNCTIONS
282
283
log_failure_msg() {
284
echo $1
285
}
286
287
log_action_msg() {
288
echo $1
289
}
290
291
# list running domains
292
list_running_domains() {
293
  $VIRSH list | grep running | awk '{ print $2}'
294
}
295
296
# Check for running machines every few seconds; return when all machines are
297
# down
298
wait_for_closing_machines() {
299
RUNNING_MACHINES=`list_running_domains | wc -l`
300
if [ $RUNNING_MACHINES != 0 ]; then
301
  log_action_msg "machines running: "$RUNNING_MACHINES
302
  sleep 2
303
304
  wait_for_closing_machines
305
fi
306
}
307
308
################################################################################
309
# RUN
310
case "$1" in
311
  start)
312
    if [ -f /etc/kvm_box/machines_enabled_start ]; then
313
314
      cat /etc/kvm_box/machines_enabled_start | while read VM; do
315
        log_action_msg "Starting VM: $VM ..."
316
        $VIRSH start $VM
317
        sleep 20
318
        RETVAL=$?
319
      done
320
      touch /tmp/kvm_control
321
    fi
322
  ;;
323
  stop)
324
    # NOTE: this stops first the listed VMs in the given order
325
    # and later all running VM's. 
326
    # After the defined timeout all remaining VMs are killed
327
328
    # Create some sort of semaphore.
329
    touch /tmp/shutdown-kvm-guests
330
331
    echo "Try to cleanly shut down all listed KVM domains..."
332
    # Try to shutdown each listed domain, one by one.
333
    if [ -f /etc/kvm_box/machines_enabled_stop ]; then
334
      cat /etc/kvm_box/machines_enabled_stop | while read VM; do
335
        log_action_msg "Shutting down VM: $VM ..."
336
        $VIRSH shutdown $VM --mode acpi
337
        sleep 10
338
        RETVAL=$?
339
      done
340
    fi
341
    sleep 10
342 4 Jeremias Keihsler
343
    echo "give still running machines some more time..."
344
    # wait 20s per still running machine
345
    list_running_domains | while read VM; do
346
      log_action_msg "waiting 20s ... for: $VM ..."
347
      sleep 20
348
    done
349
350 1 Jeremias Keihsler
    echo "Try to cleanly shut down all running KVM domains..."
351
    # Try to shutdown each remaining domain, one by one.
352
    list_running_domains | while read VM; do
353
      log_action_msg "Shutting down VM: $VM ..."
354
      $VIRSH shutdown $VM --mode acpi
355
      sleep 10
356
    done
357
358
    # Wait until all domains are shut down or timeout has reached.
359
    END_TIME=$(date -d "$TIMEOUT seconds" +%s)
360
361
    while [ $(date +%s) -lt $END_TIME ]; do
362
      # Break while loop when no domains are left.
363
      test -z "$(list_running_domains)" && break
364
      # Wait a litte, we don't want to DoS libvirt.
365
      sleep 2
366
    done
367
368
    # Clean up left over domains, one by one.
369
    list_running_domains | while read DOMAIN; do
370
      # Try to shutdown given domain.
371
      $VIRSH destroy $DOMAIN
372
      # Give libvirt some time for killing off the domain.
373
      sleep 10
374
    done
375
376
    wait_for_closing_machines
377
    rm -f /tmp/shutdown-kvm-guests
378
    rm -f /tmp/kvm_control
379
  ;;
380
  export)
381
    JKE_DATE=$(date +%F)
382
    if [ -f /etc/kvm_box/machines_enabled_export ]; then
383
      cat /etc/kvm_box/machines_enabled_export  | while read VM; do
384
        rm -f /tmp/kvm_control_VM_isrunning
385
        VM_isrunning=0
386
        list_running_domains | while read RVM; do
387
          #echo "VM list -$VM- : -$RVM-"
388 4 Jeremias Keihsler
          if [[ "$VM" ==  "$RVM" ]]; then
389 1 Jeremias Keihsler
            #echo "VM found running..."
390
            touch /tmp/kvm_control_VM_isrunning
391
            VM_isrunning=1
392
            #echo "$VM_isrunning"
393
            break
394
          fi
395
          #echo "$VM_isrunning"
396 4 Jeremias Keihsler
        done
397 1 Jeremias Keihsler
398
        # took me a while to figure out that the above 'while'-loop 
399
        # runs in a separate process ... let's use the 'file' as a 
400
        # kind of interprocess-communication :-) JKE 20161229
401
        if [ -f /tmp/kvm_control_VM_isrunning ]; then
402
          VM_isrunning=1
403
        fi
404
        rm -f /tmp/kvm_control_VM_isrunning
405 4 Jeremias Keihsler
406 1 Jeremias Keihsler
        #echo "VM status $VM_isrunning"
407 4 Jeremias Keihsler
        if [ "$VM_isrunning" -ne 0 ]; then
408 1 Jeremias Keihsler
          log_failure_msg "Exporting VM: $VM is not possible, it's running ..."
409
        else
410
          log_action_msg "Exporting VM: $VM ..."
411
          VM_BAK_DIR="$VM"_"$JKE_DATE"
412
          mkdir "$VM_BAK_DIR"
413
          $VIRSH dumpxml $VM > ./$VM_BAK_DIR/$VM.xml
414
          $VIRSH -q domblklist $VM | awk '{ print$2}' | while read VMHDD; do
415
            echo "$VM hdd=$VMHDD"
416
            if [ -f "$VMHDD" ]; then
417 4 Jeremias Keihsler
              ionice -c 3 rsync --progress $VMHDD ./$VM_BAK_DIR/`basename $VMHDD`
418 1 Jeremias Keihsler
            else
419
              log_failure_msg "Exporting VM: $VM image-file $VMHDD not found ..."
420
            fi
421
          done
422
        fi
423
      done
424
    else
425
      log_action_msg "export-list not found"
426
    fi
427
  ;;
428
  start-vm)
429
    log_action_msg "Starting VM: $2 ..."
430
    $VIRSH start $2
431
    RETVAL=$?
432
  ;;
433
  stop-vm)
434
    log_action_msg "Stopping VM: $2 ..."
435
    $VIRSH shutdown $2 --mode acpi
436
    RETVAL=$?
437
  ;;
438
  poweroff-vm)
439
    log_action_msg "Powering off VM: $2 ..."
440
    $VIRSH destroy $2
441
    RETVAL=$?
442
  ;;
443
  export-vm)
444
    # NOTE: this exports the given VM
445
    log_action_msg "Exporting VM: $2 ..."
446
    rm -f /tmp/kvm_control_VM_isrunning
447
    VM_isrunning=0
448
    JKE_DATE=$(date +%F)
449
    list_running_domains | while read RVM; do
450
      #echo "VM list -$VM- : -$RVM-"
451 4 Jeremias Keihsler
      if [[ "$2" ==  "$RVM" ]]; then
452 1 Jeremias Keihsler
        #echo "VM found running..."
453
        touch /tmp/kvm_control_VM_isrunning
454
        VM_isrunning=1
455
        #echo "$VM_isrunning"
456
        break
457
      fi
458
      #echo "$VM_isrunning"
459 4 Jeremias Keihsler
    done
460 1 Jeremias Keihsler
461
    # took me a while to figure out that the above 'while'-loop 
462
    # runs in a separate process ... let's use the 'file' as a 
463
    # kind of interprocess-communication :-) JKE 20161229
464
    if [ -f /tmp/kvm_control_VM_isrunning ]; then
465
      VM_isrunning=1
466
    fi
467
    rm -f /tmp/kvm_control_VM_isrunning
468 4 Jeremias Keihsler
469 1 Jeremias Keihsler
    #echo "VM status $VM_isrunning"
470 4 Jeremias Keihsler
    if [ "$VM_isrunning" -ne 0 ]; then
471 1 Jeremias Keihsler
      log_failure_msg "Exporting VM: $VM is not possible, it's running ..."
472
    else
473
      log_action_msg "Exporting VM: $VM ..."
474
      VM_BAK_DIR="$2"_"$JKE_DATE"
475
      mkdir "$VM_BAK_DIR"
476
      $VIRSH dumpxml $2 > ./$VM_BAK_DIR/$2.xml
477
      $VIRSH -q domblklist $2 | awk '{ print$2}' | while read VMHDD; do
478
        echo "$2 hdd=$VMHDD"
479
        if [ -f "$VMHDD" ]; then
480 4 Jeremias Keihsler
          ionice -c 3 rsync --progress $VMHDD ./$VM_BAK_DIR/`basename $VMHDD`
481 1 Jeremias Keihsler
        else
482
          log_failure_msg "Exporting VM: $2 image-file $VMHDD not found ..."
483
        fi
484
      done
485
    fi
486
  ;;
487
  status)
488
    echo "The following virtual machines are currently running:"
489
    list_running_domains | while read VM; do
490
      echo -n "  $VM"
491
      echo " ... is running"
492
    done
493
  ;;
494
495
  *)
496
    echo "Usage: $0 {start|stop|status|export|start-vm <VM name>|stop-vm <VM name>|poweroff-vm <VM name>}|export-vm <VMname>"
497
    echo "  start      start all VMs listed in '/etc/kvm_box/machines_enabled_start'"
498
    echo "  stop       1st step: acpi-shutdown all VMs listed in '/etc/kvm_box/machines_enabled_stop'"
499 4 Jeremias Keihsler
    echo "             2nd step: wait 20s for each still running machine to give a chance to shut-down on their own"
500
    echo "             3rd step: acpi-shutdown all running VMs"
501
    echo "             4th step: wait for all machines shutdown or $TIMEOUT s"
502
    echo "             5th step: destroy all sitting VMs"
503 1 Jeremias Keihsler
    echo "  status     list all running VMs"
504
    echo "  export     export all VMs listed in '/etc/kvm_box/machines_enabled_export' to the current directory"
505
    echo "  start-vm <VM name>     start the given VM"
506
    echo "  stop-vm <VM name>      acpi-shutdown the given VM"
507
    echo "  poweroff-vm <VM name>  poweroff the given VM"
508
    echo "  export-vm <VM name>    export the given VM to the current directory"
509
    exit 3
510
esac
511
512
exit 0
513 4 Jeremias Keihsler
514 1 Jeremias Keihsler
</pre>
515 5 Jeremias Keihsler
516
h2. restore 'exported' kvm-machines
517
518
<pre><code class="shell">
519
tar xvf mach-name_202x-01-01.tar.gz 
520
</code></pre>
521 6 Jeremias Keihsler
522 7 Jeremias Keihsler
* copy the image-files to @/var/lib/libvirt/images/@
523
524
set ownership
525
<pre><code class="shell">
526
chown qemu:qemu /var/lib/libvirt/images/*
527
</code></pre>
528
529 6 Jeremias Keihsler
530
define the machine by
531
532
<pre><code class="shell">
533
virsh define mach-name.xml
534
</code></pre>
535 10 Jeremias Keihsler
536
h2. install Windows10 as a KVM guest
537
538
https://getlabsdone.com/10-easy-steps-to-install-windows-10-on-linux-kvm/