一、kvm虚拟机热添加技术

       热添加技术就是不停机的情况下，在线热添加硬盘，内存，cpu，网卡等设备，热添加技术一般都是在虚拟机资源不够了，又不能停机的情况下使用的，热添加技术是虚拟机相对于物理机的一个很大的优势，它让资源分配变得更灵活！

1.热添加硬盘

1.1. 创建硬盘

[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
1 web01 running

[root@kvm01 ~]# qemu-img create -f qcow2 /data/centos7-add01.qcow2 5G
Formatting '/data/centos7-add01.qcow2', fmt=qcow2 size=5368709120 encryption=off cluster_size=65536 lazy_refcounts=off

1.2  热添加硬盘

[root@kvm01 ~]# virsh attach-disk web01 /data/centos7-add01.qcow2 vdb --live --cache=none --subdriver=qcow2
Disk attached successfully

[root@kvm01 ~]# virsh console web01
Connected to domain web01
Escape character is ^]

CentOS Linux 7 (Core)
Kernel 3.10.0-514.el7.x86_64 on an x86_64

localhost login: root
Password:
Last login: Sat Sep 5 17:26:12 from 10.0.0.253
[root@localhost ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr0 11:0 1 1024M 0 rom
vda 253:0 0 10G 0 disk
└─vda1 253:1 0 10G 0 part /
vdb 253:16 0 5G 0 disk

1.3 对新添加的磁盘进行分区，格式化，挂载等操作

[root@localhost ~]# fdisk -l

Disk /dev/vda: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000047ba

Device Boot Start End Blocks Id System
/dev/vda1 * 2048 20971519 10484736 83 Linux

Disk /dev/vdb: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

[root@localhost ~]# fdisk /dev/vdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x1ef6234a.

Command (m for help): p

Disk /dev/vdb: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x1ef6234a

Device Boot Start End Blocks Id System

Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-10485759, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-10485759, default 10485759):
Using default value 10485759
Partition 1 of type Linux and of size 5 GiB is set

Command (m for help): p

Disk /dev/vdb: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x1ef6234a

Device Boot Start End Blocks Id System
/dev/vdb1 2048 10485759 5241856 83 Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@localhost ~]# mkfs.ext4 /dev/vdb1
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
327680 inodes, 1310464 blocks
65523 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

[root@localhost ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr0 11:0 1 1024M 0 rom
vda 253:0 0 10G 0 disk
└─vda1 253:1 0 10G 0 part /
vdb 253:16 0 5G 0 disk
└─vdb1 253:17 0 5G 0 part
[root@localhost ~]# mount /dev/vdb1 /mnt
[root@localhost ~]# ll /mnt
total 16
drwx------. 2 root root 16384 Sep 10 22:44 lost+found
[root@localhost ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 9.8G 1.1G 8.2G 11% /
devtmpfs 488M 0 488M 0% /dev
tmpfs 497M 0 497M 0% /dev/shm
tmpfs 497M 6.6M 490M 2% /run
tmpfs 497M 0 497M 0% /sys/fs/cgroup
tmpfs 100M 0 100M 0% /run/user/0
/dev/vdb1 4.8G 20M 4.6G 1% /mnt

2. 热添加网卡

[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
1 web01 running

2.1 热添加网卡

[root@kvm01 ~]# virsh attach-interface web01 --type network --model virtio --source default
Interface attached successfully

[root@kvm01 ~]# virsh console web01
Connected to domain web01
Escape character is ^]

[root@localhost ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:f4:2d:06 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.160/24 brd 192.168.122.255 scope global dynamic eth0
valid_lft 1935sec preferred_lft 1935sec
inet6 fe80::4ff4:9fdc:8840:a641/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:74:74:45 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.64/24 brd 192.168.122.255 scope global dynamic eth1
valid_lft 3593sec preferred_lft 3593sec
inet6 fe80::8e05:c2f9:46b6:e8a4/64 scope link
valid_lft forever preferred_lft forever

3.热添加cpu

3.1  添加cpu前，操作系统的个数

[root@kvm01 ~]# virt-install --virt-type kvm --os-type=linux --os-variant rhel6 --name web02 --memory 1024 --vcpus 1,maxvcpus=4 --disk /data/web02.qcow2,format=qcow2,size=10 --boot hd --network network=default --graphics vnc,listen=0.0.0.0 --noautoconsole

Starting install...
Domain creation completed.
[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
1 web01 running
2 web02 running

[root@kvm01 ~]# virsh console web02
Connected to domain web02
Escape character is ^]

Red Hat Enterprise Linux Server release 6.4 (Santiago)
Kernel 2.6.32-358.el6.x86_64 on an x86_64

localhost login: root
Password:
Last login: Sun Aug 30 22:50:01 from 10.0.0.253
[root@localhost ~]# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 61
model name : Intel Core Processor (Broadwell, IBRS)
stepping : 2
cpu MHz : 2711.998
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc up rep_good unfair_spinlock pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm 3dnowprefetch xsaveopt fsgsbase bmi1 avx2 smep bmi2 invpcid
bogomips : 5423.99
clflush size : 64
cache_alignment : 64
address sizes : 43 bits physical, 48 bits virtual
power management:

3.2 热添加cpu

[root@kvm01 ~]# virsh setvcpus web02 --count=2

 #查看系统cpu

[root@localhost ~]# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 61
model name : Intel Core Processor (Broadwell, IBRS)
stepping : 2
cpu MHz : 2711.998
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc rep_good unfair_spinlock pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm 3dnowprefetch xsaveopt fsgsbase bmi1 avx2 smep bmi2 invpcid
bogomips : 5423.99
clflush size : 64
cache_alignment : 64
address sizes : 43 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 61
model name : Intel Core Processor (Broadwell, IBRS)
stepping : 2
cpu MHz : 2711.998
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc rep_good unfair_spinlock pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm 3dnowprefetch xsaveopt fsgsbase bmi1 avx2 smep bmi2 invpcid
bogomips : 5423.99
clflush size : 64
cache_alignment : 64
address sizes : 43 bits physical, 48 bits virtual
power management:

4.热添加内存

4.1  添加内存前操作系统

[root@kvm01 ~]# virt-install --virt-type kvm --os-type=linux --os-variant rhel6 --name web03 --memory 512,maxmemory=2048 --vcpus=1,maxvcpus=2 --disk /data/web03.qcow2,format=qcow2,size=10 --boot hd --network network=default --graphics vnc,listen=0.0.0.0 --noautoconsole

Starting install...
Domain creation completed.
[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
1 web01 running
3 web03 running

[root@kvm01 ~]# virsh console web03
Connected to domain web03
Escape character is ^]

Red Hat Enterprise Linux Server release 6.4 (Santiago)
Kernel 2.6.32-358.el6.x86_64 on an x86_64

localhost login: root
Password:
Last login: Thu Sep 10 23:29:22 on ttyS0
[root@localhost ~]# free -m
total used free shared buffers cached
Mem: 341 132 209 0 6 35
-/+ buffers/cache: 90 250
Swap: 2015 0 2015

4.2 热添加内存

[root@kvm01 ~]# virsh setmem web03 1G

[root@localhost ~]# free -m
total used free shared buffers cached
Mem: 853 142 710 0 7 43
-/+ buffers/cache: 91 761
Swap: 2015 0 2015

注：内存已增加

二、kvm虚拟机热迁移

1. 什么是热迁移

    相比KVM虚拟机冷迁移中需要拷贝虚拟机虚拟磁盘文件，kvm虚拟机热迁移无需拷贝虚拟磁盘文件，但是需要迁移到的宿主机之间需要有相同的目录结构虚拟机磁盘文件，也就是共享存储，这次通过大家熟悉的nfs来实现，当然也可以采用Glusterfs等分布式文件系统来实现。

2.热迁移过程

   假设有2台宿主机kvm01和kvm02,在kvm01上挂起虚拟机vm01，发送vm的虚拟机配置文件和运行时内存中的数据到kvm02, 接受完毕，kvm02恢复vm01,热迁移完成

3. 热迁移架构图

4. 环境准备

5. 实战部署

5.1 安装nfs插件（kvm插件已安装）

[root@kvm01 ~]# yum install nfs-utils openssh-askpass -y

5.2 配置网桥

 [root@kvm01 ~]# virsh iface-bridge eth0 br0

 Created bridge br0 with attached device eth0

5.3 在nfs01上安装配置nfs

[root@nfs01 ~]# yum install nfs-utils -y

[root@nfs01 ~]# vim /etc/exports
[root@nfs01 ~]# cat /etc/exports
/data  10.0.0.0/24 (rw,sync)

[root@nfs01 ~]# mkdir /data/opt

[root@nfs01 ~]# chown -R nfsnobody:nfsnobody /data

[root@nfs01 ~]# systemctl restart rpcbind
[root@nfs01 ~]# systemctl restart nfs

[root@nfs01 ~]# showmount -e
Export list for nfs01:
/data 10.0.0.0/24

5.4 在kvm01和kvm02上挂载共享目录

[root@kvm01 ~]# showmount -e 10.0.0.31
Export list for 10.0.0.31:
/data 10.0.0.0/24

[root@kvm01 ~]# mount -t nfs 10.0.0.31:/data  /opt

[root@kvm01 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 48G 18G 31G 37% /
devtmpfs 2.0G 0 2.0G 0% /dev
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 2.0G 8.7M 2.0G 1% /run
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/sda1 1014M 116M 899M 12% /boot
tmpfs 394M 0 394M 0% /run/user/0
10.0.0.31:/data 18G 3.5G 15G 19% /opt

5.5 安装一台基于桥接模式的虚拟机

     将磁盘镜像文件web02.qcow2上传到10.0.0.31:/data目录下，可以看到kvm01和kvm02的/opt目录下都有此磁盘文件

[root@nfs01 ~]# ll /data
total 10487616
-rw-r--r-- 1 nfsnobody nfsnobody 10739318784 Sep 12 17:16 web02.qcow2

[root@kvm01 ~]# ll /opt
total 10487616
-rw-r--r-- 1 nfsnobody nfsnobody 10739318784 Sep 12 17:16 web02.qcow2

[root@kvm02 ~]# ll /opt
total 10487616
-rw-r--r-- 1 nfsnobody nfsnobody 10739318784 Sep 12 17:16 web02.qcow2

#在kvm01上安装一台虚拟机

[root@kvm01 ~]# virt-install --virt-type kvm --os-type=linux --os-variant rhel6 --name web02 --memory 512,maxmemory=2048 --vcpus 1 --disk /opt/web02.qcow2 --boot hd --network bridge=br0 --graphics vnc,listen=0.0.0.0 --noautoconsole

Starting install...
Domain creation completed.
[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
2 web02 running
- web01 shut off
- web03 shut off

5.5 将宿主机10.0.0.131上的kvm虚拟机web03迁移到10.0.0.7（命令行迁移）

[root@kvm01 ~]# virsh migrate --live --verbose web02 qemu+ssh://10.0.0.7/system --unsafe
root@10.0.0.7's password:
Migration: [100 %]

[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
- web01 shut off
- web02 shut off
- web03 shut off

#在kvm02上查看

[root@kvm02 ~]# virsh list --all
Id Name State
----------------------------------------------------
1 web02 running

[root@kvm02 ~]# virsh console web02
Connected to domain web02
Escape character is ^]

Red Hat Enterprise Linux Server release 6.4 (Santiago)
Kernel 2.6.32-358.el6.x86_64 on an x86_64

localhost login: root
Password:
Last login: Sun Aug 30 22:50:01 from 10.0.0.253
[root@localhost ~]# uname -a
Linux localhost 2.6.32-358.el6.x86_64 #1 SMP Tue Jan 29 11:47:41 EST 2013 x86_64 x86_64 x86_64 GNU/Linux

至此迁移成功！！！！

5.6 在kvm01上安装图形界面、vnc服务端和virt-manager（工具迁移）

[root@kvm01 ~]# yum groups install "GNOME Desktop" -y

[root@kvm01 ~]# yum install tigervnc-server  -y

[root@kvm01 ~]# yum install virt-manager -y

5.7 启动vnc服务端

vncserver :1 启动5901端口的vnc服务端

[root@kvm01 ~]# vncserver :1

You will require a password to access your desktops.

Password:
Verify:
Would you like to enter a view-only password (y/n)? n
A view-only password is not used

New 'kvm01:1 (root)' desktop is kvm01:1

Creating default startup script /root/.vnc/xstartup
Creating default config /root/.vnc/config
Starting applications specified in /root/.vnc/xstartup
Log file is /root/.vnc/kvm01:1.lo

vncserver -kill :1 关闭5901端口的vnc服务端

[root@kvm01 ~]# vncserver -kill :1
Killing Xvnc process ID 1656

[root@kvm01 ~]# vncserver :1

New 'kvm01:1 (root)' desktop is kvm01:1

Starting applications specified in /root/.vnc/xstartup
Log file is /root/.vnc/kvm01:1.log

[root@kvm01 ~]# netstat -lntup
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:5901 0.0.0.0:* LISTEN 2150/Xvnc

5.8 使用vnc连接宿主机，使用virt-manager进行迁移

#迁移前

[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
2 web02 running
- web01 shut off
- web03 shut off

[root@kvm02 ~]# virsh list --all
Id Name State
----------------------------------------------------

 #vnc登录，实施迁移

迁移完成！！！

查看kvm01和kvm02虚拟机状况

[root@kvm01 ~]# virsh list --all
Id Name State
----------------------------------------------------
- web01 shut off
- web03 shut off

[root@kvm02 ~]# virsh list --all
Id Name State
----------------------------------------------------
2 web02 running

[root@kvm02 ~]# virsh console web02
Connected to domain web02
Escape character is ^]

Red Hat Enterprise Linux Server release 6.4 (Santiago)
Kernel 2.6.32-358.el6.x86_64 on an x86_64

localhost login: root
Password:
Last login: Sat Sep 12 17:22:59 on ttyS0
[root@localhost ~]# uname -a
Linux localhost 2.6.32-358.el6.x86_64 #1 SMP Tue Jan 29 11:47:41 EST 2013 x86_64 x86_64 x86_64 GNU/Linux