Cyclops was originally setup to be a failover for the primary hypervisor (Wolverine) however after realizing that I should probably cover the babysteps of KVM before jumping into high availability, I just set it up as normal hypervisor. It has the following specs:
Dell R510 Server 2 (Cyclops)
- Primary Purpose: Hypervisor
- CPU: Dual L5520 Xeon CPUs (quad core)
- RAM: 32GB DDR3 ECC Registered
- First RAIDset: Two 160GB SATA drives in RAID1 (Operating System)
- Ethernet: Single NIC (em2) for storage network
- Ethernet: Single NIC (em1) for management network
- Ethernet: Single NIC for Dell iDRAC/IPMI
The volumes on the server as setup in the following way:
Operating system – CentOS 6.4 x64
First drive (/dev/sda) – This is a RAID1 stripe of the 160GB drives
Primary partition (dev/sda1) – the /boot partition which is just a 100MB
Second partition (/dev/sda2) – the swap partition which is 8GB
Third partition (/dev/sda3) – the / partition which is where the operating system is loaded
Configuring the ethernet ports
Networking in CentOS/RedHat (via commandline at least) is found in /etc/sysconfig/network-scripts/ folder location. The NICs will be labeled anything from eth0 and eth1 to em1/em2 or if you have PCI devices (added nic cards) will show up as p1p1 and p1p2 (or whichever PCI slots they are located in).
My particular em1 setup (which is located at /etc/sysconfig/network-scripts/ifcfg-em1) is:
DEVICE=em1 HWADDR=A4:BA:DB:10:D2:98 TYPE=Ethernet UUID=76447f85-4c38-4fd2-97a2-cf797c69a813 ONBOOT=yes NM_CONTROLLED=no BOOTPROTO=static IPADDR=192.168.2.3 NETMASK=255.255.255.0 NETWORK=192.168.2.0 BROADCAST=192.168.2.255 GATEWAY=192.168.2.1 DNS1=18.104.22.168
Similarly my em2 setup is:
DEVICE=em2 HWADDR=A4:BA:DB:10:D2:98 TYPE=Ethernet UUID=f34dee23-d32e-4247-a3f1-3d68f5f3caae ONBOOT=yes NM_CONTROLLED=no BOOTPROTO=static IPADDR=10.0.0.3 NETMASK=255.255.255.0 NETWORK=10.0.0.0
The fields are pretty self-explanatory. The reason that em2 does not have a gateway is that you generally only want one gateway per machine (unless you have more complex routing rules, but that is easier done at the router level, although you can create custom routes anywhere). More on this in the networking section. So in the above setup, I have one NIC that goes to the storage network (em2) and one NIC that goes to the management network. They don’t crosstalk with each other since they are on different vlans and in reality do not know about each other.
Installing KVM on Cyclops
Installing KVM on CentOS is pretty easy (as it is on most linux distributions with any kind of package manager). In my case, we run the following commands:
rpm –import /etc/pki/rpm-gpg/RPM-GPG-KEY* # This just permits downloading from the CentOS repos without asking for permission each time
yum install kvm libvirt python-virtinst qemu-kvm bridge-utils # This installs the required packages. You can install the virtualization group, but that installs other packages as well, and I was shooting for the most simple setup possible (to start at least).
One important note here is that you need to setup a virtual bridge before you can start building machines. Browse into /etc/sysconfig/network-scripts and create a file called ifcfg-br0 (or anything else you want to name it really, I just keep it simple; you can call it ifcfg-virtualbridge if you like). Modify this file so that it contains (in my case at least):
DEVICE="br0" NM_CONTROLLED="no" ONBOOT=yes TYPE=Bridge BOOTPROTO=none IPADDR=192.168.2.3 PREFIX=24 GATEWAY=192.168.2.1 DNS1=22.214.171.124 DNS2=126.96.36.199 DEFROUTE=yes IPV4_FAILURE_FATAL=yes IPV6INIT=no NAME="System br0"
I know what you’re thinking: this has the same IP as em1! This will cause networking fail if you restart! That is correct, so we will need to modify the ifcfg-em1 file so that it is a member of the bridge above. We will modify our /etc/sysconfig/network-scripts/ifcfg-em1 file to:
DEVICE=em1 HWADDR=A4:BA:DB:10:D2:98 TYPE=Ethernet UUID=76447f85-4c38-4fd2-97a2-cf797c69a813 ONBOOT=yes NM_CONTROLLED=no #BOOTPROTO=static #IPADDR=192.168.2.3 #NETMASK=255.255.255.0 #NETWORK=192.168.2.0 #BROADCAST=192.168.2.255 #GATEWAY=192.168.2.1 #DNS1=188.8.131.52 BRIDGE=br0
What I did above was comment out the IP information in em1, and add the line BRIDGE=br0. This makes the interface a member of the bridge we just created. This way once the virtual machines are created, all their IP addresses will be created as part of the bridge above. To reactivate all the changes, just type service network restart. This will regenerate all the ethernet ports and network connections.
Firing it all up!
The last steps will be below:
chkconfig libvirtd on # Make sure that the KVM virtualization engine starts on boot.
chkconfig iptables off # This is ill-advised for production use. Since this is for my home lab, I can do this. This stops the firewall. In the networking section, we will re-enable iptables and add the proper rules so that qemu and libvirt can be accessed properly.
service libvirtd start # This starts the virtualization service.
ps -ef | grep libvirt # Make sure you can libvirt as a running process.
And voila! We now have a fully running hypervisor. The configuration for Wolverine (our second hypervisor) is similar, but slightly different as it uses bonds (multiple NICs bonded together as a single interface, and configured on the network switch using LACP). The configuration for that server will be found here (once I create that page :-))
This will be a moving target, but the servers I anticipate running on the Cyclops hypervisor are:
- Elektra – Ubuntu 12.04 Server – MySQL and PostGRES SQL database server
- Colossus – CentOS 6.4 Server – Rsync backup server
- Kingpin – Windows Server 2012 – Microsoft SQL Server machine that will power the Microsoft apps
- Blade – Windows Server 2012 – Microsoft Sharepoint Server
- Apocalypse – Windows Server 2012 – Microsoft Exchange Server 2013
- Venom – Windows Server 2012 – Microsoft System Center