I was running a Xen server with Debian Etch as dom0 (
Linux 2.6.18-6 with
Xen 3.0.3-1 on
AMD64) for some time now. Today, I decided to upgrade the dom0 to Debian Lenny (
Linux 2.6.26-2 with
Xen 3.2.1-2). The domUs are all running a Debian-based OS (3x Lenny, 1x Ubuntu Hardy). The upgrade was quite straightforward, however there were some pitfalls you can avoid in advance.
1. Prepare domUs
After the upgrade I had problems with domUs which had no
udev installed, so better install it before upgrading:
aptitude install udev
It’s also neccessary to install the kernel modules for the new kernel for the VMs to work properly. On the VMs running Lenny I just installed them via apt, on the Ubuntu VM I copied them manually (see 7).
aptitude install linux-modules-2.6.26-2-xen-amd64
Shutdown all your virtual machines and delete all symlinks in
/etc/xen/auto to avoid booting of VMs after rebooting the dom0.
Edit all of your VM config files in
/etc/xen/*.cfg and change the following:
Edit the kernel and initrd image to match the new kernel we’ll use:
kernel = '/boot/vmlinuz-2.6.26-2-xen-amd64' ramdisk = '/boot/initrd.img-2.6.26-2-xen-amd64'
Add the following line:
extra = "console=hvc0 xencons=tty"
2. Edit apt sources
If not done so yet, update your
/etc/apt/sources.list on the dom0 to use Lenny. Mine looks like this:
# # Lenny # deb http://ftp.de.debian.org/debian/ lenny main contrib non-free deb-src http://ftp.de.debian.org/debian/ lenny main contrib non-free # # Security updates # deb http://security.debian.org/ lenny/updates main contrib non-free deb-src http://security.debian.org/ lenny/updates main contrib non-free
3. Run the upgrade
Now its time to run the dist upgrade on the dom0. Make sure you don’t overwrite your custom config files on upgrading. Just don’t accept all new configs but make sure your custom changes match with the new config files. If neccessary, go with the “Backup upgrade process” option and edit the new config files.
$ aptitude update $ aptitude upgrade $ aptitude dist-upgrade
4. Install Xen 3.2
$ aptitude install xen-hypervisor-3.2-1-amd64 xen-linux-system-2.6.26-2-xen-amd64 xen-utils-3.2-1 xenstore-utils xenwatch xen-shell xen-tools
After the installation edit
/boot/grub/menu.lst and make sure the new Xen kernel is booted by default. The simplest way to do this is to adjust the
default option (0 is the first in the kernel list). In my case, the right kernel was the second in the list, so I used
5. Edit xen-tools.conf
/etc/xen-tools/xen-tools.conf to make sure new VMs are created correctly.
Make sure the correct kernel is used:
kernel = /boot/vmlinuz-`uname -r` initrd = /boot/initrd.img-`uname -r`
Add the following lines:
serial_device = hvc0 disk_device = xvda
Reboot the dom0 and hope all worked correctly ;) After the reboot, check if you are running the new kernel:
maff@dom0:~$ uname -r 2.6.26-2-xen-amd64
Alright, so lets go on.
7. Copy kernel modules to VMs
Mount your VM disks and copy the new kernel modules to the virtual machine. I’m running the VMs on LVM partitions, so I can easily mount them. Do this for every virtual machine which doesn’t have the modules installed (see 1).
mkdir /mnt/vm01 mount /dev/lvmstore/vm01-disk /mnt/vm01 cp -a /lib/modules/`uname -r` /mnt/vm01/lib/modules/ umount /mnt/vm01 rm -rf /mnt/vm01
8. Boot VMs
Now try to start your VMs:
xm create /etc/xen/vm01.cfg
The VMs should start properly and work like before. It may be that you get some problems with an error like the following:
clocksource/0: Time went backwards: delta=-6917292717540055641 shadow=80182004054805 offset=400471
In this case, try rebooting the VMs (or destroy them from dom0 if they freeze on shutdown) and restarting them afterwards.