mafflog

I was running a Xen server with Debian Etch as dom0 (Linux 2.6.18-6 with Xen 3.0.3-1 on AMD64) for some time now. Today, I decided to upgrade the dom0 to Debian Lenny (Linux 2.6.26-2 with Xen 3.2.1-2). The domUs are all running a Debian-based OS (3x Lenny, 1x Ubuntu Hardy). The upgrade was quite straightforward, however there were some pitfalls you can avoid in advance.

1. Prepare domUs

After the upgrade I had problems with domUs which had no udev installed, so better install it before upgrading:

aptitude install udev

It’s also neccessary to install the kernel modules for the new kernel for the VMs to work properly. On the VMs running Lenny I just installed them via apt, on the Ubuntu VM I copied them manually (see 7).

aptitude install linux-modules-2.6.26-2-xen-amd64

Shutdown all your virtual machines and delete all symlinks in /etc/xen/auto to avoid booting of VMs after rebooting the dom0.

Edit all of your VM config files in /etc/xen/*.cfg and change the following:

Edit the kernel and initrd image to match the new kernel we’ll use:

kernel      = '/boot/vmlinuz-2.6.26-2-xen-amd64'
ramdisk     = '/boot/initrd.img-2.6.26-2-xen-amd64'

Add the following line:

extra = "console=hvc0 xencons=tty"

2. Edit apt sources

If not done so yet, update your /etc/apt/sources.list on the dom0 to use Lenny. Mine looks like this:

#
#  Lenny
#
deb     http://ftp.de.debian.org/debian/     lenny main contrib non-free
deb-src http://ftp.de.debian.org/debian/     lenny main contrib non-free

#
#  Security updates
#
deb     http://security.debian.org/ lenny/updates  main contrib non-free
deb-src http://security.debian.org/ lenny/updates  main contrib non-free

3. Run the upgrade

Now its time to run the dist upgrade on the dom0. Make sure you don’t overwrite your custom config files on upgrading. Just don’t accept all new configs but make sure your custom changes match with the new config files. If neccessary, go with the “Backup upgrade process” option and edit the new config files.

aptitude update
aptitude upgrade
aptitude dist-upgrade

4. Install Xen 3.2

aptitude install xen-hypervisor-3.2-1-amd64 xen-linux-system-2.6.26-2-xen-amd64 xen-utils-3.2-1 xenstore-utils xenwatch xen-shell xen-tools

After the installation edit /boot/grub/menu.lst and make sure the new Xen kernel is booted by default. The simplest way to do this is to adjust the default option (0 is the first in the kernel list). In my case, the right kernel was the second in the list, so I used default 1.

5. Edit xen-tools.conf

Edit /etc/xen-tools/xen-tools.conf to make sure new VMs are created correctly.

Make sure the correct kernel is used:

kernel = /boot/vmlinuz-`uname -r`
initrd = /boot/initrd.img-`uname -r`

Add the following lines:

serial_device = hvc0
disk_device = xvda

6. Reboot

Reboot the dom0 and hope all worked correctly ;) After the reboot, check if you are running the new kernel:

maff@dom0:~$ uname -r
2.6.26-2-xen-amd64

Alright, so lets go on.

7. Copy kernel modules to VMs

Mount your VM disks and copy the new kernel modules to the virtual machine. I’m running the VMs on LVM partitions, so I can easily mount them. Do this for every virtual machine which doesn’t have the modules installed (see 1).

mkdir /mnt/vm01
mount /dev/lvmstore/vm01-disk /mnt/vm01
cp -a /lib/modules/`uname -r` /mnt/vm01/lib/modules/
umount /mnt/vm01
rm -rf /mnt/vm01

8. Boot VMs

Now try to start your VMs:

xm create /etc/xen/vm01.cfg

The VMs should start properly and work like before. It may be that you get some problems with an error like the following:

clocksource/0: Time went backwards: delta=-6917292717540055641 shadow=80182004054805 offset=400471

In this case, try rebooting the VMs (or destroy them from dom0 if they freeze on shutdown) and restarting them afterwards.

Sources

• http://www.howtoforge.com/virtualization-with-xen-on-debian-lenny-amd64
• https://systemausfall.org/wikis/howto/XenUpgrade3.2
• http://www.ducea.com/2009/03/18/lenny-domu-xencons/