On this page, you will find a howto regarding the installation of Xen on a hetzner server.
Preliminary Setup
Boot using the rescue system
First thing first, we need to boot our system from the rescue system, in order to do the installation process. Activate that feature using the robot (Leistungsuebersicht > Rescuesystem) and reboot.
Partitioning
If everything went smoothly and that we are now in the rescue system, we are ready to begin the installation.
Create the partitions on the 2 hard-disks using cfdisk. The setup we have used is the following. The first partition is for /boot, the second for / and the third one for swap and the fourth for the data.
sda1 Primary Linux raid autodetect 98.71 sda2 Boot Primary Linux raid autodetect 10051.30 sda3 Primary Linux swap / Solaris 1019.94 sda4 Primary Linux raid autodetect 148869.35
The Linux raid autodetect type has the number fd within cfdisk.
Raid Creation
Create the 3 raid partitions on the 2 disks, using the mdadm command.
mdadm --create /dev/md0 -n 2 -l 1 /dev/sda1 /dev/sdb1 mdadm --create /dev/md1 -n 2 -l 1 /dev/sda2 /dev/sdb2 mdadm --create /dev/md2 -n 2 -l 1 /dev/sda4 /dev/sdb4
Remarks:
Use missing at the place of a device for partial array construction. This is useful e.g. for creating the raid partition first, copying some data on it and adding the second device afterwards.
A device can be added later using the following syntax: mdadm --add /dev/md0 /dev/sda1 (we add to the raid device /dev/md0 the physical partition /dev/sda1.
A device can be started using mdadm --assemble <raid device>
A device can be stopped using mdadm --stop <raid device>
More information about a raid device, e.g. which physical drive is currently part of the array, using mdadm --detail <raid device>
Raid Check
Now we can check that everything went smoothly regarding the RAID array, and that the raid partitions are currently in sync, by inspecting the file /proc/mdstat:
~# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdb1[1] sda1[0]
58596992 blocks [2/2] [UU]
md1 : active raid1 sda2[0] sdb2[1]
1020032 blocks [2/2] [UU]
md2 : active raid1 sda3[0] sdb3[1]
5124608 blocks [2/2] [UU]
unused devices: <none>
Filesystem Creation
Create the filesystem (ext3 format) along with the swap
mkfs.ext3 /dev/md0 mkfs.ext3 /dev/md1 mkfs.ext3 /dev/md2 mkswap /dev/sda3 mkswap /dev/sdb3
Remark:
We have not used the software RAID for the two swap partitions. Not sure if it would be useful. It could perhaps speed up slightly the access time for read operations but would slightly increase it for write operations.
Debootstrapping Debian Sarge
Alright. So now we have got the partitions set up. We now need to install the operating system itself. For that we will use the debootstrap command.
Mount Point
Create a mount point within the rescue system and mount our newly created partitions
mkdir /mnt mkdir /mnt/md1 mount /dev/md1 /mnt/md1 mkdir /mnt/md1/boot mount /dev/md0 /mnt/md1/boot/
debootstrapping
Use the following command to install a base debian sarge system on partition /dev/md1
debootstrap sarge /mnt/md1/ ftp://ftp2.de.debian.org/debian
Now we can do a chroot inside the mount folder before continuing with the installation
mount --bind /proc/ /mnt/md1/proc mount --bind /sys /mnt/md1/sys mount --bind /dev /mnt/md1/dev chroot /mnt/md1/
Configure some stuffs like the timezone, root password and hostname using the base-config command.
base-config
Set Up APT and Update the System
Set the /etc/apt/sources.list file as follows.
# stable deb http://ftp.de.debian.org/debian sarge main non-free contrib deb-src http://ftp.de.debian.org/debian sarge main non-free contrib # security deb http://security.debian.org/ stable/updates main contrib non-free # non-us deb http://non-us.debian.org/debian-non-US stable/non-US main contrib non-free
Let's update the system now and install a few useful stuffs.
apt-get update apt-get dist-upgrade apt-get install less wget tree vim nano bzip2 mdadm ssh initrd-tools raidtools2
During the configuration of mdadm, specify not to start the RAID devices automatically, if the following point applies.
If there were precedently old raid arrays, get rid of them using the following command for each (non-raid) partition, e.g. /dev/sda1.
# for each partition mdadm --zero-superblock /dev/<device>
Reconfigure now mdadm with automatic RAID device start
dpkg-reconfigure mdadm
The Kernel
In this tutorial, we have used a precompiled kernel.
We take advantage of the fact that mkinitrd is launched during kernel image install. As we need some specific modules to be included in the initrd, we specify them directly now.
Edit /etc/mkinitrd/modules and add the following module names:
ext3 jbd via82cxxx libata scsi_mod sd_mod sata_via raid1 md
Run the following command (otherwise mkinitrd complains)
mount -tproc none /proc
and edit /etc/mkinitrd/mkinitrd.conf:
Change ROOT=probe to ROOT="/dev/md0 ext3".
We are now ready to install the kernel image. This will also automatically generate the initrd image. (The image for AMD64 is not a good candidate, as some bugs with iptables have been reported.)
apt-get install kernel-image-2.6.8-3-k7-smp
Remark:
In case it is necessary (it shouldn't be in the normal case, as this is done automatically when installing the kernel), the command to re-generate the initrd image is the following one, specifying the right initrd file and kernel version:
mkinitrd -o initrd.img-2.6.8-3-k7-smp 2.6.8-3-k7-smp
Network Configuration
/etc/network/interfaces
Configure the network by entering the necessary information in the aforementioned file.
# /etc/network/interfaces # Loopback device: auto lo iface lo inet loopback # static conf auto eth0 iface eth0 inet static address <ip addr> broadcast <broadcast addr> netmask <netmask> gateway <gateway addr>
Where <ip addr>, <broadcast addr>, <netmask> and <gateway addr> should be set according to your network configuration.
Kernel Module
Add r8169 to the /etc/modules file, for network card module.
Notes: this is specific to the Hetzner DS3000 hardware
perhaps you need 8139too module
/etc/hosts
Define the hosts file:
echo "127.0.0.1 localhost" > /etc/hosts
The network should be now properly configured.
The Bootloader
Install Grub with support for a separate boot partition and software raid1. I you have trouble read the error list on a gentoo documentation site.
Grub installation
apt-get install grub
Install Grub on our 2 partitions
grub-install /dev/sda grub-install /dev/sdb
Rem: Is this necessary? Rem: please correct, extend if necessary
Setup Grub correctly on the boot partition (it should recognize the /boot/grub/ directory)
grub > root (hd0,0) > setup (hd0) > root (hd1,0) > setup (hd1)
Configure grub
Let's populate /boot/grub/menu.lst with the following content:
timeout 1 # which config to boot from default 1 fallback 0 # using boot partition from 1st disk title Linux (hd0,0) kernel (hd0,0)/vmlinuz-2.6.8-3-k7-smp root=/dev/md1 initrd (hd0,0)/initrd.img-2.6.8-3-k7-smp # using boot partition from 2nd disk title Linux (hd1,0) kernel (hd1,0)/vmlinuz-2.6.8-3-k7-smp root=/dev/md1 initrd (hd1,0)/initrd.img-2.6.8-3-k7-smp
Remark:
We use grub, because Lilo is presently not compatible with XEN.
fstab
Define fstab as follows, adapted to your partitionning scheme.
proc /proc proc defaults 0 0 none /dev/pts devpts gid=5,mode=620 0 0 /dev/md0 /boot ext3 defaults 0 0 /dev/md1 / ext3 defaults 0 0 /dev/md2 /data ext3 defaults 0 0 /dev/sda3 none swap sw 0 0 /dev/sdb3 none swap sw 0 0
and create the folder for the data partition, if not already done:
mkdir /data
mdadm.conf
Define and populate the mdadm.conf file.
echo "DEVICE /dev/sda* /dev/sdb*" > /etc/mdadm/mdadm.conf mdadm --detail --scan >> /etc/mdadm/mdadm.conf echo "MAILADDR meine@emailadresse.de" >> /etc/mdadm/mdadm.conf
Reboot
Pray
Type exit and reboot
XEN Installation
Grub Configuration
Extend /etc/grub/menu.lst for support of the XEN kernel (two last paragraphs added):
timeout 1 # which config to boot from default 2 fallback 0 # using boot partition from 1st disk title Linux (hd0,0) kernel (hd0,0)/vmlinuz-2.6.8-3-k7-smp root=/dev/md1 initrd (hd0,0)/initrd.img-2.6.8-3-k7-smp # using boot partition from 2nd disk title Linux (hd1,0) kernel (hd1,0)/vmlinuz-2.6.8-3-k7-smp root=/dev/md1 initrd (hd1,0)/initrd.img-2.6.8-3-k7-smp # xen kernel on 1st disk title XEN (hd0,0) kernel (hd0,0)/xen-3.0-i386.gz dom0_mem=128000 module (hd0,0)/vmlinuz-2.6.16-1-xen-k7 root=/dev/md1 module (hd0,0)/initrd.img-2.6.16-1-xen-k7 # xen kernel on 2nd disk title XEN (hd1,0) kernel (hd1,0)/xen-3.0-i386.gz dom0_mem=128000 module (hd1,0)/vmlinuz-2.6.16-1-xen-k7 root=/dev/md1 module (hd1,0)/initrd.img-2.6.16-1-xen-k7
Xen
Forum thread: http://de.xen-forum.net/viewtopic.php?t=144
Tutorial: http://www.huschi.net/16_185_de.html
Other nice tutorial: http://wiki.kartbuilding.net/index.php/XenDebianBackports
Follow the instructions in the tutorial. For the kernel, install the linux-image-2.6.16-1-xen-k7 at the place of the one mentioned.
xen page: http://www.xensource.com/xen/xen/
Tutorial that actually did work (drawback: using a binary version of xen, but otherwise the setup is really cool): http://www.howtoforge.com/xen_3.0_ubuntu_dapper_drake
TODO:
Extend this part.
domU Creation
In this section we will install and setup a small Ubuntu Edgy (6.10) guest system using the debootstrap method similar as seen above.
File system
The user of sparse files for disk image is describe in the xen documentation (6.2) and under http://intgat.tigress.co.uk/rmy/uml/sparsify.html. This has the advantage that the file on disk will take just the space needed (e.g. a few GB) and not the maximal size of the file (in our example 30 GB).
The command we use to create a sparse image of 30 GB along with another one of 500 MB for the swap, is the following:
dd if=/dev/zero of=/xen-images/edgy_base.img bs=1k seek=30M count=1 dd if=/dev/zero of=/xen-images/edgy_base.swap bs=1k seek=500k count=1
We have now to create the file system within the files and to mount it:
cd /xen-images mkfs.ext3 edgy_base.img mkswap edgy_base.swap mkdir mnt mount -o loop edgy_base.img mnt
Debootstrap Edgy as DomU
Launch the debootstrapping
debootstrap edgy mnt http://archive.ubuntu.com/ubuntu
Setup apt. For this replace mnt/etc/apt/source.list by the following file, or see http://www.ubuntu-nl.org/source-o-matic/ for a custom generated source.list.
## I want the main rep. along with universe and multiverse deb http://ch.archive.ubuntu.com/ubuntu/ edgy main restricted universe multiverse deb-src http://ch.archive.ubuntu.com/ubuntu/ edgy main restricted universe multiverse ## Major bug fix updates produced after the final release of the ## distribution. deb http://ch.archive.ubuntu.com/ubuntu/ edgy-updates main restricted universe multiverse deb-src http://ch.archive.ubuntu.com/ubuntu/ edgy-updates main restricted universe multiverse ## We don't want the backports # deb http://ch.archive.ubuntu.com/ubuntu/ edgy-backports main restricted universe multiverse # deb-src http://ch.archive.ubuntu.com/ubuntu/ edgy-backports main restricted universe multiverse ## security deb http://ch.archive.ubuntu.com/ubuntu edgy-security main restricted universe multiverse deb-src http://ch.archive.ubuntu.com/ubuntu edgy-security main restricted universe multiverse
Setup the network, that is, the file mnt/etc/network/interfaces according to your network config. In my case:
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.1.101
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
gateway 192.168.1.1
and mnt/etc/hosts
127.0.0.1 localhost 192.168.1.101 <hostname> # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ff02::3 ip6-allhosts
along with mnt/etc/hostname
<hostname>
Edit domU's /etc/fstab to use the swap-file and Xen block device (i.e. /dev/hda3).
# <fs> <mountpoint> <type> <opts> <dump/pass> proc /proc proc defaults 0 0 /dev/hda1 / ext3 defaults,errors=remount-ro 0 1 /dev/hda2 none swap sw 0 0 /dev/hda3 /data ext3 defaults,errors=remount-ro 0 1
umount your domU and copy your kernel modules to domU.
cp -r /lib/modules/ <path to your dom>/lib/modules/
set its config file, and create it
cd /xen-images umount mnt
The xen config file for my domU is the following
# -*- mode: python; -*-
kernel = "/boot/vmlinuz-2.6.16.29-xen"
ramdisk = "/boot/initrd.img-2.6.16.29-xen"
memory = 400
name = "public"
vif = ['bridge=xenbr0']
disk = ['file:/xen-images/public.img,hda1,w',
'file:/xen-images/public.swap,hda2,w']
root = "/dev/hda1 ro"
extra = "4"
At this stage, if you have a dom-0 with two network interfaces, that you have adapted the Xen scripts to create two bridges, you can connect your domU specifically to the second bridge by specifying for instance vif = ['bridge=xenbr1']. More info about Xen networking in this page.
Now we can create our domU
xm create -c <name>
where <name> is the name of the config file inside /etc/xen. We can now go on with a few config stuffs within our domU.
Remarks:
TODO: extend
more information here: https://help.ubuntu.com/community/XenOnUbuntuBinaryInstall under Installing DomU w/ debootstrap
Loop Devices
If you are using files for storing the different filesystems of your xen domUs, it is likely that you will quickly, after creating 3 or 4 domUs, run out of loop devices (by default, there is just 8 of them).
If this is the case, you will get an error message of the following type when creating an additional domU:
Error: Device 769 (vbd) could not be connected. Backend device not found.
A workaround for that under debian is to create a file /etc/modprobe.d/xen with the following content:
# xen workaround - define enough loop devices options loop max_loop=128
I had also additionnaly to put loop into /etc/modules. Not sure if this is generally needed.
Several Network Interfaces
If you have more than 1 interface card on you server and that you would like to bind the host system (dom-0) to one card and one domU to another one, here is how to do.
First we need to create a small script inside /etc/xen/script called for example 2-int-network-script:
#!/bin/sh dir=$(dirname "$0") "$dir/network-bridge" "$@" vifnum=0 "$dir/network-bridge" "$@" vifnum=1
Then we need to change just one line in the xend config file /etc/xen/xend-config.sxp. We need to replace the line showing
(network-script network-dummy)
by the following one. This will activate the bridge between the dom-0 and the domUs using the two interfaces.
(network-script 2-int-network-script)
Now, within the domU configuration file, we can specify that we want to bind the guest to a single bridge interface. For that, we add the following line (and replace a corresponding existing line):
vif = ['bridge=xenbr1']
In the latter case, we would bind our domU to the second bridge only (the first being xenbr0).
Tipps & Tricks
List running domains: xm list
Start a server: xm create <domname>
Connect to the console of some domain: xm console <domname>
Exit the console: CTRL + ']'
Shutdown a domain: xm shutdown <domname>
Related Links
The present tutorial is based upon the following ones:
Xen on Ubuntu using a binary install:
Xen on Ubuntu Feisty Fawn is very easy. The server distribution comes already with xen packages. sudo apt-get install ubuntu-xen-server and you have a server running with xen on Dom0
Some info regarding the XEN network configuration. Useful in particular if you want a special config with two physical network interfaces:
Might be useful to mount an initrd image
Mount a cramfs image: http://www.applieddata.net/forums/topic.asp?TOPIC_ID=1636
Contact
Feel free to contact me for any comment or suggestion (did I mention bug report? 
).
pajai (a_t) extrabright.com