Routed XEN VM based on LVM (step-by-step for Hetzner DS3000)

Author: Sergey Kuznetsov kuznecov.sg@gmail.com, February 2010
License: CC BY-SA (Creative Commons Attribution-ShareAlike)

Reset server to default

Go to https://robot.your-server.de/server, "Rescue" tab, choose "Linux" and "64 bit" option, save the password. Next go to "Reset" tab and "Send CTRL+ALT+DEL to the server". In 2 minutes you can connect via SSH to IP address. User name will be "root" and password is the saved at the "rescue" tab password.

After logging into system you will see something like this initial screen, and we can just type "installimage":

[...]
Hardware data:
   CPU Athlon 64 X2 Dual Core 4200+
   CPU Athlon 64 X2 Dual Core 4200+
   RAM 1946 MB
   Disk /dev/sda: 320.0 GB (=> 305 GIB)
   Disk /dev/sdb: 320.0 GB (=> 305 GIB)
root@rescue ~ # installimage

From the ncurses menu we need to choose the image system for installing. My choice is:

Debian -> Debian-50-lenny-64-minimal

After that we see the install configuration file which can be edited before installation. We need to check some features and change some parts of it.

SWRAID 1 # check this!
SWRAIDLEVEL 1 # check this!
[...]
HOSTNAME root.summercode.ru
[...]
PART swap swap 4G
PART / ext3 10G
PART /data ext3 all

Here we give 4Gb for the swap partition, 10Gb for the Dom0 root partition, all of the free space we give for /data directory, which will be converted to the LVM after installing host system.

Ok. [F10]. [Yes]. [Yes].

Hetzner Online AG - installimage
Your server will be installed now, this will take some minutes
You can abort at any time with CTRL+C ...

      : Reading configuration done
 1/13 : Deleting partitions done
 2/13 : Creating partitions and /etc/fstab done
 3/13 : Creating software RAID level 1 done
 4/13 : Formatting partitions 
      : formatting /dev/md0 with swap done
      : formatting /dev/md1 with ext3 done 
      : formatting /dev/md2 with ext3 done
 5/13 : Mounting partitions done
 6/13 : Extracting image (local) done
 7/13 : Setting up network for eth0 done
 8/13 : Executing commands inside chroot done
 9/13 : Setting up miscellaneous files done
10/13 : Setting root password done
11/13 : Installing bootloader grub done
12/13 : Running some debian specific functions done
13/13 : Clearing log files done

INSTALLATION COMPLETE
You can now reboot and log in to your new system with the same
password as you logged in to the rescue system.
root@rescue ~ #

Now we can just reboot to our new server system.

root@rescue ~ # reboot

After reboot

After logging in system (using "root" and saved from "Rescue" tab password) first of all change the root password for the security reasons:

root:~# passwd
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully

Then we will need to update our new server system:

root:~ # apt-get update
root:~ # apt-get dist-upgrade

Creating LVM

Try to install lvm2 package if it is not installed yet:

root:~# apt-get install lvm2

Unmount our future VMs volume:

root:~# umount /data/

Remove line, started with "/dev/md2..." from /etc/fstab. Create physical volume:

root:~# pvcreate /dev/md2
   Physical volume "/dev/md2" successfully created

Create volume group on it:

root:~# vgcreate vgvm /dev/md2
   Volume group "vgvm" successfully created

Request additional IP subnet

Before next part we need to go to https://robot.your-server.de/server, choose you server (click on it), and choose the IP tab. At the bottom we can see "Ordering of additional IP, failover IP or subnet" link. Here we need to order additional subnet for our server. IP-addresses and configuration we will use at the next part of configuring our XEN environment. After some little time you will receive an email which gives you information about your additional subnet, for example it may be like this:

IP: 78.46.242.88
Mask: 255.255.255.248
Broadcast: 78.46.242.95

XEN Installation and Configuration

Next we need to install xen system and tools:

root:~# aptitude install xen-hypervisor xen-linux-system-2.6.26-2-xen-amd64 xen-utils xenstore-utils xenwatch xen-shell xen-tools

Change the /etc/modules file:

[...]
loop max_loop=64
Change the /etc/xen/xend-config.sxp file - we search and comment lines "(network-script network-dummy)" and "(vif-script vif-bridge)", and then add this two lines (for Debian Lenny we use netdev=eth0):
[...]
(network-script 'network-route netdev=eth0')
(vif-script vif-route)
[...]

Next configuration file is"/etc/xen-tools/xen-tools.conf", here we make following changes:

[...]
##
#
# If you don't wish to use loopback images then you may specify an
# LVM volume group here instead
#
##
lvm = vgvm
# here we write the volume group we created early by "vgreate" command

[...]

##
# Disk and Sizing options.
##
#
size = 32Gb # Disk image size.
memory = 256Mb # Memory size
swap = 1024Mb # Swap size
# noswap = 1 # Don't use swap at all for the new system.
fs = ext3 # use the EXT3 filesystem for the disk image.
dist = lenny # Default distribution to install.
image = sparse # Specify sparse vs. full disk images.

[...]

##
# Networking setup values.
##
#
# Uncomment and adjust these network settings if you wish to give your
# new instances static IP addresses.
#
gateway = 78.46.242.89
netmask = 255.255.255.248
broadcast = 78.46.242.95

[...]

#
# Uncomment the following line if you wish to interactively setup
# a new root password for images.
#
passwd = 1

[...]

#
# The architecture to use when using debootstrap, rinse, or rpmstrap.
#
# This is most useful on 64 bit host machines, for other systems it
# doesn't need to be used.
#
arch=amd64

[...]

#
# The default mirror for debootstrap to install Debian-derived
# distributions
#
mirror = http://ftp2.de.debian.org/debian/

[...]

#
# If you're using the lenny or later version of the Xen guest kernel
# you will need to make sure that you use 'hvc0' for the guest serial
# device, and 'xvdX' instead of 'sdX' for serial devices.
#
# You may specify the things to use here:
#
serial_device = hvc0 #default
# serial_device = tty1
#
disk_device = xvda #default
# disk_device = sda

[...]

Next we need to add additional IP-address for our host system's ethernet card. Following shell script we need to put to "/etc/network/if-up.d/addresses" file:

#!/bin/sh
#
# /etc/network/if-up.d/addresses
# /etc/network/if-down.d/addresses
#
# Stefan Tomanek <stefan.tomanek@wertarbyte.de>
# http://wertarbyte.de/debian/
#
# Place in /etc/network/if-up.d/ and link to
# /etc/network/if-down.d/
#
# Configure additional addresses by specifying
# "addresses" in your /etc/network/interfaces file:
#
# iface eth0 inet dhcp
# addresses 10.0.0.1/8 192.168.10.4/24
# addresses-foo 192.168.2.4/24
# addresses-bar 172.17.2.1/16
#
# To make the addresses visible in legacy commands like
# "ifconfig", the script can create the deprecated alias
# devices (e.g. eth0:1) for you. To achieve this bevahiour,
# specify "create_alias_devices yes" in your configuration
# stanca.
#
# You can also use the labels used in /etc/network/interfaces
# to distinguish the configured addresses in the output of
# "ifconfig" and "ip addr". Simply set the configuration
# option "label_addresses yes". This option can also be
# combined with "create_alias_devices": While any addresses
# line accompanied by a label will be marked with it, anonymous
# addresses will be enumerated by the script.
#
# NOTE!
# Due to kernel limitations, the complete label of an address
# must not exceed 16 characters. This includes the device name
# and the : separating device name und custom string.
#
# To allow the inclusion of selfexplaining labels in the
# configuration, a short alias label can be specified:
#
# iface eth0 inet dhcp
# label_addresses yes
# addresses-www-webserver 10.0.0.3/24
# addresses-smtp-mailrelay 10.0.0.4/24
# addresses-dns 10.0.0.5/24
#
# With this configuration, the addresses will be
# labeled "www", "smtp",
# and "dns", ignoring the (optional) longer descriptions following
#
# This script is published under the rules of the GPLv3
# http://www.gnu.org/licenses/gpl.html

CMD_IP=/bin/ip

if [ ! -x $CMD_IP ]; then
  exit 0
fi

if [ "$MODE" = "start" ]; then
  OP="add"
else
  OP="del"
fi
COMPAT_DEVS=""
LABEL_ADDRESSES=""

if [ "$IF_CREATE_ALIAS_DEVICES" = "yes" ]; then
  # create enumerated labels (eth0:1, eth0:2...)
  COMPAT_DEVS="1"
fi

if [ "$IF_LABEL_ADDRESSES" = "yes" ]; then
  # use labels specifed in the configation (eth0:https, eth0
    :vserver...)
  LABEL_ADDRESSES="1"
fi

# print all addresses with the label next to it
ADDRESSES="$(/usr/bin/awk 'BEGIN {
   for (key in ENVIRON) {
      if (key ~ /^IF_ADDRESSES(_|$)/) {
          # we just use the third item of the splitted key and
          # consider it the "short" label - since device
          # names are limited
          # to 16 characters, a short label can be placed there while
          # adding
          # a more elaborate one to the config as well
          # (e.g. addresses-www-webserver 10.0.0.1/24)
          split( key, L, "_" );
          label = tolower( L[3] );
          split(ENVIRON[key], A);
          for (j in A) {
              addr = A[j];
              print addr " " label;
          }
      }
  }
}')"

# Nothing to do?
if [ -z "$ADDRESSES" ]; then
    exit 0
fi

N=1
echo "$ADDRESSES" | while read ADDR ALABEL; do
 LABEL=""
 # we don't want to specify the label when removing addresses
 if [ "del" != "$OP" ]; then
  # we don't want to use the supplied labels
  if [ ! "$LABEL_ADDRESSES" ]; then
      ALABEL=""
  fi
  # Numeric labels (old style eth0:X)
  if [ "$COMPAT_DEVS" -a -z "$ALABEL" ]; then
      ALABEL="$N"
      N=$(($N+1))
  fi
  # If we have a label and want it...
  if [ "$ALABEL" ] && [ "$COMPAT_DEVS" -o "$LABEL_ADDRESSES" ]; then
      LABEL="label $IFACE:$ALABEL"
  fi
 fi
FULL_CMD="$CMD_IP addr $OP $ADDR dev $IFACE $LABEL"
$FULL_CMD || \
echo "Error executing: $FULL_CMD" >&2
done

Then put this file in the right place and make proper link for if-down scripts:

root:~# cd /etc/network/if-down.d/
root:/etc/network/if-down.d# ln -s ../if-up.d/addresses .
root:/etc/network/if-down.d# chmod +x /etc/network/if-up.d/addresses

And now we can add simple instruction to our "/etc/nework/interfaces" file with new IP-address:

[...]
# device: eth0
auto  eth0
iface eth0 inet static
  address   213.239.234.27
  broadcast 213.239.234.31
  netmask   255.255.255.248
  gateway   213.239.234.25
  addresses 78.46.242.89/29
[...]

Ok. Let's play with sysctl.conf file:

### Hetzner Online AG installimage
# sysctl config
net.ipv4.ip_forward=1
net.ipv4.conf.all.rp_filter=1
net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv6.conf.all.forwarding=1
net.ipv4.conf.default.proxy_arp=1

Ok. Reboot the system with the new "xenned" kernel and new configurations:

root:~# shutdown -r now

Virtual Machines Rock'n'Roll

Yeah! Now we can create our new virtual machines and use it! Let's start creating one:

root:~# xen-create-image --hostname=summercode --ip=78.46.242.90 --
memory=512 --role=udev

General Information
--------------------
Hostname       :  summercode
Distribution   :  lenny
Partitions     :  swap            1024Mb (swap)
                  /               32Gb  (ext3)
Image type     :  full
Memory size    :  512
Kernel path    :  /boot/vmlinuz-2.6.26-2-xen-amd64
Initrd path    :  /boot/initrd.img-2.6.26-2-xen-amd64

Networking Information
----------------------
IP Address 1   : 78.46.242.90 [MAC: 00:16:3E:41:1C:7D]
Netmask        : 255.255.255.248
Broadcast      : 78.46.242.95
Gateway        : 78.46.242.89

Creating swap on /dev/vgvm/summercode-swap
Done

Creating ext3 filesystem on /dev/vgvm/summercode-disk
Done
Installation method: debootstrap
Done

Running hooks
Done

Role: udev
      File: /etc/xen-tools/role.d/udev
Role script completed.

Creating Xen configuration file
Done
Setting up root password
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
All done

Logfile produced at:
       /var/log/xen-tools/summercode.log

Yes. Everything is fine. Now we create virtual machine (it will be runned) and check that it's running:

root:~# xm create /etc/xen/summercode.cfg 
Using config file "/etc/xen/summercode.cfg". 
Started domain summercode 
root:~# xm list 
Name                           ID   Mem VCPUs     State   Time(s) 
Domain-0                        0  1485     2     r-----    123.3 
summercode                      1   512     1     r-----      1.7

Now we can go to the created VM's console and start to configure it:

root:~# xm console summercode
A+ a-
Clip in Evernote