Glenn: +link

2011-10-21T20:25:15Z

+link

Glenn: + Category:Documentation

2011-10-21T19:32:47Z

+ Category:Documentation

Jze: migrated from old wiki Vserver+DRBD

2009-04-08T11:07:05Z

migrated from old wiki Vserver+DRBD

New page

== A Vserver guest mirrored by DRBD with heartbeat failover ==

The idea is simple:
Install a vserver on a DRBD resource and let it failover to the other node of a cluster.

This configuration uses vserver, DRBD and heartbeat. This page will not describe, how these packages are installed and configured. Only the important topics are mentioned.

Steps of installation:
* compile a vserver kernel
* compile the DRBD module against this kernel
* install the kernel, the module and reboot
* install the user space utilities of DRBD, vserver and heartbeat
* install the same packages at the second node
* configure and activate a DRBD resource
* make the DRBD resource primary on one node
* create a journalled filesystem (ext3, xfs, reiser) on top of the DRBD device
* mount the DRBD device as /var/lib/vservers/<vserver-name>
* install a vserver
* finish configuration
* sync the configuration to the other node

==In detail==

===Install the necessary packages===
On Debian Sarge we can use ready packages. Your <tt>/etc/apt/sources.list</tt> should have an entry pointing to unstable.
apt-get update

Maybe some additional packages are necessary:
apt-get install libdb3-dev libncurses-dev docbook-utils
apt-get install fakeroot dpatch

Get the kernel source and unpack it:
apt-get install linux-source-2.6.12
cd /usr/src
tar xjf /usr/src/linux-source-2.6.12.tar.bz2

Maybe you want to have your old .config
cd linux-source-2.6.12
cp /boot/config-2.6.12-1-386 .config

Now it's time to install patches:
apt-get install util-vserver -t unstable
apt-get install kernel-patch-vserver -t unstable

Compile the Kernel:
make-kpkg --rootcmd fakeroot --revision custom01 --added-patches vserver--append-to-version +vserver --initrd binary-arch

Now you should find a .deb ready to install:
cd ..
dpkg -i kernel-image-2.6.12+vserver_custom01_i386.deb

The configuration of GRUB should be updated now to boot the new kernel - try a reboot.

Install the DRBD packages:
apt-get install drbd0.7-module-source -t unstable
apt-get install drbd0.7-utils -t unstable

Unpack the module source:
cd /usr/src/
tar xzf drbd0.7.tar.gz

Compile the DRBD module:
cd linux-source-2.6.12
make-kpkg --append-to-version +vserver modules_image

Install the module:
cd ..
dpkg -i /usr/src/drbd0.7-module-2.6.12+vserver_0.7.11-1+custom01_i386.deb
update-modules
modprobe drbd

===Prepare the disk space===
You will need some block device like <tt>/dev/hdaX</tt>, or a better choice is to use LVM.
lvcreate -L3000 -nlvvs03 vg0

A recommended method is, to have DRBD metadata on an extra block device for each DRBD device.

Prepare space for it:
lvcreate -L132 -nlvvs03md vg0

===Configure DRBD===
Create an entry in <tt>/etc/drbd.conf</tt> using your favorite editor:
<pre>
resource vs03 {
protocol C;
incon-degr-cmd "echo '!DRBD! pri on incon-degr' | wall ; sleep 60 ; halt -f";

startup {
wfc-timeout 100;
degr-wfc-timeout 120; # 2 minutes.
}

disk {
on-io-error pass_on;
}

net {
# sndbuf-size 512k;
# timeout 60; # 6 seconds (unit = 0.1 seconds)
# connect-int 10; # 10 seconds (unit = 1 second)
# ping-int 10; # 10 seconds (unit = 1 second)
# max-buffers 2048;
# max-epoch-size 2048;
# ko-count 4;
# on-disconnect reconnect;
}

syncer {
rate 10M;
group 1;
al-extents 257;
}

on cel600 {
device /dev/drbd1;
disk /dev/vg0/lvvs03;
address 192.168.0.22:7789;
meta-disk /dev/vg0/lvvs03md[0];
}

on xp2400 {
device /dev/drbd1;
disk /dev/vg0/lvvs03;
address 192.168.0.21:7789;
meta-disk /dev/vg0/lvvs03md[0];
}
}
</pre>

Start DRBD
/etc/init.d/drbd start

The DRBD resource is in an undefined state. It needs to be forced to be 'primary' on the current node.
drbdsetup /dev/drbd1 primary --do-what-I-say

Obviously a filesystem is needed on top of DRBD:
mke2fs -j /dev/drbd1

Also we need a mount point somewhere:
mkdir /mnt/vs03

As heartbeat is intended to manage the vserver, using the heartbeat script for mounting is a first test:
/etc/ha.d/resource.d/Filesystem /dev/drbd1 /mnt/vs03 ext3 start

=== Install the vserver guest===
There are many methods how to install a vserver. Here 'debootstrap' is used. The important option is '--rootdir' to force the installation to the DRBD device.
vserver -v vs03 build --force --rootdir /mnt/vs03 \
-m debootstrap --hostname vs03.nack.at \
--netdev eth0 --interface 192.168.0.3/24 --context 3 -- \
-d sarge -m http://ftp.at.debian.org/debian/

We could start the vserver now. But first we try to make the configuration more pretty.
cd /mnt/vs03/vs03/etc/rc0.d
rm K20makedev K25hwclock.sh S30urandom S31umountnfs.sh S35networking S36ifupdown S40umountfs S90halt K89klogd
cd ../rc6.d
rm K20makedev K25hwclock.sh S30urandom S31umountnfs.sh S35networking S36ifupdown S40umountfs S90reboot K89klogd
cd ../rcS.d
rm S05keymap.sh S48console-screen.sh S50hwclock.sh S40networking S45mountnfs.sh S10checkroot.sh S02mountvirtfs
rm S30procps.sh S35mountall.sh S36mountvirtfs S39ifupdown S30checkfs.sh S18ifupdown-clean S18hwclockfirst.sh
cd ../rc2.d
rm S20makedev S11klogd

If we want to have virtual IPs in the host context managed by heartbeat, there can be conflicts (and funny effects), if we do not 'name' the alias of the vserver IP.
echo vs03 > /etc/vservers/vs03/interfaces/0/name

sshd should only listen to the IPs of the host context:
xp2400:/# cat /etc/ssh/sshd_config | grep ListenAddress
ListenAddress 192.168.0.21:22
ListenAddress 192.168.0.20:22

We want to limit the vserver to 1 GB AS/VM (virtual address space) and 128 MB RSS (real memory). Note, that the units are pages.
xp2400:/# mkdir /etc/vservers/vs03/rlimits
xp2400:/usr/src# echo 250000 > /etc/vservers/vs03/rlimits/as
xp2400:/# echo 32000 > /etc/vservers/vs03/rlimits/rss

We want control CPU consumption:
xp2400:/# echo sched_prio > /etc/vservers/vs03/flags
xp2400:/# echo 6 > /etc/vservers/vs03/schedule
xp2400:/# echo 30 >> /etc/vservers/vs03/schedule
xp2400:/# echo 1000 >> /etc/vservers/vs03/schedule
xp2400:/# echo 200 >> /etc/vservers/vs03/schedule
xp2400:/# echo 3000 >> /etc/vservers/vs03/schedule
xp2400:/# echo dummy >> /etc/vservers/vs03/schedule
xp2400:/# cat /etc/vservers/vs03/schedule
6
30
1000
200
3000
dummy

After these checks it's time to start the vserver:
xp2400:/# vserver vs03 start

Check if the interface is nicely named and shown with ifconfig:
xp2400:/# ip addr show
[...]
2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:30:bd:6b:d6:8f brd ff:ff:ff:ff:ff:ff
inet 192.168.0.21/24 brd 192.168.0.255 scope global eth0
inet 192.168.0.3/24 brd 192.168.0.255 scope global secondary eth0:vs03
inet6 fe80::230:bdff:fe6b:d68f/64 scope link
valid_lft forever preferred_lft forever

xp2400:/# ifconfig
eth0 Link encap:Ethernet [...]

eth1 [...]

eth0:vs03 Link encap:Ethernet HWaddr 00:30:BD:6B:D6:8F
inet addr:192.168.0.3 Bcast:192.168.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:177 Base address:0xd000

Enter the vserver:
xp2400:/# vserver vs03 enter

In the vserver we check the base configuration and install some packages.
vs03:/# base-config
vs03:/# apt-get update
vs03:/# apt-get install locales
vs03:/# apt-get install less
vs03:/# nano /etc/resolv.conf
vs03:/# ping www.linux-vserver.org

We want to install at least one service, sshd.
vs03:/# apt-get install ssh
vs03:/# nano /etc/ssh/sshd_config
vs03:/# cat /etc/ssh/sshd_config | grep ListenAddress
ListenAddress 192.168.0.3:22
vs03:/# /etc/init.d/ssh restart
Restarting OpenBSD Secure Shell server: sshd.
vs03:/# passwd
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
vs03:/# exit
logout

Test the ssh login:
xp2400:/# ssh root@192.168.0.3
The authenticity of host '192.168.0.3 (192.168.0.3)' can't be established.
RSA key fingerprint is e2:19:d1:8d:7f:ac:0a:ce:12:f5:06:b7:ae:41:96:59.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.0.3' (RSA) to the list of known hosts.
Password: <secret>
vs03:~#

As another service Apache2 should be used:
vs03:/# apt-get install apache2
[...]
Starting web server: Apache2(98)Address already in use: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
Unable to open logs
invoke-rc.d: initscript apache2, action "start" failed.

Setting up apache2 (2.0.54-4) ...

To avoid the above error:
vs03:/# nano /etc/apache2/ports.conf
vs03:/# cat /etc/apache2/ports.conf
Listen 192.168.0.3:80
vs03:/# /etc/init.d/apache2 start
Starting web server: Apache2(98)Address already in use: make_sock: could not bind to address 192.168.0.3:80
no listening sockets available, shutting down
Unable to open logs

Do we have an Apache listening on the vserver IP in the host context?
xp2400:/# cat /etc/apache2/ports.conf
Listen 80
xp2400:/# nano /etc/apache2/ports.conf
xp2400:/# cat /etc/apache2/ports.conf
Listen 192.168.0.21:80
Listen 127.0.0.1:80
xp2400:/# /etc/init.d/apache2 restart
Forcing reload of web server: Apache2.

vs03:/# /etc/init.d/apache2 start
Starting web server: Apache2.

xp2400:/# lynx http://192.168.0.3/
Test Page for Apache Installation

If you can see this, it means that the installation of the Apache web
server software on this system was successful. You may now add content
to this directory and replace this page.
[...]

The configuration on one node is ready.

To be continued ...

← Older revision		Revision as of 19:32, 21 October 2011
Line 290:		Line 290:

	To be continued ...		To be continued ...
		+
		+	[[Category:Documentation]]

@@ Line 2: / Line 2: @@
 The idea is simple:
-Install a vserver on a DRBD resource and let it failover to the other node of a cluster.
+Install a vserver on a [http://en.wikipedia.org/wiki/DRBD DRBD] resource and let it failover to the other node of a cluster.
 This configuration uses vserver, DRBD and heartbeat. This page will not describe, how these packages are installed and configured. Only the important topics are mentioned.

DRBD - Revision history

Glenn: +link

Glenn: + Category:Documentation

Jze: migrated from old wiki Vserver+DRBD