Upstart issues

2010-10-22T13:18:26Z

Faxioman: /* Running Ubuntu Lucid */

== Running Ubuntu Lucid ==

There's a inipost-file ( http://static.thedudes.org/initpost ) that can be used when creating hosts, so there's no need for manual tweaking.
Put it in <pre>/usr/lib/util-vserver/distributions/lucid</pre> Install the vserver with --initstyle plain. You still need to look over services when installing since we're tweaking Upstarts normal emits.

!! Note !! The inipost-file is missing. You can temporarely download it from: http://dl.dropbox.com/u/1171189/initpost

You can also have a look at http://wiki.raceme.org/dokuwiki/tof:computer:vserver:upstart_ubuntu_lucid for a manual procedure.

== Running an upstart based guest ==

currently upstart based guests need some manual tweaking to make them work inside a vserver guest.

=== Prepare the host ===
First of all you need to set the init style of your vserver to plain
# echo plain > /etc/vservers/<vservername>/apps/init/style

Or use '--initstyle plain' when creating the host.

I'd also strongly recommend to mount <code>/var/run</code> and <code>/var/lock</code> of the guest as tmpfs. So add the following lines to
<code>/etc/vservers/<vservername>/fstab</code>
none /var/run tmpfs size=16m,nosuid,mode=0755 0 0
none /var/lock tmpfs size=16m,noexec,nosuid,nodev 0 0

=== Remove low level services inside the guest ===
Then you need to get rid of anything in <code>/etc/init/</code> that tries to fiddle with your Server's Hardware or other low-level Stuff that's already handled by your host's kernel/system. Having one such scriptlet still enabled might cause the whole upstart system to fail (like not beeing able to open <code>/dev/console</code> or failing to start the udev system).
Just remove these scriptlets, move them out of the way or rename them to something not ending in <code>.conf</code>. Be aware that upstart also parses subfolders of <code>/etc/init/</code> for <code>.conf</code> files.

This is what I deleted on a basic Ubuntu 9.10 guest:
dmesg.conf
hostname.conf
hwclock-save.conf
hwclock.conf
mountall-net.conf
mountall-reboot.conf
mountall-shell.conf
mountall.conf
network-interface.conf
networking.conf
procps.conf
rsyslog-kmsg.conf
tty1.conf
tty2.conf
tty3.conf
tty4.conf
tty5.conf
tty6.conf
upstart-udev-bridge.conf

In Ubuntu 10.04 you also want to remove:
console-setup.conf

=== Services you want to look over ===
There are files in the /etc/init dir that you want to check the startup line of.
It could be waiting for something that will never happend since it is in a VServer and thus never start.

for example mysql.conf needs a 'net-device-up'-emit also.
by running
vserver <name> exec initctl emit net-device-up

it will come up like a charm.
Either put this in vserver.conf or remove in from mysql.conf.

Jobs that doesn't start from scratch:
mysql-server-5.1 (mysql.conf)

=== Fake low level events inside the guest ===
Finally you need to fake emit the events that some of these scripts would do during regular boot. Probably the cleanest approach is to add a scriptlet <code>vserver.conf</code> to <code>/etc/init/</code> with this content:
start on startup
script
touch /var/run/utmp
chown root:utmp /var/run/utmp
chmod 664 /var/run/utmp
initctl emit virtual-filesystems
initctl emit local-filesystems
initctl emit remote-filesystems
initctl emit filesystem
end script

The utmp stuff is needed if you mount <code>/var/run</code> as tmpfs and you don't create utmp anywhere else.

=== Remove console outputs ===

If your VServer hangs on the command
vserver <name> stop

and puts out
A timeout occured while waiting for the vserver to finish and it will
be killed by sending a SIGKILL signal. The following process list
might be useful for finding out the reason of this behavior:

----------------------------------------------------------------------
<pid> <context> <name> ? Ss 0:00 /sbin/init
<pid> <context> <name> ? Ss 0:00 /sbin/init
----------------------------------------------------------------------

you should run
sed -i 's/^$console output$$/#\1/1' /etc/vservers/$name/vdir/etc/init/*

which will disable all lines with 'console output', and Upstart won't try output to it. Which is a good thing since it doesn't exist.

=== Adapt /etc/init/rc-sysinit.conf inside the guest ===

This depends on net-device-up IFACE=lo, remove this from the start line so that it reads:<br/>
<code>start on filesystem</code>

This scriptlet tries to parse <code>/proc/cmdline</code> which will not work and probably isn't needed inside a vserver.<br/>
Simply comment out or delete the whole <code>for ARG in $(cat /proc/cmdline) do...done</code> loop.

=== Notes for debugging ===

First you need to run Upstart with the --debug flag,
one way of doing it is inserting
"--debug"
in INITCMD_START under the xplain-clause inside /usr/lib/util-vserver/vserver.functions (row 316 here)
so it reads INITCMD_START=( /sbin/init "--debug" )

Second you should enable console output in the rc-files inside /etc/init, if you had commented them out before you could do
sed -i 's/#^$console output$$/\1/1' /etc/vservers/$name/vdir/etc/init/*

Third you need a device to act as console, I used tty0.
cp -a /dev/tty0 /etc/vservers/$name/vdir/dev/console

If you do things at a console of the machine it will now print out detailed information. More on how to debug a Upstart job you'll find on http://upstart.ubuntu.com/wiki/Debugging.
Notice that if one fails they all fail.

=== Notes for older kernels ===
If you encounter upstart errors like this:

$ vserver foo start
init: missing runlevel
Try `init --help' for more information.

or

$ vserver foo start
telinit: Failed to connect to socket /com/ubuntu/upstart: Connection refused

then you're probably running the current stable version <code>2.6.22.19-vs2.2.0.7</code>.

Upstart's <code>/sbin/init</code> implementation requires to be PID 1. It is linked against NPTL (in contrast to sysvinit), which has a bug in it's caching <code>getpid(2)</code> wrapper. With this kernel version <code>getpid(2)</code> returns the cached PID to init, resulting in init replacing itself with telinit. You can fix this by upgrading to a <code>2.6.26+</code> kernel and a newer vserver patch, or by patching upstart:

diff --git a/init/main.c b/init/main.c
index bfcd528..c6c9304 100644
--- a/init/main.c
+++ b/init/main.c
@@ -37,6 +37,7 @@
#include <string.h>
#include <syslog.h>
#include <unistd.h>
+#include <syscall.h>

#include <linux/kd.h>

@@ -131,7 +132,7 @@ main (int argc,
}

/* Check we're process #1 */
- if (getpid () > 1) {
+ if (syscall(SYS_getpid) > 1) {
execv (TELINIT, argv);
/* Ignore failure, probably just that telinit doesn't exist */

For more information see the man pages of <code>getpid(2)</code> (NOTES) and <code>clone(2)</code> (BUGS).

=== Init procedure doesn't complete ===
==== SSH ====
If you've completed all above steps and some init scripts doesn't get invoked at all (most notably old SYSV init scripts) you might have encountered same error that I did.
In Lucid guest I had to comment out following line in /etc/init/ssh.conf:
# replaces SSHD_OOM_ADJUST in /etc/default/ssh
oom never
to
# replaces SSHD_OOM_ADJUST in /etc/default/ssh
#oom never

After commenting out this line initialization completed successfully.

==== D-BUS ====
When creating guest as chroot some packages you might expect are missing. One of these is D-BUS. Strangely enough the /etc/init/dbus.conf gets installed which prevents Upstart to finish its starting sequence.

Solution: install dbus package

sudo aptitude install dbus

Linux-VServer - User contributions [en]

Upstart issues