Linux-VServer - User contributions [en]

util-vserver:Cgroups

2009-12-02T19:46:13Z

Kbad: /* Kernel configuration */

Bears run away when you yell at them, even lynxes. ,

== Kernel configuration ==

When configuring your kernel for cgroups with util-vserver you must make sure CONFIG_CGROUP_NS is unset so guests start properly for the time being.

== Draft - Distributing cpu shares with cgroups ==

From what i gathered in sched-design-CFS.txt [http://people.redhat.com/mingo/cfs-scheduler/sched-design-CFS.txt]

This is simply done by adjusting the cpu.shares. Just do:

echo '512' > /dev/cgroup/<guest name>/cpu.shares

The share you get is equal to the guest's share divided by the sum of the cpu shares of all the guest. So for exemple :

<pre>
vserver guest 1 => 512
vserver guest 2 => 512
vserver guest 3 => 2048
vserver guest 4 => 512
</pre>

so you have a total of 3584 cpu shares (2048+512+512+512) , then you get :

<pre>
vserver guest 1 => 512 / 3584 = 14% cpu
vserver guest 2 => 512 / 3584 = 14% cpu
vserver guest 3 => 2048 / 3584 = 57% cpu
vserver guest 4 => 512 / 3584 = 14% cpu
</pre>
 

Note that this is fair scheduling and this will not enfore HARD limit (as far as i know).

== Making share permanent with util vserver ==

You must use the "cgroup" directory. You can apply defaults to all vservers or choose different settings for each guest:

* /etc/vservers/.default/cgroup , this directory contains settings applying to all guest when they start
* /etc/vservers/<guestname>/cgroup , this directory contains settings for the guest when it starts.

Exemple :

<pre>
mkdir /etc/vservers/.defaults/cgroup
mkdir /etc/vservers/<guestname>/cgroup
echo '2048' > /etc/vservers/<guestname>/cgroup/cpu.shares
# List of CPUs
echo 1 > /etc/vservers/<guestname>/cgroup/cpuset.cpus
# NUMA nodes
echo 1 > /etc/vservers/<guestname>/cgroup/cpuset.mems
</pre>

Note that /etc/vservers is an exemple, in my Aqueos install i use /usr/local/etc/vservers but /etc/vservers seems to be the defaults for the classic installs.

Regards,
Ghislain.

== cgroup and CFS based CPU hard limiting that replaces sched_hard ==

This feature is currently available in patch-2.6.31.2-vs2.3.0.36.15.diff and is in testing phase as of this patch set so report any bugs to the mailing list.

Example for an upper bound of 2/5th (or 40%) of the all cpu power that a guest/cgroup can use :

<pre>
# force CFS hard limit
echo 1 > /etc/vservers/<guestname>/cgroup/cpu.cfs_hard_limit
# time assigned to guest (in microseconds) 200000 = 0,2 sec
echo 200000 > /etc/vservers/<guestname>/cgroup/cpu.cfs_runtime_us
# in each specified period (in microseconds) 500000 = 0,5 sec
echo 500000 > /etc/vservers/<guestname>/cgroup/cpu.cfs_period_us
</pre>

This limit is an hard limit, see it like an upper wall for the ressources used by the cgroup.
If you set both cpu share AND hard limit the system will do fine but hard limits takes priority over cpu share scheduling, so cpu share will do the job but each cgroup will have an upper bound that it cannot cross even if the cpu share you gived it is higher.

<pre>
Hard limit feature adds 3 cgroup files for CFS group scheduler:
cfs_runtime_us: Hard limit for the group in microseconds.
cfs_period_us: Time period in microseconds within which hard limits is enforced.
cfs_hard_limit: The control file to enable or disable hard limiting for the group.
</pre>

 

== real world exemples of scheduling ==

this part is to be filled with exemple you have put in place and are working and have been tested, please add the patch and kernel version for each exemple you put here.

util-vserver:Cgroups

2009-12-02T19:41:27Z

Kbad:

Bears run away when you yell at them, even lynxes. ,

== Kernel configuration ==

When configuring your kernel for cgroups with vserver you must make sure CONFIG_CGROUP_NS is unset so guests start properly.

== Draft - Distributing cpu shares with cgroups ==

From what i gathered in sched-design-CFS.txt [http://people.redhat.com/mingo/cfs-scheduler/sched-design-CFS.txt]

This is simply done by adjusting the cpu.shares. Just do:

echo '512' > /dev/cgroup/<guest name>/cpu.shares

The share you get is equal to the guest's share divided by the sum of the cpu shares of all the guest. So for exemple :

<pre>
vserver guest 1 => 512
vserver guest 2 => 512
vserver guest 3 => 2048
vserver guest 4 => 512
</pre>

so you have a total of 3584 cpu shares (2048+512+512+512) , then you get :

<pre>
vserver guest 1 => 512 / 3584 = 14% cpu
vserver guest 2 => 512 / 3584 = 14% cpu
vserver guest 3 => 2048 / 3584 = 57% cpu
vserver guest 4 => 512 / 3584 = 14% cpu
</pre>
 

Note that this is fair scheduling and this will not enfore HARD limit (as far as i know).

== Making share permanent with util vserver ==

You must use the "cgroup" directory. You can apply defaults to all vservers or choose different settings for each guest:

* /etc/vservers/.default/cgroup , this directory contains settings applying to all guest when they start
* /etc/vservers/<guestname>/cgroup , this directory contains settings for the guest when it starts.

Exemple :

<pre>
mkdir /etc/vservers/.defaults/cgroup
mkdir /etc/vservers/<guestname>/cgroup
echo '2048' > /etc/vservers/<guestname>/cgroup/cpu.shares
# List of CPUs
echo 1 > /etc/vservers/<guestname>/cgroup/cpuset.cpus
# NUMA nodes
echo 1 > /etc/vservers/<guestname>/cgroup/cpuset.mems
</pre>

Note that /etc/vservers is an exemple, in my Aqueos install i use /usr/local/etc/vservers but /etc/vservers seems to be the defaults for the classic installs.

Regards,
Ghislain.

== cgroup and CFS based CPU hard limiting that replaces sched_hard ==

This feature is currently available in patch-2.6.31.2-vs2.3.0.36.15.diff and is in testing phase as of this patch set so report any bugs to the mailing list.

Example for an upper bound of 2/5th (or 40%) of the all cpu power that a guest/cgroup can use :

<pre>
# force CFS hard limit
echo 1 > /etc/vservers/<guestname>/cgroup/cpu.cfs_hard_limit
# time assigned to guest (in microseconds) 200000 = 0,2 sec
echo 200000 > /etc/vservers/<guestname>/cgroup/cpu.cfs_runtime_us
# in each specified period (in microseconds) 500000 = 0,5 sec
echo 500000 > /etc/vservers/<guestname>/cgroup/cpu.cfs_period_us
</pre>

This limit is an hard limit, see it like an upper wall for the ressources used by the cgroup.
If you set both cpu share AND hard limit the system will do fine but hard limits takes priority over cpu share scheduling, so cpu share will do the job but each cgroup will have an upper bound that it cannot cross even if the cpu share you gived it is higher.

<pre>
Hard limit feature adds 3 cgroup files for CFS group scheduler:
cfs_runtime_us: Hard limit for the group in microseconds.
cfs_period_us: Time period in microseconds within which hard limits is enforced.
cfs_hard_limit: The control file to enable or disable hard limiting for the group.
</pre>

 

== real world exemples of scheduling ==

this part is to be filled with exemple you have put in place and are working and have been tested, please add the patch and kernel version for each exemple you put here.

util-vserver:Cgroups

2009-12-02T19:40:55Z

Kbad: /* Draft - Distributing cpu shares with cgroups */

Bears run away when you yell at them, even lynxes. ,

== Draft - Distributing cpu shares with cgroups ==

When configuring your kernel for cgroups with vserver you must make sure CONFIG_CGROUP_NS is unset so guests start properly.

From what i gathered in sched-design-CFS.txt [http://people.redhat.com/mingo/cfs-scheduler/sched-design-CFS.txt]

This is simply done by adjusting the cpu.shares. Just do:

echo '512' > /dev/cgroup/<guest name>/cpu.shares

The share you get is equal to the guest's share divided by the sum of the cpu shares of all the guest. So for exemple :

<pre>
vserver guest 1 => 512
vserver guest 2 => 512
vserver guest 3 => 2048
vserver guest 4 => 512
</pre>

so you have a total of 3584 cpu shares (2048+512+512+512) , then you get :

<pre>
vserver guest 1 => 512 / 3584 = 14% cpu
vserver guest 2 => 512 / 3584 = 14% cpu
vserver guest 3 => 2048 / 3584 = 57% cpu
vserver guest 4 => 512 / 3584 = 14% cpu
</pre>
 

Note that this is fair scheduling and this will not enfore HARD limit (as far as i know).

== Making share permanent with util vserver ==

You must use the "cgroup" directory. You can apply defaults to all vservers or choose different settings for each guest:

* /etc/vservers/.default/cgroup , this directory contains settings applying to all guest when they start
* /etc/vservers/<guestname>/cgroup , this directory contains settings for the guest when it starts.

Exemple :

<pre>
mkdir /etc/vservers/.defaults/cgroup
mkdir /etc/vservers/<guestname>/cgroup
echo '2048' > /etc/vservers/<guestname>/cgroup/cpu.shares
# List of CPUs
echo 1 > /etc/vservers/<guestname>/cgroup/cpuset.cpus
# NUMA nodes
echo 1 > /etc/vservers/<guestname>/cgroup/cpuset.mems
</pre>

Note that /etc/vservers is an exemple, in my Aqueos install i use /usr/local/etc/vservers but /etc/vservers seems to be the defaults for the classic installs.

Regards,
Ghislain.

== cgroup and CFS based CPU hard limiting that replaces sched_hard ==

This feature is currently available in patch-2.6.31.2-vs2.3.0.36.15.diff and is in testing phase as of this patch set so report any bugs to the mailing list.

Example for an upper bound of 2/5th (or 40%) of the all cpu power that a guest/cgroup can use :

<pre>
# force CFS hard limit
echo 1 > /etc/vservers/<guestname>/cgroup/cpu.cfs_hard_limit
# time assigned to guest (in microseconds) 200000 = 0,2 sec
echo 200000 > /etc/vservers/<guestname>/cgroup/cpu.cfs_runtime_us
# in each specified period (in microseconds) 500000 = 0,5 sec
echo 500000 > /etc/vservers/<guestname>/cgroup/cpu.cfs_period_us
</pre>

This limit is an hard limit, see it like an upper wall for the ressources used by the cgroup.
If you set both cpu share AND hard limit the system will do fine but hard limits takes priority over cpu share scheduling, so cpu share will do the job but each cgroup will have an upper bound that it cannot cross even if the cpu share you gived it is higher.

<pre>
Hard limit feature adds 3 cgroup files for CFS group scheduler:
cfs_runtime_us: Hard limit for the group in microseconds.
cfs_period_us: Time period in microseconds within which hard limits is enforced.
cfs_hard_limit: The control file to enable or disable hard limiting for the group.
</pre>

 

== real world exemples of scheduling ==

this part is to be filled with exemple you have put in place and are working and have been tested, please add the patch and kernel version for each exemple you put here.