Linux-VServer - User contributions [en]

util-vserver:Cgroups

2009-10-06T17:15:58Z

Arekm: /* cgroup and CFS based CPU hard limiting that replaces sched_hard */

== Draft - Cgroups and vservers ==

to make cgroups works with vserver your kernel must have the following settings:

<pre>
CONFIG_CGROUPS=y
# CONFIG_CGROUP_DEBUG is not set
# CONFIG_CGROUP_NS is not set
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_SCHED=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_MEM_RES_CTLR=y
</pre>

Or in the menu:

<pre>
[*] Control Group support [ ] Example debug cgroup subsystem
[ ] Namespace cgroup subsystem
[*] control group freezer subsystem
[*] Device controller for cgroups
[*] Cpuset support
[*] Group CPU scheduler
[*] Group scheduling for SCHED_OTHER
[ ] Group scheduling for SCHED_RR/FIFO
Basis for grouping tasks (Control groups) --->
[*] Simple CPU accounting cgroup subsystem
[*] Resource counters
[*] Memory Resource Controller for Control Groups
</pre>
 
Note about this from the list:
<pre>

> > I currently have the CONFIG_CGROUP_NS option set in my kernels. Does
> > this mean add extra namespace support, or I am preventing the Cgroup
> > based scheduling from working by enabling it? Should I unset it and
> > rebuild?

yes, with CGROUP_NS enabled, you basically block
the 'normal' cgroup scheduling which is used by
util-vserver, and have to handle the croup namespace
yourself ...
</pre>
 
You must have the /dev/cgroup mounted with the 'cpu' option. For exemple for cpuset, memory and cpu cgroup handling:

<pre>(root)> grep cgroup /etc/fstab
cpuset /dev/cgroup cgroup rw,cpuset,cpu,memory 0 0
</pre>

 
Then you must create on you guest /etc/vservers/<guest>/cgroup/ all the files put here will be copied to the guest cgroup when created. The doc for cgroup is not on the great flower page but in the source of the latests utils. You can have a default cgroup directory for all guest that can be overrided by each guest's cgroup directory.

== Draft - Distributing cpu shares with cgroups ==

From what i gathered in sched-design-CFS.txt [http://people.redhat.com/mingo/cfs-scheduler/sched-design-CFS.txt]

This is simply done by adjusting the cpu.shares. Just do:

echo '512' > /dev/cgroup/<guest name>/cpu.shares

The share you get is equal to the guest's share divided by the sum of the cpu shares of all the guest. So for exemple :

<pre>
vserver guest 1 => 512
vserver guest 2 => 512
vserver guest 3 => 2048
vserver guest 4 => 512
</pre>

so you have a total of 3584 cpu shares (2048+512+512+512) , then you get :

<pre>
vserver guest 1 => 512 / 3584 = 14% cpu
vserver guest 2 => 512 / 3584 = 14% cpu
vserver guest 3 => 2048 / 3584 = 57% cpu
vserver guest 4 => 512 / 3584 = 14% cpu
</pre>
 

Note that this is fair scheduling and this will not enfore HARD limit (as far as i know).

== Making share permanent with util vserver ==

You must use the "cgroup" directory. You can apply defaults to all vservers or choose different settings for each guest:

* /etc/vservers/.default/cgroup , this directory contains settings applying to all guest when they start
* /etc/vservers/<guestname>/cgroup , this directory contains settings for the guest when it starts.

Exemple :

<pre>
mkdir /etc/vservers/.defaults/cgroup
mkdir /etc/vservers/<guestname>/cgroup
echo '2048' > /etc/vservers/<guestname>/cgroup/cpu.shares
# if you use those then enable them if not put 0 instead of 1
echo 1 > /etc/vservers/<guestname>/cgroup/cpuset.cpus
echo 1 > /etc/vservers/<guestname>/cgroup/cpuset.mems
</pre>

Note that /etc/vservers is an exemple, in my Aqueos install i use /usr/local/etc/vservers but /etc/vservers seems to be the defaults for the classic installs.

Regards,
Ghislain.

== cgroup and CFS based CPU hard limiting that replaces sched_hard ==

This feature is currently avilable in patch-2.6.31.2-vs2.3.0.36.15.diff.

Example for 2/5th of the all cpu power, or 40%

<pre>
# force CFS hard limit
echo 1 > /etc/vservers/<guestname>/cgroup/cpu.cfs_hard_limit
# time assigned to guest (in us)
echo 200000 > /etc/vservers/<guestname>/cgroup/cpu.cfs_runtime_us
# in each specified period (in us)
echo 500000 > /etc/vservers/<guestname>/cgroup/cpu.cfs_period_us
</pre>

util-vserver:Cgroups

2009-10-06T17:11:55Z

Arekm: /* cgroup and CFS based CPU hard limiting that replaces sched_hard */

== Draft - Cgroups and vservers ==

to make cgroups works with vserver your kernel must have the following settings:

<pre>
CONFIG_CGROUPS=y
# CONFIG_CGROUP_DEBUG is not set
# CONFIG_CGROUP_NS is not set
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_SCHED=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_MEM_RES_CTLR=y
</pre>

Or in the menu:

<pre>
[*] Control Group support [ ] Example debug cgroup subsystem
[ ] Namespace cgroup subsystem
[*] control group freezer subsystem
[*] Device controller for cgroups
[*] Cpuset support
[*] Group CPU scheduler
[*] Group scheduling for SCHED_OTHER
[ ] Group scheduling for SCHED_RR/FIFO
Basis for grouping tasks (Control groups) --->
[*] Simple CPU accounting cgroup subsystem
[*] Resource counters
[*] Memory Resource Controller for Control Groups
</pre>
 
Note about this from the list:
<pre>

> > I currently have the CONFIG_CGROUP_NS option set in my kernels. Does
> > this mean add extra namespace support, or I am preventing the Cgroup
> > based scheduling from working by enabling it? Should I unset it and
> > rebuild?

yes, with CGROUP_NS enabled, you basically block
the 'normal' cgroup scheduling which is used by
util-vserver, and have to handle the croup namespace
yourself ...
</pre>
 
You must have the /dev/cgroup mounted with the 'cpu' option. For exemple for cpuset, memory and cpu cgroup handling:

<pre>(root)> grep cgroup /etc/fstab
cpuset /dev/cgroup cgroup rw,cpuset,cpu,memory 0 0
</pre>

 
Then you must create on you guest /etc/vservers/<guest>/cgroup/ all the files put here will be copied to the guest cgroup when created. The doc for cgroup is not on the great flower page but in the source of the latests utils. You can have a default cgroup directory for all guest that can be overrided by each guest's cgroup directory.

== Draft - Distributing cpu shares with cgroups ==

From what i gathered in sched-design-CFS.txt [http://people.redhat.com/mingo/cfs-scheduler/sched-design-CFS.txt]

This is simply done by adjusting the cpu.shares. Just do:

echo '512' > /dev/cgroup/<guest name>/cpu.shares

The share you get is equal to the guest's share divided by the sum of the cpu shares of all the guest. So for exemple :

<pre>
vserver guest 1 => 512
vserver guest 2 => 512
vserver guest 3 => 2048
vserver guest 4 => 512
</pre>

so you have a total of 3584 cpu shares (2048+512+512+512) , then you get :

<pre>
vserver guest 1 => 512 / 3584 = 14% cpu
vserver guest 2 => 512 / 3584 = 14% cpu
vserver guest 3 => 2048 / 3584 = 57% cpu
vserver guest 4 => 512 / 3584 = 14% cpu
</pre>
 

Note that this is fair scheduling and this will not enfore HARD limit (as far as i know).

== Making share permanent with util vserver ==

You must use the "cgroup" directory. You can apply defaults to all vservers or choose different settings for each guest:

* /etc/vservers/.default/cgroup , this directory contains settings applying to all guest when they start
* /etc/vservers/<guestname>/cgroup , this directory contains settings for the guest when it starts.

Exemple :

<pre>
mkdir /etc/vservers/.defaults/cgroup
mkdir /etc/vservers/<guestname>/cgroup
echo '2048' > /etc/vservers/<guestname>/cgroup/cpu.shares
# if you use those then enable them if not put 0 instead of 1
echo 1 > /etc/vservers/<guestname>/cgroup/cpuset.cpus
echo 1 > /etc/vservers/<guestname>/cgroup/cpuset.mems
</pre>

Note that /etc/vservers is an exemple, in my Aqueos install i use /usr/local/etc/vservers but /etc/vservers seems to be the defaults for the classic installs.

Regards,
Ghislain.

== cgroup and CFS based CPU hard limiting that replaces sched_hard ==

This feature is currently avilable in patch-2.6.31.2-vs2.3.0.36.15.diff.

<pre>
# force CFS hard limit
echo 1 > /etc/vservers/<guestname>/cgroup/cpu.cfs_hard_limit
# time assigned to guest (in us)
echo 200000 > /etc/vservers/<guestname>/cgroup/cpu.cfs_runtime_us
# in each specified period (in us)
echo 500000 > /etc/vservers/<guestname>/cgroup/cpu.cfs_period_us
</pre>

util-vserver:Cgroups

2009-10-06T17:10:25Z

Arekm: /* cgroup and CFS based CPU hard limiting that replaces sched_hard */

== Draft - Cgroups and vservers ==

to make cgroups works with vserver your kernel must have the following settings:

<pre>
CONFIG_CGROUPS=y
# CONFIG_CGROUP_DEBUG is not set
# CONFIG_CGROUP_NS is not set
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_SCHED=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_MEM_RES_CTLR=y
</pre>

Or in the menu:

<pre>
[*] Control Group support [ ] Example debug cgroup subsystem
[ ] Namespace cgroup subsystem
[*] control group freezer subsystem
[*] Device controller for cgroups
[*] Cpuset support
[*] Group CPU scheduler
[*] Group scheduling for SCHED_OTHER
[ ] Group scheduling for SCHED_RR/FIFO
Basis for grouping tasks (Control groups) --->
[*] Simple CPU accounting cgroup subsystem
[*] Resource counters
[*] Memory Resource Controller for Control Groups
</pre>
 
Note about this from the list:
<pre>

> > I currently have the CONFIG_CGROUP_NS option set in my kernels. Does
> > this mean add extra namespace support, or I am preventing the Cgroup
> > based scheduling from working by enabling it? Should I unset it and
> > rebuild?

yes, with CGROUP_NS enabled, you basically block
the 'normal' cgroup scheduling which is used by
util-vserver, and have to handle the croup namespace
yourself ...
</pre>
 
You must have the /dev/cgroup mounted with the 'cpu' option. For exemple for cpuset, memory and cpu cgroup handling:

<pre>(root)> grep cgroup /etc/fstab
cpuset /dev/cgroup cgroup rw,cpuset,cpu,memory 0 0
</pre>

 
Then you must create on you guest /etc/vservers/<guest>/cgroup/ all the files put here will be copied to the guest cgroup when created. The doc for cgroup is not on the great flower page but in the source of the latests utils. You can have a default cgroup directory for all guest that can be overrided by each guest's cgroup directory.

== Draft - Distributing cpu shares with cgroups ==

From what i gathered in sched-design-CFS.txt [http://people.redhat.com/mingo/cfs-scheduler/sched-design-CFS.txt]

This is simply done by adjusting the cpu.shares. Just do:

echo '512' > /dev/cgroup/<guest name>/cpu.shares

The share you get is equal to the guest's share divided by the sum of the cpu shares of all the guest. So for exemple :

<pre>
vserver guest 1 => 512
vserver guest 2 => 512
vserver guest 3 => 2048
vserver guest 4 => 512
</pre>

so you have a total of 3584 cpu shares (2048+512+512+512) , then you get :

<pre>
vserver guest 1 => 512 / 3584 = 14% cpu
vserver guest 2 => 512 / 3584 = 14% cpu
vserver guest 3 => 2048 / 3584 = 57% cpu
vserver guest 4 => 512 / 3584 = 14% cpu
</pre>
 

Note that this is fair scheduling and this will not enfore HARD limit (as far as i know).

== Making share permanent with util vserver ==

You must use the "cgroup" directory. You can apply defaults to all vservers or choose different settings for each guest:

* /etc/vservers/.default/cgroup , this directory contains settings applying to all guest when they start
* /etc/vservers/<guestname>/cgroup , this directory contains settings for the guest when it starts.

Exemple :

<pre>
mkdir /etc/vservers/.defaults/cgroup
mkdir /etc/vservers/<guestname>/cgroup
echo '2048' > /etc/vservers/<guestname>/cgroup/cpu.shares
# if you use those then enable them if not put 0 instead of 1
echo 1 > /etc/vservers/<guestname>/cgroup/cpuset.cpus
echo 1 > /etc/vservers/<guestname>/cgroup/cpuset.mems
</pre>

Note that /etc/vservers is an exemple, in my Aqueos install i use /usr/local/etc/vservers but /etc/vservers seems to be the defaults for the classic installs.

Regards,
Ghislain.

== cgroup and CFS based CPU hard limiting that replaces sched_hard ==

This feature is currently avilable in patch-2.6.31.2-vs2.3.0.36.15.diff.

<pre>
# force CFS hard limit
echo 1 > /etc/vservers/<guestname>/cgroup/cpu.cfs_hard_limit
# time assigned to guest (in us)
echo 500000 > /etc/vservers/<guestname>/cgroup/cpu.cfs_runtime_us
# in each specified period (in us)
echo 200000 > /etc/vservers/<guestname>/cgroup/cpu.cfs_period_us
</pre>

util-vserver:Cgroups

2009-10-06T17:09:53Z

Arekm:

== Draft - Cgroups and vservers ==

to make cgroups works with vserver your kernel must have the following settings:

<pre>
CONFIG_CGROUPS=y
# CONFIG_CGROUP_DEBUG is not set
# CONFIG_CGROUP_NS is not set
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_SCHED=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_MEM_RES_CTLR=y
</pre>

Or in the menu:

<pre>
[*] Control Group support [ ] Example debug cgroup subsystem
[ ] Namespace cgroup subsystem
[*] control group freezer subsystem
[*] Device controller for cgroups
[*] Cpuset support
[*] Group CPU scheduler
[*] Group scheduling for SCHED_OTHER
[ ] Group scheduling for SCHED_RR/FIFO
Basis for grouping tasks (Control groups) --->
[*] Simple CPU accounting cgroup subsystem
[*] Resource counters
[*] Memory Resource Controller for Control Groups
</pre>
 
Note about this from the list:
<pre>

> > I currently have the CONFIG_CGROUP_NS option set in my kernels. Does
> > this mean add extra namespace support, or I am preventing the Cgroup
> > based scheduling from working by enabling it? Should I unset it and
> > rebuild?

yes, with CGROUP_NS enabled, you basically block
the 'normal' cgroup scheduling which is used by
util-vserver, and have to handle the croup namespace
yourself ...
</pre>
 
You must have the /dev/cgroup mounted with the 'cpu' option. For exemple for cpuset, memory and cpu cgroup handling:

<pre>(root)> grep cgroup /etc/fstab
cpuset /dev/cgroup cgroup rw,cpuset,cpu,memory 0 0
</pre>

 
Then you must create on you guest /etc/vservers/<guest>/cgroup/ all the files put here will be copied to the guest cgroup when created. The doc for cgroup is not on the great flower page but in the source of the latests utils. You can have a default cgroup directory for all guest that can be overrided by each guest's cgroup directory.

== Draft - Distributing cpu shares with cgroups ==

From what i gathered in sched-design-CFS.txt [http://people.redhat.com/mingo/cfs-scheduler/sched-design-CFS.txt]

This is simply done by adjusting the cpu.shares. Just do:

echo '512' > /dev/cgroup/<guest name>/cpu.shares

The share you get is equal to the guest's share divided by the sum of the cpu shares of all the guest. So for exemple :

<pre>
vserver guest 1 => 512
vserver guest 2 => 512
vserver guest 3 => 2048
vserver guest 4 => 512
</pre>

so you have a total of 3584 cpu shares (2048+512+512+512) , then you get :

<pre>
vserver guest 1 => 512 / 3584 = 14% cpu
vserver guest 2 => 512 / 3584 = 14% cpu
vserver guest 3 => 2048 / 3584 = 57% cpu
vserver guest 4 => 512 / 3584 = 14% cpu
</pre>
 

Note that this is fair scheduling and this will not enfore HARD limit (as far as i know).

== Making share permanent with util vserver ==

You must use the "cgroup" directory. You can apply defaults to all vservers or choose different settings for each guest:

* /etc/vservers/.default/cgroup , this directory contains settings applying to all guest when they start
* /etc/vservers/<guestname>/cgroup , this directory contains settings for the guest when it starts.

Exemple :

<pre>
mkdir /etc/vservers/.defaults/cgroup
mkdir /etc/vservers/<guestname>/cgroup
echo '2048' > /etc/vservers/<guestname>/cgroup/cpu.shares
# if you use those then enable them if not put 0 instead of 1
echo 1 > /etc/vservers/<guestname>/cgroup/cpuset.cpus
echo 1 > /etc/vservers/<guestname>/cgroup/cpuset.mems
</pre>

Note that /etc/vservers is an exemple, in my Aqueos install i use /usr/local/etc/vservers but /etc/vservers seems to be the defaults for the classic installs.

Regards,
Ghislain.

== cgroup and CFS based CPU hard limiting that replaces sched_hard ==

This feature is currently avilable in patch-2.6.31.2-vs2.3.0.36.15.diff.

<pre>
# force CFS hard limit
echo 1 > /etc/vservers/<guestname>/cgroup/cpu.cfs_hard_limit
# time assigned to guest (in us)
echo 500000 > /etc/vservers/<guestname>/cgroup/cpu.cfs_runtime_us
# in each specified period (in us)
echo 1 > /etc/vservers/<guestname>/cgroup/cpu.cfs_period_us
</pre>