Linux Cgroups

Every thought of how linux organizes processes in parent-child relationship and how resources are managed and controlled efficiently like: limiting CPU usage, memory,etc. Cgroup in linux is the same technology that empowers Dockers and containers and the interesting part is — you can try it yourself right now.

A Linux machine can have dozens of processes competing for the same CPU and memory. Without some way to organize them, one badly behaved program can consume far more than its fair share.

Linux cgroups, short for control groups, solve this by letting the kernel put processes into groups and apply resource rules to those groups. The idea is simple: instead of asking only “How much memory is this process using?”, Linux can also ask “How much memory is this whole group of processes allowed to use?”

This is one of the important pieces behind containers, but cgroups are useful even when you never run a container.

Why Does Linux Need Cgroups?

Imagine a university server shared by two groups:

• professors
• students

Both groups run programs on the same machine.

If everyone is allowed to use resources without any limits, one group could consume almost all of the CPU or memory and leave very little for everyone else.

So the administrator might decide:

University Server

CPU
├── Professors → larger share
└── Students   → smaller share

Memory
├── Professors → larger limit
└── Students   → smaller limit

The important part is that the administrator is controlling groups of work, not manually controlling every program one by one.

This is the simple version of the Linux kernel’s professor/student example: the same process can be classified differently for different resources. For example, a professor’s browser and a student’s browser might both be browsers, but their CPU or memory rules can be different depending on which group they belong to.

What Does a Cgroup Actually Do?

A cgroup gives the kernel two things:

1. A group of processes to keep track of
2. Rules about resources for that group

The resources are controlled by controllers.

Some important controllers include:

The exact controls available depend on the Linux kernel and cgroup version.

So a useful mental model is:

Processes
    ↓
  cgroup
    ↓
resource controllers
    ↓
CPU / Memory / I/O / Process count

Cgroups do not create another computer.

The processes are still ordinary Linux processes. They simply belong to a group that the kernel can control and account for.

How Does a Cgroup Work?

Think of a cgroup as a room with rules.

Suppose we create:

student-group
├── terminal
├── compiler
└── browser

All of those processes are still running on the same Linux machine.

But the kernel now knows:

These processes belong to student-group.

We can then give that group a memory limit:

student-group
  │  │
  │  ├── terminal
  │  ├── compiler
  │  └── browser
  │
  ▼
Memory limit
  500 MB

The limit applies to the group’s processes together, rather than being a separate 500 MB allowance for every process.

Cgroups Are Hierarchical

Cgroups can also be nested:

system
└── university
    ├── professors
    │   ├── teaching
    │   └── research
    └── students
        ├── projects
        └── labs

This is useful when a large machine needs several levels of resource management.

A child group operates within the limits imposed by its parents.

What Happens to Child Processes?

A useful property of cgroups is that processes can bring their children into the same group.

For example:

terminal
   │
   ├── compiler
   └── test program

If the terminal is placed into a cgroup and it creates those child processes, they can inherit the cgroup membership.

This is important because otherwise a program could simply create another process and escape the resource accounting.

Controllers Are the Actual Resource Rules

The cgroup itself is mainly the grouping (processes) mechanism.

Controllers provide the resource-specific behavior.

                 cgroup
                   │
       ┌───────────┼───────────┐
       ▼           ▼           ▼
     CPU         Memory        I/O
   controller   controller   controller
       │           │           │
       ▼           ▼           ▼
    CPU rules   RAM rules   disk I/O rules

This separation is why cgroups can control different kinds of resources without turning the whole feature into one giant rule system.

Where Do You Find Cgroups on Linux?

Modern Linux systems generally use cgroup v2, although some systems and software may still expose or use cgroup v1.

The two versions are different interfaces, so don’t mix their commands and files.

For this lesson, we’ll use cgroup v2.

You can check whether your system is using cgroup v2 with:

mount | grep cgroup

A cgroup v2 mount normally looks similar to:

cgroup2 on /sys/fs/cgroup type cgroup2

You can also inspect the filesystem:

ls /sys/fs/cgroup

On a cgroup v2 system, you’ll see files such as:

cgroup.procs
cgroup.controllers
cgroup.subtree_control
memory.current
memory.max
cpu.max
pids.max

The names are useful because they make the purpose fairly obvious.

Why Are These Files There?

Cgroups expose their configuration through a virtual filesystem.

You create a cgroup by creating a directory inside this virtual filesystem.

For example:

sudo mkdir /sys/fs/cgroup/my-group

Now this directory is not an ordinary folder containing application files.

It represents a cgroup.

Inside it, the kernel exposes files for:

• which processes belong to the group
• which controllers are available
• resource limits
• current resource usage
• other cgroup settings

This is a recurring Linux idea:

filesystem interface
        ↓
kernel feature

You are reading and writing files, but the files represent kernel state.

Exercise: Let’s Limit Memory for a Process

Now let’s create a small cgroup ourselves.

We’ll use memory because it makes the effect easy to see.

Before Cgroup Implementation

We are using this simple C program that allocates 200MB and exits. Create memory.c for this:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void) {
    size_t size = 200 * 1024 * 1024;

    char *memory = malloc(size);

    if (memory == NULL) {
        perror("malloc");
        return 1;
    }

    printf("HI Before memset!\n");
    memset(memory, 0, size);

    printf("Allocated and touched 200 MB. Press Enter to exit...\n");
    getchar();

    free(memory);
    return 0;
}

Compile and run:

gcc memory.c -o memory
./memory

This should successfuly output:

HI Before memset!
Allocated and touched 200 MB. Press Enter to exit...

You can press Enter to exit the program. What this shows is 200 MB is happily allocated and touched (written). Now with the Cgroup exercise below, we are going to restrict this usage to 100MB only.

Create a Memory-Limited Cgroup

First create the group:

sudo mkdir /sys/fs/cgroup/memory-demo

Now give it a 100MB memory limit:

# Set RAM limit to 10MB
echo $((100 * 1024 * 1024)) | sudo tee /sys/fs/cgroup/memory-demo/memory.max

Because OS can also make use of swap memory when main memory is not available, set swap memory limit anything below 100MB. This is because our C program is allocating 200MB but 100MB is allowed by main memory. Let me go with 99MB of hard limit for swap memory, so that in total will be 199MB which is still 1MB less than what is required.

# Force Swap limit to ZERO for this cgroup
echo $((99 * 1024 * 1024)) | sudo tee /sys/fs/cgroup/memory-demo/memory.swap.max

Check the value:

cat /sys/fs/cgroup/memory-demo/memory.max
cat /sys/fs/cgroup/memory-demo/memory.swap.max

You should see:

104857600
103809024

The value is in bytes.

So we have:

memory-demo
      │
      |── memory.smax = 100 MB
      └── memory.swap.max = 200 MB

Put a Process Into the Cgroup

The cgroup.procs file contains the processes belonging to the cgroup.

To run a process inside the group, we can start a shell and then move that shell into the cgroup.

First get the shell’s PID:

echo $$

Then attach the current shell:

echo $$ | sudo tee /sys/fs/cgroup/memory-demo/cgroup.procs

Check:

cat /sys/fs/cgroup/memory-demo/cgroup.procs

You should see the shell’s PID.

From this point, processes created by that shell can be part of the cgroup as well.

Check the Memory Usage

You can see the group’s current memory usage with:

# main memory usage
cat /sys/fs/cgroup/memory-demo/memory.current
# swap memory usage
cat /sys/fs/cgroup/memory-demo/memory.swap.current

You can also see the limit:

# max limit of main memory 
cat /sys/fs/cgroup/memory-demo/memory.max
# max limit of swap memory 
cat /sys/fs/cgroup/memory-demo/memory.swap.max

These files are saying:

memory.current      → how much memory the group is currently using
memory.max          → how much memory the group is allowed to use

memory.swap.current → how much swap memory the group is currently using
memory.swap.max     → how much swap memory the group is allowed to use

Create a Process That Uses a Lot of Memory

Now let’s see if our memory program will allocate 200MB or not:

./memory

You will get:

HI Before memset!
Killed

The program was ok before memset i.e. before it tries to write 200MB but when it tries, Cgroup policy doesn’t allow it.

The program tries to keep the allocated memory alive until you press Enter. But our Cgroup allows only about 100MB and swap is 99MB, combining both is only 199MB — still 1MB less than what’s requried and that was enough for the program to crash.

Conceptually:

`./memory` process (child of the shell process)
      │
      ▼
memory-demo cgroup
      │
      ├── current usage
      │
      └── maximum = 100 MB or 199 MB with swap
                    │
                    ▼
             process asks for
                ~200 MB
                    │
                    ▼
             limit is reached

Depending on the rest of the system and the exact point at which the limit is reached, the process may be killed by the kernel’s memory controller rather than successfully reaching 200 MB.

That’s the important observation:

The process doesn’t get a private 100 MB machine. The cgroup tells the kernel that the group cannot exceed its configured memory limit.

Observe the Result

After the test, inspect:

cat /sys/fs/cgroup/memory-demo/memory.events

You may see counters such as:

low 0
high 0
max 829
oom 1
oom_kill 1
oom_group_kill 0

memory.peak records the highest memory usage reached by the cgroup since it was created (or since the counter was last reset). Same thing applies for swap memory usage as memory.swap.peak. You can inspect these values:

# peak memory usage since creation
cat /sys/fs/cgroup/memory-demo/memory.peak
# peak swap memory usage since creation
cat /sys/fs/cgroup/memory-demo/memory.swap.peak

I think you are already guessing the values:

104857600
103809024

These are the same max limits you configured before and ./memory program explodes them all.

What Did We Actually Do?

The entire exercise was basically:

Create cgroup
     ↓
Set memory.max
     ↓
Put process in cgroup
     ↓
Process uses memory
     ↓
Kernel enforces the limit

Notice that we never changed the C program to “understand” cgroups. The program doesn’t need to know. The kernel enforces the rule from outside the program.

What Else Can Cgroups Control?

Memory is only one example.

CPU

The CPU controller can control how much CPU time a group receives or how it is weighted relative to other groups.

Conceptually:

CPU
├── important-work → more CPU
└── background-work → less CPU

The exact controls in cgroup v2 are exposed through files such as cpu.max and cpu.weight.

Process Count

The pids controller can limit how many processes a group can create.

This is useful for preventing a runaway program from creating thousands of processes.

For example:

application-group
        │
        └── pids.max = 100

The group cannot simply keep creating processes forever.

Disk I/O

The io controller can control or account for block-device I/O.

This is different from limiting how much disk space a process can store.

For example:

I/O limit
    → how quickly / how much data can be read or written

Disk capacity
    → how much data can actually be stored

Cgroups are primarily about controlling resources such as CPU, memory, I/O, and process count. They are not a general-purpose “give this process only 1 GB of disk space” mechanism.

That distinction matters because it is easy to confuse disk I/O limits with disk space quotas.

Why Do Containers Care About Cgroups?

Now the connection to containers becomes straightforward.

A container is still made from ordinary Linux processes.

For example:

Container
├── web server
├── worker
└── helper process

Without resource controls, those processes could compete with everything else on the host.

Cgroups let the container runtime put those processes into a group and give that group resource limits.

Conceptually:

Linux Host
│
├── Container A
│   ├── process
│   └── process
│       └── cgroup A
│           ├── memory limit
│           ├── CPU rules
│           └── process limit
│
└── Container B
    ├── process
    └── process
        └── cgroup B
            ├── memory limit
            ├── CPU rules
            └── process limit

This is why a container can be given something like:

Memory: 512 MB
CPU:    limited
PIDs:   limited

without the application itself knowing anything about Docker or cgroups.

The container runtime configures the kernel; the kernel enforces the rules.

Cgroups vs Namespaces

This distinction is worth remembering because both are important to containers:

Namespaces
    → "What can this process see?"

Cgroups
    → "How much can this group use?"

For example:

Namespace
    → hide or isolate processes

Cgroup
    → limit their memory

They solve different problems and work together.

The Mental Model to Keep

Think of a cgroup as a resource-controlled room for processes.

                 Linux Host
                     │
          ┌──────────┴──────────┐
          │                     │
      normal work          cgroup room
                                │
                     ┌──────────┼──────────┐
                     │          │          │
                  process    process    process
                     │          │          │
                     └──────────┼──────────┘
                                │
                         resource rules
                         ├── CPU
                         ├── memory
                         ├── I/O
                         └── process count

The three ideas worth remembering are:

1. Cgroup = group of processes

2. Controllers = resource-specific rules

3. Kernel = enforces those rules

Or, even more simply:

Namespaces control what a process can see. Cgroups control how much a group of processes can use.

That combination is a major part of what makes Linux containers possible.