Understanding Out of Memory Killer (OOM Killer) in Linux

Abhishek Prakash -

Out of Memory (OOM) is a condition that occurs in Linux when the system starts running out of memory (RAM) because system is under heavy load or some processes are using too much memory.

This may result in the program crashing and the kernel panicking. This is when OOM Killer comes into the picture.

What is OOM Killer in Linux?

The Out of Memory Killer, or OOM Killer, is a mechanism in the Linux kernel that handles the situation when the system is critically low on memory (physical or swap). It verifies that the system is truly out of memory and if that's the case, it selects a process to kill it and free the memory.

But why does this happen? Does the Linux kernel not know how to handle memory properly?

Well, yes and no. Here's the thing. When a process starts, it demands a chunk of memory from the kernel. The Linux kernel understands that the process will use all the allotted memory immediately or ever. So, it 'over allocates' the memory to the process.

Let's say a Linux server is running eight applications that demand a total of 5 GB. The kernel may "allocate" 5 GB of memory even if the system has only 4 GB RAM. It's because not all applications will utilize their allocated memory quota all the time.

However, if enough applications start using all of their requested memory blocks, there will not be enough memory to allocate, and thus, the Linux system will run out of memory.

This is when OOM Killer feature of Linux kernel jumps into action and kills one or more processes based on their OOM score and frees up memory to keep the system running.

What is OOM Score?

Every running process in Linux has an oom_score. The operating system calculates it based on several criteria, which are mainly influenced by the amount of memory the process is using. Typically, oom_score varies between -1000 and 1000. The process with the highest oom_score gets killed first by OOM Killer when the system starts running critically low on memory.

You can check the oom_score of a running process using its process ID:

cat /proc/process_ID/oom_score

You probably know that /proc is an interface for Linux processes. It has all kinds of details on all the running processes.

You can change the OOM Score of a process in this fashion:

echo -200 > /proc/process_ID/oom_score_adj
💡
You can check the logs and see if a process was killed because of out of memory:

grep -i kill /var/log/messages*

It should show something like this:

host kernel: Out of Memory: Killed process 2763

More on OOM in Linux

Here are a few resources you may explore to learn more on the concept of OOM in Linux:

Here's a funny comic by turnoff.us which you may find more funny now that you know about OOM Killer.

Let me know if you have questions or suggestions.