Troubleshooting Low System Performance: Optimizing Swap Configuration on Ubuntu 22.04 LTS

Resolve sluggish Ubuntu 22.04 performance caused by inadequate or misconfigured swap memory. Learn to monitor, create, and tune swap space for optimal system responsiveness.

Introduction

As an experienced SysAdmin, you've likely encountered the frustrating scenario of a seemingly robust Ubuntu server inexplicably grinding to a halt. While many factors contribute to system performance, a common culprit, often overlooked in modern cloud environments, is an improperly configured or insufficient swap memory. On Ubuntu 22.04 LTS, a server struggling with memory pressure due to inadequate swap can manifest as unresponsive web applications (e.g., Nginx serving 502/504 errors for PHP-FPM or Node.js), delayed SSH logins, high load averages, or even unexpected service restarts triggered by the Out-Of-Memory (OOM) killer. This guide provides a highly technical, step-by-step approach to diagnose and resolve swap-related performance bottlenecks.

Symptom & Error Signature

Users experiencing low system performance due to swap issues typically observe one or more of the following:

• System Unresponsiveness: Applications like web servers (Nginx), databases (PostgreSQL, MySQL), or containerized services (Docker) become slow or unresponsive, leading to timeouts.
• High Load Average: The system load average (visible via top, htop, or uptime) is consistently high, even when CPU utilization isn't at 100%. This often indicates heavy I/O wait.
• Out-Of-Memory (OOM) Killer Invocations: The Linux kernel's OOM killer steps in to terminate processes when physical RAM is exhausted and swap is either full, too small, or not being used effectively.
• Excessive Disk I/O: Monitoring tools show high disk I/O, particularly on the root filesystem, which can be exacerbated by inefficient swapping.

Here are typical log entries or command outputs indicative of swap-related performance issues:

1. OOM Killer in dmesg or syslog:

[ 1234.567890] oom-kill:constraint=UNBOUNDED, nodemask=(null), cpuset=/,mems_allowed=0,global_oom, task_name=php-fpm, pid=12345, uid=33
[ 1234.567891] Out of memory: Killed process 12345 (php-fpm) total-vm:4194304kB, anon-rss:2097152kB, file-rss:0kB, shmem-rss:0kB, UID:33 pgtables:4194304kB oom_score_adj:0
[ 1234.567892] Memory cgroup out of memory: Killed process 54321 (docker-compose)

2. High I/O Wait Percentage (top/htop):

top - 12:30:00 up 1 day, 1:23,  1 user,  load average: 8.50, 7.20, 6.10
Tasks: 250 total,   1 running, 249 sleeping,   0 stopped,   0 zombie
%Cpu(s):  5.1 us,  2.3 sy,  0.0 ni, 12.5 id, **78.9 wa**,  0.0 hi,  0.2 si,  1.0 st
MiB Mem :   7987.0 total,    120.5 free,   7500.0 used,    366.5 buff/cache
MiB Swap:    **511.0 total**,     0.0 free,    **511.0 used**,    300.0 avail Mem

Notice the 78.9 wa (I/O wait) and 511.0 used swap with very little free physical memory.

3. Small or No Swap Space (free -h, swapon --show):

free -h

               total        used        free      shared  buff/cache   available
Mem:           7.8Gi       7.3Gi       123Mi       1.0Mi       400Mi       100Mi
Swap:          **511Mi       511Mi          0B**

The above shows an undersized 511MB swap file completely utilized.

swapon --show

NAME      TYPE      SIZE USED PRIO
/swapfile file      511M 511M   -2

This confirms a single, small swap file that is fully in use.

4. journalctl entries related to memory pressure:

journalctl -r -n 50 | grep -i "memory|swap"

Jul 25 10:05:01 webserver kernel: Memory cgroup stats for system.slice: anon 2048000kB cache 512000kB slab 102400kB shmem 0kB kernel_stack 16384kB pagetables 81920kB vm_swap 0kB
Jul 25 10:04:59 webserver kernel: (OOM) php-fpm: page allocation failure: order:4, mode:0xcc0(GFP_KERNEL|__GFP_COMP), nodemask=(null)
Jul 25 10:04:59 webserver kernel: Mem-Info:
Jul 25 10:04:59 webserver kernel:  Normal free:123MB active_anon:7.0GB inactive_anon:0GB active_file:0GB inactive_file:0GB unevictable:0GB writeback:0GB dirty:0GB slab_reclaimable:50MB slab_unreclaimable:50MB kernel_stack:16MB pagetables:80MB bounce:0MB free_pcp:0MB local_pcp:0MB free_cma:0MB
Jul 25 10:04:59 webserver kernel: Swap cache: 0kB

Root Cause Analysis

Low system performance attributed to swap configuration on Ubuntu 22.04 LTS typically stems from one or more of these core issues:

1. Insufficient Swap Space: Many cloud providers or default installations for smaller VMs (especially those with SSDs) often configure minimal or no swap space, assuming ample RAM. While acceptable for burstable or lightweight workloads, demanding applications (databases, large Java applications, Docker containers) can quickly exhaust physical RAM. Without adequate swap, the kernel has no overflow mechanism, leading directly to OOM killer invocations.
2. Misconfigured vm.swappiness: The vm.swappiness kernel parameter (ranging from 0 to 100) dictates how aggressively the kernel swaps out idle processes from RAM to swap space.
   • swappiness=0 (or near 0): This tells the kernel to avoid swapping almost entirely unless absolutely necessary to prevent an OOM event. While this minimizes disk I/O, it can lead to OOM situations much faster if RAM fills up rapidly.
   • swappiness=100: The kernel aggressively swaps out even active processes, which can cause excessive disk I/O, leading to severe performance degradation as the system constantly pages data in and out of slow storage.
   • An inappropriate value for your workload (e.g., 0 for a database server or 60 for a low-latency web server) can lead to suboptimal memory management.
3. High vm.vfs_cache_pressure: This parameter controls the kernel's tendency to reclaim memory used for directory and inode caches. A high value (default is 100) means the kernel reclaims these caches aggressively, freeing memory but potentially degrading filesystem performance. A lower value keeps more of these caches in memory, potentially starving application memory if not balanced correctly. While not directly swap, it impacts overall memory availability.
4. Underlying Memory Leaks or Inefficient Applications: While swap helps mitigate the impact, it cannot fix applications that continuously consume more memory without releasing it. Persistent OOM events even with sufficient swap may indicate a memory leak within an application (e.g., a misconfigured PHP-FPM pool, an unoptimized Java application, or a Docker container exceeding its memory limits).

Step-by-Step Resolution

Follow these steps to diagnose, configure, and optimize swap memory on your Ubuntu 22.04 LTS server.

#### 1. Assess Current System Performance and Swap Status

Begin by gathering critical information about your server's current memory and swap configuration.

• Check Physical and Swap Memory Usage:

  free -h
  

  This command provides a human-readable summary of RAM and swap usage. Pay close attention to the total, used, and free columns for both Mem and Swap.

• View Active Swap Devices:

  swapon --show
  

  This shows which swap files or partitions are currently active, their size, and usage.

• Inspect Current Swappiness Setting:

  cat /proc/sys/vm/swappiness
  

  The output (a value between 0 and 100) indicates the kernel's current preference for swapping.

• Monitor Real-time System Resources:

  htop # or top
  

  Observe the Mem and Swp bars, %Cpu(s) (especially the wa or I/O wait percentage), and processes consuming high memory or CPU. Look for kswapd0 activity if the system is actively swapping.

• Check for OOM Killer Events:

  dmesg | grep -i oom
  journalctl -r -n 100 | grep -i "out of memory|oom-kill"
  

  These commands will reveal if the OOM killer has been active recently, providing clues about which processes were terminated due to memory exhaustion.

#### 2. Determine Optimal Swap Size

The "ideal" swap size depends on your system's RAM and workload. While there are old rules of thumb (e.g., 2x RAM), modern recommendations are more nuanced.

For servers, swap primarily acts as a safety net to prevent OOM errors, rather than a performance booster for active data. Excessive swapping indicates a need for more RAM, not just more swap.

General Guidelines:

• RAM < 2GB: Swap = 2 * RAM
• 2GB <= RAM < 8GB: Swap = 1 * RAM
• 8GB <= RAM < 64GB: Swap = 0.5 * RAM or minimum 4GB (e.g., for hibernation support or stability)
• RAM >= 64GB: Swap = 4GB – 8GB (often sufficient, as OOM is less likely with large RAM, and excessive swap just slows things down).

Consider your specific applications:

• Databases (PostgreSQL, MySQL): May benefit from more swap as a buffer during large queries or memory spikes, but typically prefer to stay entirely in RAM.
• Java Applications: Can be memory-hungry and might benefit from adequate swap, especially during garbage collection cycles.
• Docker Hosts: Each container consumes memory. The combined memory footprint can quickly exceed physical RAM.

#### 3. Create or Resize a Swap File

Most Ubuntu 22.04 LTS cloud instances use a swap file rather than a dedicated partition. We'll focus on creating/resizing a swap file.

If you already have an existing small swap file (swapon --show will tell you), you should disable it, remove it, and then create a new, larger one. Incorrectly modifying an active swap can lead to data loss or system instability.

a. (Optional) Disable and Remove Existing Swap File:

If you have an undersized /swapfile as shown in swapon --show, first disable it:

sudo swapoff /swapfile

Then, remove its entry from /etc/fstab:

sudo sed -i '/swapfile/d' /etc/fstab

And finally, delete the old file:

sudo rm /swapfile

b. Create a New Swap File (e.g., 4GB):

# Create a 4GB file named /swapfile. Adjust '4G' as per your determined optimal size.
sudo fallocate -l 4G /swapfile

c. Set Correct Permissions:

Swap files must have strict permissions to prevent unauthorized access.

sudo chmod 600 /swapfile

d. Designate the File as Swap Space:

sudo mkswap /swapfile

e. Activate the Swap File:

sudo swapon /swapfile

f. Verify New Swap Configuration:

free -h
swapon --show

You should now see the new, larger swap file listed and active.

g. Make Swap Persistent Across Reboots:

The swap file created above will not persist after a reboot unless added to /etc/fstab.

Edit /etc/fstab (use sudo nano /etc/fstab or sudo vim /etc/fstab) and add the following line to the end of the file:

/swapfile none swap sw 0 0

Alternatively, use tee for automation:

echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

This ensures the swap file is automatically mounted at boot.

#### 4. Adjust Swappiness (vm.swappiness)

This is a critical tuning parameter for server performance.

• Understanding swappiness:

  • 0 (or 1 on recent kernels): The kernel will only use swap when absolutely necessary to prevent an OOM event. It strongly prefers to keep processes in physical RAM. Good for servers where disk I/O is a bottleneck and you want applications to stay in memory.
  • 60 (default for Ubuntu): A balanced value for desktops, where the system might proactively swap out inactive applications to keep more free RAM for disk cache, improving responsiveness. For servers, this can be too aggressive, leading to unnecessary disk I/O.
  • 100: The kernel aggressively swaps out processes, even if they are relatively active. This should generally be avoided for servers.

• Recommendation for Servers: For most production servers (especially web servers, databases), a swappiness value between 10 and 30 is recommended.

  • 10: Suitable for critical servers where you want to minimize disk I/O and keep as much data as possible in RAM (e.g., database servers, caching layers).
  • 30: A good general-purpose value for other servers, providing a balance between avoiding OOM and minimizing unnecessary swapping.

a. Check Current Swappiness:

cat /proc/sys/vm/swappiness

b. Temporarily Set Swappiness (until next reboot):

To test the effect, set it temporarily:

sudo sysctl vm.swappiness=10

Replace 10 with your desired value.

c. Make Swappiness Persistent:

Changes made with sysctl are temporary. To make them permanent, add the setting to /etc/sysctl.conf.

Open /etc/sysctl.conf:

sudo nano /etc/sysctl.conf

Add or modify the following line:

vm.swappiness=10

Replace 10 with your chosen value. Save and exit the editor.

d. Apply Persistent Changes Immediately:

sudo sysctl -p

#### 5. Adjust Cache Pressure (vm.vfs_cache_pressure)

While less commonly tuned for general swap issues, vm.vfs_cache_pressure can impact overall memory availability.

• Understanding vfs_cache_pressure: This parameter controls the kernel's tendency to reclaim memory used for directory and inode caches (which are crucial for filesystem operations). A higher value means the kernel reclaims these caches more aggressively, freeing memory but potentially forcing more disk I/O for filesystem metadata. The default is 100.
• Recommendation: For most servers, the default 100 is acceptable. If you suspect filesystem I/O is a bottleneck due to cache thrashing and have sufficient RAM, lowering it to 50 to 80 can sometimes help keep more filesystem metadata in memory. Be cautious, as too low a value can lead to memory starvation for applications.

a. Check Current vfs_cache_pressure:

cat /proc/sys/vm/vfs_cache_pressure

b. Temporarily Set vfs_cache_pressure:

sudo sysctl vm.vfs_cache_pressure=50

Replace 50 with your desired value.

c. Make vfs_cache_pressure Persistent:

Add or modify the following line in /etc/sysctl.conf:

vm.vfs_cache_pressure=50

Replace 50 with your chosen value. Then apply changes:

sudo sysctl -p

#### 6. Monitor and Optimize Post-Configuration

After implementing these changes, continuous monitoring is crucial to ensure the desired performance improvements and to further fine-tune if necessary.

• Continuous Monitoring:

  • Regularly check free -h and swapon --show to observe memory and swap usage patterns.
  • Keep htop or top open to watch CPU usage (especially wa%), load average, and active processes.
  • Monitor application-specific logs for errors, timeouts, or resource exhaustion.
  • Check dmesg and journalctl for any new OOM killer events.
  • Utilize advanced monitoring tools like Prometheus/Grafana or Netdata to gather historical data on memory, swap, and I/O metrics.

• Iterative Tuning:

  • If performance is still an issue, incrementally adjust vm.swappiness (e.g., from 10 to 20 or 30) and re-evaluate.
  • If OOM events persist despite increased swap and optimized swappiness, this strongly suggests an application-level memory leak or an underlying need for more physical RAM. Consider memory profiling for the problematic application or upgrading your server's RAM.
  • For Dockerized environments, review container memory limits (--memory flag or memory in docker-compose.yml) to ensure they are appropriate and that containers are not exceeding available resources.

By systematically troubleshooting and optimizing your swap configuration, you can significantly enhance the stability and performance of your Ubuntu 22.04 LTS server, preventing frustrating slowdowns and OOM-related outages.