Summary

• The false alarm: I thought I was under a massive DDoS attack or exploited. Turns out, I just accidentally choked my own server.
• The death loop: MySQL ran out of memory, Ubuntu's crash reporter tried to dump gigabytes of RAM to disk, and Docker kept restarting the container. Cue 400% CPU and 30,000 IOPS.
• The $0 fix: Bypassing the need to upgrade my VPS by simply creating a 4GB Swap file and tweaking the kernel's swappiness.
• The architecture reality: Why Swap makes total sense for a cheap, personal VPS running 50 idle microservices, but is absolute garbage for AWS ECS or Kubernetes.

Introduction

Recently, I was spammed with notifications about my portfolio websites going down.

Initially, I brushed it off. I figured it was just general instability with Hetzner. But after taking a closer look at their status page, I was alarmed. Everything was green.

I logged into Hetzner to check the VPS performance and my stomach dropped. The vCPU was pegged at 400%. Disk writes were screaming at 2 GB per second. I thought I was under attack 😨. I couldn't access any of my websites. I couldn't even SSH in. The terminal just hung there.

This time around, I had heavily hardened my VPS. No alerts of zero-day exploits. My applications were completely containerised. What was going on?

I had to force a shutdown from Hetzner just to gain SSH access again. Upon getting back in, I started digging through the logs.

The Autopsy

I checked the kernel logs from the previous boot using a simple

sudo journalctl -k -b -1 | grep -i "out of memory" -A 10.

There it was. Plain as day.

Out of memory: Killed process 3111 (mysqld)

Nobody was hacking me. I did this to myself.

My server runs about 50 microservice containers on a single 7.6GB RAM VPS. We're talking three full product stacks (Singhealth, Mappalette, Yorkshire) plus my personal blog. Because Node.js, Java JVMs, and database pools hoard idle memory, my baseline RAM was sitting at around 70% capacity before any actual traffic hit.

When MySQL tried to process a query, it spiked its memory allocation and pushed total system RAM past 100%. Because Linux had zero safety cushion, the kernel invoked the OOM (Out Of Memory) Killer and brutally executed the MySQL process.

But that didn't explain the insane disk I/O.

When MySQL died, Ubuntu launched apport (its automatic crash reporting tool). apport tried to dump gigabytes of raw memory state onto the NVMe disk as fast as the hardware could physically spin. That flooded the disk queue, hitting 30,000 IOPS and 2 GB/s throughput. Then, Docker saw MySQL was dead and instantly restarted it. MySQL crashed again. apport ran again.

A complete death loop.

Enter Swap

When physical RAM fills up, Linux has two choices. It can kill a process immediately, or it can temporarily offload cold, unused memory pages onto the disk. That disk space is Swap.

My server had 0 bytes of swap.

At a low level, RAM is divided into pages (usually 4 KB chunks). The Linux kernel constantly tracks which pages are "hot" (being actively used by the CPU) and which are "cold" (idle data, like a background worker waiting for a cron job). When memory gets tight, the kernel's Page Frame Reclaim Algorithm (PFRA) writes those cold pages to the swap file on disk. If a process suddenly needs that data, a page fault occurs, and the kernel pulls it back into RAM.

I ran a few simple commands to allocate a 4GB file on disk, format it, and turn it on:

sudo fallocate -l 4G /swapfile

sudo mkswap /swapfile

sudo swapon /swapfile

But you can't just turn it on and walk away. You have to tune vm.swappiness. This is a kernel parameter from 0 to 100 that dictates how aggressively Linux trades RAM for Swap. The default is 60. I set mine to 10. That basically tells the OS: "Use physical RAM as much as humanly possible, and only touch the swap file if we are in absolute emergency territory."

I also disabled apport completely so it stops thrashing my disk every time a container hiccups.

The Architectural Dilemma: VPS vs Cloud

If you ask a DevOps purist about swap, they will laugh at you. They'll say you need to fix your resource limits. And for enterprise production environments, they are right. But this is a personal portfolio server.

VPS pricing models are notoriously rigid. If I want more RAM on Hetzner, I am forced to upgrade my vCPU count, which instantly doubles my monthly bill. I have 50 microservices. 90% of them are sitting idle waiting for a recruiter or guest to click a link. Buying a 16 GB VPS just to hold idle Node.js memory is a massive waste of cash. Creating a 4 GB swap file on a fast NVMe drive costs nothing.

But if I were running this in AWS ECS, EKS, or Lambda? I would never use Swap.

In cloud infrastructure, you want predictable latency. A page fault reading from disk adds latency spikes. It is way better for a pod to crash instantly (fail fast) so Kubernetes can just reschedule it on a node with free RAM. Plus, AWS lets you scale memory independently of CPU anyway.

Conclusion

I'm keeping my VPS. It is incredibly easy to assume your provider is having an outage or some hacker is hammering your ports. But sometimes, it's just a misconfigured config file or an unconstrained database.

Leaving heavy apps without memory limits in Docker means they will treat your host RAM like an open buffet until the OS crashes. A simple swap file was the difference between a minor latency blip and my server locking me out completely.