NodeMemoryLow: what to do when it fires

Alert rule

alert: NodeMemoryLow
expr: node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes * 100 < 10
for: 10m
severity: warning

Alert message: Low available memory ({{ $labels.instance }})

What it means

Less than 10% of RAM is available (based on MemAvailable, i.e. counting cache the kernel can give back). The system still works, but the headroom is small.

Common causes

A memory leak in an application, too many workers, growth of an application cache (Redis, JVM), a heavy task being run.

Diagnostics

free -h
ps aux --sort=-%mem | head -n 10
smem -tk 2>/dev/null | tail -n 15      # if smem is installed
docker stats --no-stream               # if Docker is present
dmesg -T | grep -i -E 'oom|killed process' | tail

How to fix

Find the consuming process, restart it if it leaks, reduce the number of workers or the cache size, add RAM. For a JVM, check -Xmx.

When it is OK to silence

On servers where memory is deliberately used for cache (databases with a large shared_buffers and the like), but then it is better to raise the threshold in the rule to suit your profile.

Related alerts

• NodeMemoryCritical
• NodeInodesLow
• NodeSwapHigh
• NodeDiskWillFill
• NodeCpuHigh

The full pack of 44 tested rules + Alertmanager → Telegram: 790 ₽ (≈ $9, card payment in rubles)
Open in Telegram · Free Monitoring Lite kit · All runbook alerts