Русская версия

KubeNodePressure: what to do when it fires

Alert rule

alert: KubeNodePressure
expr: kube_node_status_condition{condition=~"MemoryPressure|DiskPressure|PIDPressure",status="true"} == 1
for: 5m
severity: warning

Alert message: Node {{ $labels.node }} has condition {{ $labels.condition }}

What it means

A node has reported one of the resource-shortage conditions: MemoryPressure, DiskPressure or PIDPressure (the name is in the condition label), for 5 minutes. Kubelet may start evicting Pods.

Common causes

A shortage of memory (many Pods without limits), a full disk (images, logs, emptyDir), too many processes.

Diagnostics

kubectl describe node <node>                    # Conditions
kubectl top node <node>
ssh <node> 'df -h /var/lib/kubelet /var/lib/containerd; free -h'
ssh <node> 'crictl images | head; crictl ps | wc -l'
kubectl get pods -A --field-selector spec.nodeName=<node> | grep -i evicted

How to fix

For DiskPressure, clean up images and logs (crictl rmi --prune) and expand the disk. For MemoryPressure, set requests/limits and redistribute Pods. For PIDPressure, find the process that spawns threads.

When it is OK to silence

A short-lived peak with a known cause; not if it recurs.

Related alerts

The full pack of 44 tested rules + Alertmanager → Telegram: 790 ₽ (≈ $9, card payment in rubles)
Open in Telegram · Free Monitoring Lite kit · All runbook alerts