KubePodNotReady: what to do when it fires

Alert rule

alert: KubePodNotReady
expr: sum by (namespace, pod) (kube_pod_status_phase{phase=~"Pending|Unknown"}) > 0
for: 15m
severity: warning

Alert message: Pod {{ $labels.namespace }}/{{ $labels.pod }} is not ready

What it means

A Pod has been in the Pending or Unknown phase for 15 minutes: Kubernetes cannot start it on a node or has lost contact with the node.

Common causes

Not enough resources (CPU/memory) on the nodes, nodeSelector/affinity/taints that cannot be satisfied, a PVC that does not bind, an unavailable node, namespace resource quotas.

Diagnostics

kubectl -n <ns> describe pod <pod>              # Events section: FailedScheduling and the reason
kubectl get nodes
kubectl top nodes
kubectl -n <ns> get pvc
kubectl -n <ns> describe resourcequota

How to fix

Add resources/nodes, relax the scheduling constraints, fix the PVC/StorageClass, bring the node back. If the requests were badly chosen, reduce them.

When it is OK to silence

If the Pod is waiting on purpose (for example for a GPU node that does not exist yet) and this is known.

Related alerts

The full pack of 44 tested rules + Alertmanager → Telegram: 790 ₽ (≈ $9, card payment in rubles)
Open in Telegram · Free Monitoring Lite kit · All runbook alerts