KubePvcAlmostFull: what to do when it fires

Alert rule

alert: KubePvcAlmostFull
expr: kubelet_volume_stats_available_bytes
  / kubelet_volume_stats_capacity_bytes * 100 < 15
and kubelet_volume_stats_capacity_bytes > 0
for: 10m
severity: warning

Alert message: PVC {{ $labels.namespace }}/{{ $labels.persistentvolumeclaim }} is filling up

What it means

Less than 15% is free on a PersistentVolumeClaim volume. The metrics come from kubelet and are not available for all volume types.

Common causes

Database or log growth, no rotation, accumulated dumps and temporary files.

Diagnostics

kubectl -n <ns> get pvc <name>
kubectl -n <ns> get pods -o wide | grep <pod-using-the-volume>
kubectl -n <ns> exec <pod> -- df -h <mount-path>
kubectl -n <ns> exec <pod> -- du -xh --max-depth=2 <mount-path>
kubectl get storageclass

How to fix

Clean up what is not needed or expand the PVC (if the StorageClass supports allowVolumeExpansion: true): kubectl -n <ns> patch pvc <name> -p '{"spec":{"resources":{"requests":{"storage":"<new-size>"}}}}'. The size cannot be reduced.

When it is OK to silence

If the growth is a one-off and an expansion is planned.

Related alerts

• KubePvcCritical
• KubeJobFailed
• KubeNodeNotReady
• KubeDaemonSetNotReady
• KubeNodePressure

The full pack of 44 tested rules + Alertmanager → Telegram: 790 ₽ (≈ $9, card payment in rubles)
Open in Telegram · Free Monitoring Lite kit · All runbook alerts