KubeApiCertExpiresSoon: what to do when it fires

Alert rule

alert: KubeApiCertExpiresSoon
expr: histogram_quantile(0.01, sum by (instance, le) (
  rate(apiserver_client_certificate_expiration_seconds_bucket{job="apiserver"}[5m])
)) / 86400 < 7
and histogram_quantile(0.01, sum by (instance, le) (
  rate(apiserver_client_certificate_expiration_seconds_bucket{job="apiserver"}[5m])
)) / 86400 >= 1
for: 10m
severity: warning

Alert message: kube-apiserver client certificate expires in less than 7 days ({{ $labels.instance }})

What it means

The client certificates used to access kube-apiserver expire in less than 7 days (and not less than 1 day). The apiserver_client_certificate_expiration_seconds metric is not available in all distributions; the rule works only if it is present. The value is an estimate based on the lowest 1% of observed certificates.

Common causes

The certificates were not renewed automatically (kubeadm did not renew them, the certificates of kubeconfig users, service accounts or components expired).

Diagnostics

kubeadm certs check-expiration           # for kubeadm clusters, on the control plane
openssl x509 -noout -enddate -in /etc/kubernetes/pki/apiserver.crt
kubectl config view --raw -o jsonpath='{.users[*].user.client-certificate-data}' | base64 -d | openssl x509 -noout -enddate

How to fix

For kubeadm: kubeadm certs renew all on each control plane node, then restart the control plane components. In managed clusters (EKS/GKE/AKS and the like) the provider manages the certificates — see its documentation.

When it is OK to silence

If you already know about the expiry and the renewal is scheduled before the critical deadline.

Related alerts

• KubeApiCertExpiresCritical
• KubeHpaMaxedOut
• KubeNodePressure
• KubeNodeNotReady
• KubePvcCritical

The full pack of 44 tested rules + Alertmanager → Telegram: 790 ₽ (≈ $9, card payment in rubles)
Open in Telegram · Free Monitoring Lite kit · All runbook alerts