DockerContainerRestartLoop: what to do when it fires

Alert rule

alert: DockerContainerRestartLoop
expr: changes(container_start_time_seconds{name!=""}[15m]) >= 3
for: 1m
severity: critical

Alert message: Container {{ $labels.name }} keeps restarting ({{ $labels.instance }})

What it means

In the last 15 minutes the container has been started again 3 or more times (judging by changes in container_start_time_seconds). The process crashes right after starting and Docker restarts it.

Common causes

An error in the configuration or environment variables, an unavailable dependency (database, network), out of memory (OOM), a wrong image or command, a missing file or volume.

Diagnostics

docker ps -a --filter name=<name>
docker inspect <name> --format '{{.State.ExitCode}} {{.State.OOMKilled}} {{.RestartCount}}'
docker logs --tail 200 <name>
docker events --since 15m --filter container=<name>
docker stats --no-stream <name>

How to fix

Fix the cause using the last lines of the log before the crash. Exit code 137 usually means SIGKILL (including OOM), 1 means an application error. If the cause is memory, see DockerContainerOomKilled.

When it is OK to silence

When debugging on a test setup. Not in production.

Related alerts

• DockerContainerOomKilled
• DockerContainerCpuThrottled
• DockerContainerGone
• DockerContainerMemoryNearLimit
• DockerCadvisorDown

The full pack of 44 tested rules + Alertmanager → Telegram: 790 ₽ (≈ $9, card payment in rubles)
Open in Telegram · Free Monitoring Lite kit · All runbook alerts