alert: NodeInodesLow
expr: node_filesystem_files_free{fstype!~"tmpfs|overlay|squashfs|devtmpfs|ramfs|iso9660|autofs|fuse.*|nfs.*|cifs"}
/ node_filesystem_files{fstype!~"tmpfs|overlay|squashfs|devtmpfs|ramfs|iso9660|autofs|fuse.*|nfs.*|cifs"} * 100 < 10
and node_filesystem_files{fstype!~"tmpfs|overlay|squashfs|devtmpfs|ramfs|iso9660|autofs|fuse.*|nfs.*|cifs"} > 0
for: 10m
severity: warningAlert message: Running out of inodes on {{ $labels.mountpoint }} ({{ $labels.instance }})
Less than 10% of inodes are free on the filesystem. Once they run out, no new file can be created even if there is disk space (the No space left on device error while df -h shows plenty of room).
Millions of small files: PHP sessions, caches, the mail queue, temporary files, node_modules directories, old kernels in /usr/src.
df -i
for d in /var /tmp /home /usr; do echo "$d: $(find $d -xdev 2>/dev/null | wc -l)"; done
find /var -xdev -type d -size +1M 2>/dev/null | head # directories with huge file counts
du -x --inodes --max-depth=2 /var 2>/dev/null | sort -n | tail -n 15The --inodes option of du is available in GNU coreutils.
Delete the unneeded small files in the directory you found (find <dir> -type f -mtime +7 -delete, after checking the list without -delete) and set up cache/session cleanup. On ext4 the number of inodes is fixed when the filesystem is created and cannot be changed afterwards; you can only recreate the filesystem or move the data.
For filesystems with a dynamic number of inodes (btrfs, xfs) that produce false positives, exclude them from the expression.
The full pack of 44 tested rules + Alertmanager → Telegram: 790 ₽ (≈ $9, card payment in rubles)
Open in Telegram · Free Monitoring Lite kit · All runbook alerts