Disk full: when databases and logs run out of space
A full disk does not crash a database the way memory does — it quietly stops it from writing. Reads keep working, which makes the failure easy to miss until users complain.
What causes it
Every write adds data: rows, indexes, write-ahead logs, messages kept for retention. If data arrives faster than it is deleted or archived, the disk fills at a predictable rate. PostgreSQL and MySQL refuse writes, Elasticsearch marks indices read-only at its flood-stage watermark, and Kafka brokers stop accepting producers.
How to recognise it
- “No space left on device” in database logs
- Writes fail while reads still succeed
- Disk usage graph rising in a straight line
The math
Time until full = free space ÷ (write rate × bytes per write)
Error messages you will see
| Error | Seen in |
|---|---|
No space left on device | Users, Router, Server, Database, Cache, Queue and 195 more |
ERROR: could not extend file: No space left on device | PostgreSQL, CockroachDB, Amazon RDS (PostgreSQL), Amazon Aurora (PostgreSQL), Cloud SQL (PostgreSQL), AlloyDB for PostgreSQL (PostgreSQL) and 5 more |
ERROR 1114 (HY000): The table is full | MySQL, MariaDB, Vitess, Amazon RDS (MySQL), Amazon RDS (MariaDB), Amazon Aurora (MySQL) and 3 more |
How to fix it
- Increase the disk size
- Shorten retention or archive old data
- Alert on disk usage before it fills
Reproduce it in the simulator
Open the simulator, build a small architecture and use Fault injection in the inspector, or push traffic past a component’s capacity.
Learn more
Part of the guide Message queues vs event streams.