Under-replicated partitions: one failure away from data loss
A partition is under-replicated when it has fewer live copies than its replication factor. The system still works, but it has lost its safety margin.
What causes it
A broker went down, or is too slow to keep up with its partitions. Until it recovers, another failure could make partitions unavailable or, with relaxed settings, lose acknowledged data.
How to recognise it
- UnderReplicatedPartitions metric above zero
- A broker missing from the cluster
- Producers still succeeding — for now
How to fix it
- Restart the broker
- Add brokers
Reproduce it in the simulator
Open the simulator, build a small architecture and use Fault injection in the inspector, or push traffic past a component’s capacity.
Learn more
Part of the guide Message queues vs event streams.