High load warnings: catching overload before it fails
Most outages announce themselves. A component running at 80% CPU or memory, or with a half-full queue, has little headroom left for the next spike.
What causes it
Because waiting grows like ρ ÷ (1 − ρ), the move from 80% to 95% utilisation costs far more latency than the move from 50% to 80%. Alerting at 80% gives time to add capacity before the steep part of the curve.
How to recognise it
- Utilisation above 80% for minutes, not seconds
- Queues starting to fill
- Latency creeping up while errors are still zero
The math
Headroom = 1 − ρ
How to fix it
- Plan more capacity for this component
Reproduce it in the simulator
Open the simulator, build a small architecture and use Fault injection in the inspector, or push traffic past a component’s capacity.
Learn more
Part of the guide Load balancing and capacity planning.