Autoscaling lag: why new servers arrive too late
Autoscaling adds replicas when a metric — usually CPU — passes a target. But new replicas take time to start, and the metric is averaged over an evaluation window first.
What causes it
Between the moment demand rises and the moment new capacity serves traffic, there are two delays: the evaluation window (often 1–5 minutes of averaging) and the start-up time (seconds for containers, minutes for VMs). During that gap the existing replicas absorb everything.
How to recognise it
- Replicas “starting” while existing ones are saturated
- Latency spikes at the start of every ramp
- Capacity catching up only after the peak
The math
Desired replicas = ⌈current × utilisation ÷ target⌉
How to fix it
- Scale earlier (lower target CPU)
- Shorten start-up time
- Keep more minimum replicas
Reproduce it in the simulator
Open the simulator, build a small architecture and use Fault injection in the inspector, or push traffic past a component’s capacity.
Learn more
Part of the guide Load balancing and capacity planning.