Cache eviction: when your working set does not fit in memory
A cache can only help with keys it holds. If the working set — the keys your traffic actually uses — is bigger than the cache’s memory, keys are evicted before they are reused and the hit ratio falls.
What causes it
Memory needed ≈ number of hot keys × item size. With Redis’s allkeys-lru policy the oldest keys are evicted to make room; with noeviction, writes to a full cache fail with “OOM command not allowed”. Either way, more of your reads reach the database.
How to recognise it
- Evicted keys counter rising
- Hit ratio well below what traffic patterns suggest
- Redis memory pinned at maxmemory
The math
Memory needed ≈ working-set keys × item size
How to fix it
- Add memory or shard the cache
- Store smaller items
- Shorten TTLs for rarely used keys
Reproduce it in the simulator
Open the simulator, build a small architecture and use Fault injection in the inspector, or push traffic past a component’s capacity.
Learn more
Part of the guide Caching in system design.