Distributed Cache System – System Design Interview Preparation

Practice designing a distributed caching system like Redis. Cover consistent hashing, replication, eviction policies, and fault tolerance.

This is a full mock interview round, not a written solution to read through. The AI interviewer poses the Distributed Cache design problem, pushes back on your choices, and asks the follow-up questions a real interviewer would: how the system behaves as traffic grows, where the bottlenecks move, which consistency guarantees you are trading away, and what happens when a component fails. You get structured feedback and a score on requirement gathering, high level architecture, data modelling, scalability and communication at the end of every attempt.

Designing Distributed Cache is a recurring question in High Level Design (HLD) rounds at Google, Amazon, Meta, Microsoft, Netflix and Uber, and at high-growth product companies hiring SDE-2 and above. Working through it out loud - repeatedly, under time pressure - is what separates candidates who know the components from candidates who can defend a design.

What this question is really testing

Key distribution that survives nodes joining and leaving, and an eviction and invalidation story that does not silently serve stale data.

Scale to establish before drawing anything

Millions of operations a second with sub-millisecond latency - the cache is in front of everything.
Hundreds of GB to several TB across a cluster, so one node cannot hold the keyspace.
Node membership changes routinely, and a naive hash-modulo-N remaps nearly every key when it does.

Core components

Consistent hashing ring – Maps keys to nodes so adding a node moves roughly 1/N of keys rather than all of them.
Virtual nodes – Many ring positions per physical node, which is what makes the distribution even.
Eviction policy – LRU, LFU or TTL per key, chosen from the access pattern.
Replication – Each key on R nodes so a single failure does not cause a miss storm.
Client library – Holds the ring, routes directly, and handles failover. Avoids a proxy hop.

Trade-offs to defend

Consistent hashing with virtual nodes – Plain modulo hashing remaps almost everything on a membership change, which is a cold cache and a thundering herd on the database. This is the core of the answer.
Write-through versus write-back versus cache-aside – Cache-aside is the common default and leaves a stale window; write-through is consistent and slower; write-back is fast and can lose data.
Hot key handling – One key taking 30% of traffic defeats even distribution. Replicating that key or fronting it locally is the fix.

Follow-up questions to expect

A node dies holding 10% of the keyspace. Describe the next 30 seconds at the database.
A key expires and 10,000 requests miss simultaneously. What stops all of them hitting the database?
How does a writer invalidate a key that three services cached independently?