Caching
Memory is about 50x faster than a database round trip. A cache with a high hit ratio cuts latency and protects the database.
- ~50xmemory vs a database round trip
- 1 ms vs 50 mscache hit vs cache miss latency
- 90% to 99%hit ratio gain cuts database load 10x
Show what to say
“I use cache-aside: check Redis, and on a miss read the database and fill the key with a jittered TTL. On writes I update the database and delete the key rather than update it, because with concurrent writers the older value can land last. For hot keys I add a per-key lock, so a stampede rebuilds the value once.”
Trade-offs
- Cache-aside vs write-through: cache-aside only gets slower if the cache dies; write-through is always fresh but writes cost double.
- Write-behind vs write-through: write-behind is very fast, but a crash loses unflushed data; fine for counters and metrics.
- Delete vs update on write: delete costs the next reader one miss, but updating can leave the older value last when two writers race.
Bring it up before they ask
Even delete-on-write has a race: a slow reader can write the old value back after the delete, so keep a short TTL as a safety net.
New to the words? Read the one-minute primer
what you read far more than you change. Going from a 90% to a 99% cuts database load 10×.
Why it matters
Your database is slow and far away.
Read more · 120 words
A keeps the answers you need most in fast memory, about 50× faster. The price: each cached value is a copy, and copies go stale.
Play , then Cache miss: 1 ms against 50 ms.
Skip the cache when the page already loads fast and the database sees 100 reads/s. Skip it for write-heavy or per-user data too: the hit ratio stays under 50%, and you pay for the copies.
Tap any box to see what it is and what it talks to.
- Client
- Service
- Cache
- Database
Text description
- GET /products/42 reaches the app.
- The app asks Redis for product:42.
- Hit. The value comes back in about 1 ms.
- Response sent. The database was never touched.
Quick checkA cache has a 50% hit ratio. What happens to database reads?
Where caches live
The fastest query never reaches the database. Play A cold request to watch a miss fall through all six layers.
Read more · 290 words
| Layer | What to cache there | Watch out for |
|---|---|---|
| Browser | Static assets, GET responses with Cache-Control | You cannot delete a copy in a user's browser, only change the URL |
| CDN edge | Public pages, images, API responses identical for everyone | Never per-user data. A user in Mumbai hits an edge in ~20 ms, not Virginia in ~250 ms |
| Reverse proxy | Rendered pages, rate-limit counters | |
| In-process memory | Tiny, hot, read-only data (feature flags, config) | Private to one server, so ten servers hold ten copies that can disagree. Keep the TTL to seconds |
| Redis / Memcached | Shared per-entity data: products, sessions, counters | The default answer: one shared copy, survives app restarts. A hit costs ~1 ms against ~50 ms |
| Database buffer pool | Hot pages, automatically |
In an interview, say first; add the rest only when the problem calls for them.
Browsers and obey HTTP headers:
Cache-Control: public, max-age=60reuses a response for 60 s;privatekeeps it out of shared ;no-storedisables caching.ETag+If-None-Matchreturns304 Not Modified(no body) while the copy is valid.- Fingerprinted assets (
app.3f9a1c.js) can be cached for a year withimmutable.
Cache what is read often and changes rarely, and put everything the value depends on in the key: product:42:v3:en-IN. Keep values small: 2 MB turns a 1 ms hit into 20 ms.
Tap any box to see what it is and what it talks to.
- Client
- CDN
- Load balancer
- Service
- Cache
- Database
Text description
- Nothing in the browser cache: the request goes to the nearest CDN edge.
- CDN miss: forwarded to your reverse proxy.
- Proxy miss: the app handles it.
- Not in the in-process LRU: ask Redis.
- Redis miss: the database finally does the work. Every layer stores the answer on the way back.
Quick checkWhich cache hit saves the most distance?
Caching patterns
Six ways to fill a , and one is the default: .
Read more · 206 words
Pick by asking what happens when the cache dies, and how stale you can afford to be.
| Pattern | How it works | Use it for | Avoid it for |
|---|---|---|---|
| Cache-aside (default) | App checks the cache; on a miss it reads the database and fills the cache. If the cache dies you only get slower | Almost everything read-heavy | Nothing, but each cold key costs one slow request |
| Read-through | Cache-aside with the loading moved into the cache: the app only calls get(key) | Loading code in one place, managed caches | Nothing: same behaviour, different owner of the code |
| Write-through | Writes the cache and the database together, then confirms. Reads are always fresh | Balances, profile settings | Write-heavy data: every write pays for two stores |
| Write-behind | Writes the cache now and the database later, in batches. Very fast, but a crash before the flush loses data | View counts, metrics, likes | Payments and orders, ever |
| Write-around | Writes go straight to the database and skip the cache | Data written once and rarely read: logs, uploads | Hot data |
| Refresh-ahead | Reloads hot keys shortly before they expire, so users never pay for a miss | The hottest keys | Keys nobody asks for again: it costs extra reads |
All six, with code and a diagram each: caching techniques.
Quick checkOn a miss, the app loads the data and fills the cache. Which pattern?
Eviction, TTL and sizing
Memory is full; something has to go.
Read more · 297 words
drops the least recently used key; LFU, the least often used. Try the simulator.
A bounds staleness. Set it by the cost of a stale read: seconds for prices, hours for config. Add so keys written together don't expire together.
A hit costs the lookup. A miss costs the lookup plus the database. Say hits take 1 ms, misses 51 ms, and 4,000 reads/s arrive:
| Hit ratio | Average read | Database reads/s |
|---|---|---|
| 50% | 26 ms | 2,000 |
| 90% | 6 ms | 400 |
| 99% | 1.5 ms | 40 |
Size for the working set: often 20% of keys serve 80% of reads. A falling with rising means the is too small.
Quick checkThe cache is full. What does LRU evict?
Invalidation and failure modes
The safe default is delete on write: update the database, delete the key, and let the next read reload it.
Read more · 301 words
Updating the cache can leave the older value last when two writers race.
One race remains. Play : a slow reader loads the old row, the writer updates and deletes, then the reader puts the old value back. Fixes:
- A short as a safety net.
- A second delete about 1 s later.
- Versioned keys (
user:42:v7), so old entries are never read.
Four classic failures, all landing on your database. Play in the first diagram: a expires and 5,000 requests hit the database at once.
| Failure | What happens | Fix |
|---|---|---|
| Stampede | A hot key expires; every request misses at once and rebuilds the same value | A per-key lock so one request rebuilds and the rest wait (single flight); serve stale while refreshing (stale-while-revalidate) |
| Penetration | Lookups for ids that don't exist (a scraper trying random ids) always miss and reach the database | Cache "not found" for 30 to 60 s; a Bloom filter for a very large id space |
| Avalanche | Many keys expire together | Jittered TTLs |
| Hot key | One key gets 100K+ reads/s, more than one cache node can serve | A tiny in-process copy on every app server for a few seconds, or copies key#1…#N on different nodes read at random |
Tap any box to see what it is and what it talks to.
- Client
- Service
- Cache
- Database
Text description
- Reader A misses the cache.
- Reader A loads the OLD row from Postgres… and then stalls (GC pause, slow network).
- Meanwhile writer B updates the row in Postgres.
- Writer B deletes the cache key, as it should.
- Reader A wakes up and SETs the OLD value. The cache is now stale until the TTL expires.
- Fixes: a short TTL as a safety net, a delayed second delete, or versioned keys.
Quick checkA hot key expires and thousands of requests hit the database. What is it?
Code
get-product.js bounds retries while another request holds the lock, releases the lock safely, and "not found" briefly.
const NOT_FOUND = '__nf__';
const RELEASE = "if redis.call('GET', KEYS[1]) == ARGV[1] then return redis.call('DEL', KEYS[1]) else return 0 end";
const sleep = ms => new Promise(r => setTimeout(r, ms));
export async function getProduct(id, attempt = 0) {
const key = `p:${id}`;
const hit = await redis.get(key);
if (hit === NOT_FOUND) return null; // negative cache stops "penetration"
if (hit) return JSON.parse(hit);
const lockKey = `lock:${key}`, token = crypto.randomUUID();
if (!(await redis.set(lockKey, token, 'PX', 5_000, 'NX'))) {
if (attempt >= 20) return db.product(id); // waited ~1 s: stop waiting, protect latency
await sleep(50);
return getProduct(id, attempt + 1); // someone else is rebuilding
}
try {
const row = await db.product(id);
const ttl = 300 + Math.floor(Math.random() * 60); // jitter avoids synchronized expiry (avalanche)
await redis.set(key, row ? JSON.stringify(row) : NOT_FOUND, 'EX', row ? ttl : 30);
return row;
} finally {
await redis.eval(RELEASE, 1, lockKey, token); // only delete the lock if it's still ours
}
}Caching in the interview
Bring up the moment the database becomes the bottleneck, then walk through five decisions in order.
Read more · 420 words
The order shows you have done this before.
Name the bottleneck
Prove it with numbers: "100K reads per second against a database that handles about 5K."
Decide what to cache
Read often, changes rarely. Say what you will not cache: per-user, write-heavy or must-be-exact data.
Pick the layer and the pattern
with . Add a for media and an in-process cache for only if the problem needs them.
Set eviction and TTL
, and a sized by the cost of a stale read, with .
Own the downsides
Staleness and invalidation first, then and hot keys.
| If the interviewer says | You raise |
|---|---|
| "Reads are 100x writes" | Cache-aside with Redis |
| "Users are worldwide" | A CDN for media, regional caches for data |
| "The same page is requested constantly" | HTTP caching, then a CDN |
| "What if the cache goes down?" | Cache-aside degrades to slower, not broken; protect the database from the herd |
| "Updating a profile must show instantly" | Delete on write, or write-through |
Quick checkWhich data do you leave out of the cache?
Interview questions
How do you prevent a cache stampede?
A per-key lock, so one request rebuilds while the rest wait. Add jittered .
Do you delete or update the cache on write?
Delete. Updating can leave the older value last.
How do you choose a TTL?
By the cost of a stale read: seconds for prices, hours for config.
Would you cache a write-heavy key?
Rarely: the collapses. the aggregate instead (the like count, not each like).
A single key gets 200K reads/s. What do you do?
An in-process cache with a short TTL on every app server, or N copies read at random.
Check yourself
Question 1 of 3Score 2 out of 3 or better to mark this page complete.
Patterns behind this
The coding patterns this design is built on. Practise them and the design questions get easier.
- In-place Linked List ManipulationThe textbook LRU is a hash map plus a doubly linked list rewired on every hit; this page gets the same O(1) move-to-front from Map's insertion order.
- Design Data Structures: LRU, LFU and FriendsCaches must evict when full. LRU drops the least recently used key and LFU the least often used, and both are built from the pairs of structures in this pattern.