Caching Techniques

Why the technique matters

A is a copy, and copies drift.

The read and write paths

Before walking to the archive room, glance at the notepad on your desk.

Hit ratio	Average read	Database sees
50%	about 25 ms	5,000 a second
90%	about 6 ms	1,000 a second
99%	about 1.5 ms	100 a second

Six patterns compared

Each pattern answers one question: who keeps the copy fresh?

Pattern	How it works	If the cache dies	Use it when	It costs
Cache-aside (default)	App updates the DB, deletes the key; app reads cache, loads on a miss	Slower, still correct	Reads outnumber writes about 10 to 1 or more: almost every product, profile and catalog read	One slow request per cold key; a stale window as long as the TTL when a delete is lost
Read-through	Cache-aside where the cache loads misses itself: the app calls `get(key)` and a good loader runs once per key, even when 100 callers ask at once	Slower, still correct	You want the loading code in one place. Plain Redis has no loader; libraries (Caffeine, Guava loading caches) and products such as Amazon DAX in front of DynamoDB do	A cache that must know how to reach your database; a library tied to your language
Write-through	Writes the database first, then the cache, then confirms (a rejected write never reaches the cache). A read after a write is always a hit on the new value: about 51 ms instead of 50 ms	Still correct	A user must see their own change at once (a profile bio, a setting) and writes are modest	Cache space for rows nobody reads again; skip it for write-heavy data
Write-behind	Writes the cache now, the database later, in batches. Repeated writes to one key collapse: 1,000 view-count updates a second flushed every 5 s become one `UPDATE` per 5 s	Unflushed writes are lost	Losing a few seconds is fine and the same keys are hit over and over: view counts, likes, metrics. Never orders or money	A persisted buffer (an append-only log) and a short flush interval narrow the loss window and never close it
Write-around	Writes go straight to the database; reads fill the cache. Write-once data (logs, audit rows, uploads) never pushes hot keys out	Still correct	The first read is far from the write, or may never come	One slow first read, and a trap: if the key was already cached, the write leaves a stale copy. Delete the key on write whenever it can exist
Refresh-ahead	Reloads a hot key in the background once most of its TTL is gone: with a 300 s TTL, the first read after 240 s (80%) starts the reload while the old value keeps serving	Slower, still correct	A few keys are expensive to rebuild and read constantly: an 800 ms homepage query at 5,000 reads a second would otherwise stall about 4,000 requests at each expiry	Extra reloads for keys nobody asks for again: keep it to the top handful, and let only one instance reload (a `SET ... NX` flag)

Races the delete does not fix

Deleting on write is the right default, but a reader that was already in flight can undo it.

Fix	How it works	Use it when	It costs
TTL + second delete	A TTL bounds the damage; a delayed second delete about 1 s after the write shrinks it	Stale is embarrassing but not wrong (a price in a list). Always set the TTL	The delay must be longer than your slowest load, and a longer pause still beats it. Needs a delayed job (a queue with a delay). It narrows the race and does not close it
Versioned keys	Keep a counter (`ver:user:42`), bump it with `INCR` on each write, read `user:42:v7` after reading the counter. A slow reader holding v7 can only write to the dead v7 key	A stale value is a bug: permissions, prices at checkout, feature flags	A second Redis round trip per read (about 1 ms more) and orphaned old keys, which the TTL removes
Fill only if newer	The cache refuses a value older than what it has seen. The writer leaves a tombstone at the new version, so a fill carrying v1 is rejected and one carrying v2 is accepted. In Redis, run the compare and the set as one Lua script so they cannot interleave	You need the exact fix	A script and a version on every row

Stampedes, hot keys and other failures

Every failure here ends on your database.

Read more · 1311 words

The was not only faster; it was a shield.

. A expires while thousands of requests want it, and every one rebuilds the same value. If the rebuild takes 800 ms and 5,000 requests arrive each second, 5,000 x 0.8 = 4,000 requests all run the same heavy query. Play the three scenarios in order.

(inside one process): the first miss starts the rebuild and every concurrent caller waits on the same promise. A few lines (the Code section has it), no cost, and it fixes the worst case inside one instance. It does not help across instances: 50 instances still send 50 rebuilds.
A lock (across instances): SET lock:home token NX PX 5000 succeeds for one caller; the rest retry the cache after about 50 ms. The timeout frees the lock after 5 s if the holder crashes, and releasing only if the token still matches stops a slow holder from deleting someone else's lock. Use it when a rebuild is expensive (hundreds of ms or a heavy query) and the key is hot. For cheap rebuilds, single flight is enough.
Serve stale while you rebuild: keep the old value for a grace period and hand it out while one request refreshes it, so nobody waits.
the : 1,000 keys loaded together with a 300 s TTL all expire in the same second (an avalanche), while a random 270 to 330 s spreads the same expiries over about 60 s, roughly 17 a second.

Hot key. A two-level cache puts a tiny map inside each app instance (L1) in front of Redis (L2). A hit in L1 costs microseconds and no network. Play L1 hit, L2 hit and Both miss.

With a short L1 TTL, each instance asks Redis for a key once per TTL, however many requests it serves. A key read 100,000 times a second across 50 instances, with a 5 s L1 TTL, costs Redis 50 / 5 = 10 reads a second instead of 100,000. That cures a hot key that saturates the one Redis node owning it.

Use it when a few keys are read thousands of times a second and a stale copy of up to 5 s is fine: , config, a celebrity profile. Do not use it for balances, stock counts or permissions.
Keep the copies honest. The writer deletes the Redis key and publishes the key name; each instance drops its L1 entry. is fire and forget: an instance that is reconnecting misses the message and keeps the old copy. So the L1 TTL is your staleness bound: choose it as the longest wrong answer you accept. Redis 6 and later can also push invalidations to clients (client-side caching with tracking).
Cap the size of L1 (an of a few thousand entries): 50 copies of a large map are 50 times the memory.

Penetration. A scraper tries random user ids at 10,000 a second; every request misses Redis and finds nothing in the database.

Cache the not-found for 30 to 60 s. Cheap and right for ordinary mistakes (a deleted product, a mistyped link). It fails against a scraper that never repeats an id: at 10,000 unique ids a second and a 60 s TTL you would store up to 600,000 empty keys a minute, pushing real data out of memory.
A of real ids. A bit array in app memory that answers "definitely not present" or "maybe present". A "definitely not" is a 404 with no Redis and no database. It never says "no" wrongly, and sometimes says "maybe" wrongly: that false-positive rate is the one number to choose. Size it from the item count n and the rate p: bits = -n x ln(p) / (ln 2) squared, and hashes = bits / n x ln 2.

Target rate	Bits per id	Hashes	100K ids need
1%	9.6	7	about 120 KB
0.1%	14.4	10	about 180 KB

A 120 KB filter in every app instance can stop nearly all of a random-id attack. The filter must learn a new id before anyone can read it, or the real user gets a wrong 404: add the id to the filter before (or in the same step as) inserting the row. You cannot delete from a plain filter (clearing a bit may erase another id), and it does not grow, so rebuild it from the table on a schedule, sized for the final count.

Cold cache after a restart. stays correct when Redis dies, but the database then takes the full read load. At 10,000 reads a second and a 90% it sees 1,000. After a restart the hit ratio is 0%, so it sees 10,000: ten times its normal load, at the moment it is slowest because every load also refills the cache. Four defences, in order of cost:

Persistence (an RDB snapshot or AOF), so Redis restarts with data.
A warm-up script that loads the top 1,000 keys before traffic returns.
A cap on concurrent database loads with a bounded queue.
A to fail over to, so a node loss is not a cold start.

A sick Redis is worse than a dead one, because every request waits on it. Set a client timeout of a few tens of ms, then skip the cache and read the database under the same concurrency cap. A stops the app from trying a failing cache on every request. Fail open for reads; for writes you must not lose, never depend on the cache to accept them.

Tap any box to see what it is and what it talks to.

Client
Service
Cache
Database

Open in playground

Text description

Noon: the key expires. Requests keep arriving at 5,000 a second.
Every one of them misses.
The rebuild takes 800 ms, so 5,000 x 0.8 = 4,000 requests all run the same heavy query.

Tap any box to see what it is and what it talks to.

Client
Service
Cache
Database

Open in playground

Text description

A request for a hot feature flag reaches instance 12.
Found in its own map. No network hop at all.

bloom-size.js

// How big must a Bloom filter be? For n ids and a target false-positive rate p:
//   bits m = -n * ln(p) / (ln 2)^2      hashes k = (m / n) * ln 2
// and the rate you actually get from m bits, k hashes and n ids is (1 - e^(-k*n/m))^k.
export function bloomSize(n, p) {
  const m = Math.ceil((-n * Math.log(p)) / Math.LN2 ** 2);
  const k = Math.max(1, Math.round((m / n) * Math.LN2));
  return { bits: m, hashes: k, bytes: Math.ceil(m / 8) };
}

export function falsePositiveRate(bits, hashes, n) {
  return (1 - Math.exp((-hashes * n) / bits)) ** hashes;
}

Quick check50 instances use single flight; a hot key expires. How many rebuilds can run?

Code

Cache-aside (lazy loading)

cache-aside.js

// Check the cache, load the database on a miss, then store the result with a TTL
export async function getItem(id) {
  const key = 'item:' + id;
  const hit = await redis.get(key);
  if (hit) return JSON.parse(hit);
  const row = await db.find(id);
  await redis.set(key, JSON.stringify(row), 'EX', 300);
  return row;
}

Write-through

write-through.js

// Every write updates the database and the cache together, so the next read is fresh
export async function save(id, data) {
  await db.save(id, data);                                           // the source of truth first
  await redis.set('item:' + id, JSON.stringify(data), 'EX', 3600);   // then the copy
}

Write-behind (write-back)

write-behind.js

// Batch writes in memory and flush every 5 s. If a flush fails, put the batch back without
// overwriting newer values, and never run two flushes at once.
const buffer = new Map();
let flushing = false;
export const write = (id, value) => buffer.set(id, value);       // the latest value per id wins

setInterval(async () => {
  if (flushing || buffer.size === 0) return;
  flushing = true;
  const batch = [...buffer];
  buffer.clear();
  try {
    await db.bulkUpsert(batch);
  } catch (err) {
    for (const [id, value] of batch) if (!buffer.has(id)) buffer.set(id, value);   // retry next tick
    log.warn({ err, size: batch.length }, 'write-behind flush failed');
  } finally {
    flushing = false;
  }
}, 5_000);
// A process crash still loses whatever is in `buffer`. That's the trade-off you accept with write-behind.

Invalidation

invalidation.js

// Option 1: delete on write. Every key also has a TTL, the safety net for races you didn't think of.
export async function updateItem(id, data) {
  await db.update(id, data);
  await redis.del('item:' + id);                  // delete, do not update
}

// Option 2: versioned keys. Bump user.version on every write and old entries are simply never read again.
export const userKey = (user) => 'user:' + user.id + ':v' + user.version;   // versioned key

Stampede protection (single flight)

One request rebuilds an expired ; the rest share the result.

Two-level cache (L1 + L2)

Each instance keeps its own copy, so invalidate over and keep L1 short.

two-level-cache.js

// Two-level cache: an in-process LRU (L1, microseconds) in front of Redis (L2, ~1 ms).
// Invalidation fans out over pub/sub. Pub/sub can drop messages, so L1 entries also expire quickly.
const l1 = new Map();                                    // key -> { value, exp }
const L1_TTL_MS = 5_000;
const sub = redis.duplicate();
await sub.subscribe('invalidate');
sub.on('message', (_channel, key) => l1.delete(key));    // every instance drops its copy

export async function get(key, load) {
  const e = l1.get(key);
  if (e && e.exp > Date.now()) return e.value;
  const cached = await redis.get(key);
  const value = cached ? JSON.parse(cached) : await load(key);
  if (!cached) await redis.set(key, JSON.stringify(value), 'EX', 300);
  l1.set(key, { value, exp: Date.now() + L1_TTL_MS });   // cap l1's size in production (e.g. the LRU from Learn 4)
  return value;
}
export async function invalidate(key) {
  await redis.del(key);
  await redis.publish('invalidate', key);
}

Refresh-ahead

Past 80% of the TTL, a read triggers a background refresh.

refresh-ahead.js

// Refresh-ahead: once 80% of the TTL has passed, a read triggers a background refresh,
// so hot keys never actually expire for readers
export async function getWithRefreshAhead(key, load, ttlSec = 300) {
  const [[, value], [, ttl]] = await redis.multi().get(key).ttl(key).exec();
  if (value && ttl > 0 && ttl < ttlSec * 0.2) {
    redis.set(`refresh:${key}`, 1, 'EX', 30, 'NX')                    // only one instance refreshes
      .then(won => won && load(key).then(v => redis.set(key, JSON.stringify(v), 'EX', ttlSec)))
      .catch(() => {});                                                  // background work must never crash the read
  }
  if (value) return JSON.parse(value);
  const fresh = await load(key);
  await redis.set(key, JSON.stringify(fresh), 'EX', ttlSec);
  return fresh;
}

Bloom filter against penetration

Ids that don't exist always hit the database.

Eviction policy

Caching techniques in the interview

Bring these techniques up after you have said " with ", when the interviewer pushes on what could go wrong.

If the interviewer says	You raise
"Users must see their own edit"	Write-through, or delete on write and read your own write
"50,000 counter updates a second"	Write-behind, flushed every few seconds, loss accepted
"One key gets 100K reads a second"	L1 per instance with a 5 s TTL and pub/sub invalidation
"Lookups for ids that don't exist"	Cache not-found, then a Bloom filter of real ids
"Redis restarts"	Warm the top keys, cap database loads, persist or fail over

Interview questions

Cache-aside or write-through?

by default. when users must read their own writes.

How do you keep in-process caches consistent across 50 instances?

invalidations plus short L1 , since a missed message leaves a stale copy until the TTL.

When is write-behind acceptable?

When losing a few seconds is fine (view counters), never for money or orders.

What's the difference between cache-aside and read-through?

Who runs the miss path: your code in cache-aside, the 's own loader in read-through.

Requests for non-existent ids are hammering your database. What do you do?

Cache "not found" briefly, add a of real ids, and rate-limit callers.

You delete on write, yet a stale price appears. How?

A slow reader loaded the old row before the write and refilled it after the delete. Fix with a TTL, a delayed second delete, or versioned keys.

How does a Bloom filter stay correct as new ids are created?

Add the id to the filter before or with the row insert, since a late add gives the real user a 404. Rebuild it on a schedule because it cannot delete.

Redis restarts empty. What protects the database?

Persistence or a , a warm-up of the hottest keys, and a cap on concurrent loads.

Check yourself

Question 1 of 5

Score 2 out of 3 or better to mark this page complete.

1. Which pattern can lose acknowledged writes?Write-throughWrite-behindCache-aside

Trade-offs

Bring it up before they ask

Why the technique matters

The read and write paths

Six patterns compared

Caching patterns side by side

Races the delete does not fix

Stampedes, hot keys and other failures

Code

Cache-aside (lazy loading)

Write-through

Write-behind (write-back)

Invalidation

Stampede protection (single flight)

Two-level cache (L1 + L2)

Refresh-ahead

Bloom filter against penetration

Eviction policy

Caching techniques in the interview

Start with cache-aside and delete on write

Change the pattern only for a reason

Name the race

Protect the database

Say what happens when the cache dies

Interview questions

Check yourself

Patterns behind this

Deep dives