Design a Unique ID Generator
Make unique, roughly time-ordered 64-bit ids on every server with no network call, and survive clock skew and shared worker ids.
- 69.7 yearsof time in 41 bits
- 4,096 / msids per worker from 12 bits
- 1,024workers from 10 bits
Show what to say
“I'd use Snowflake-style 64-bit ids: 41 bits of milliseconds since a custom epoch, 10 bits of worker id and 12 bits of sequence. Each server makes ids locally, 4,096 per millisecond, so ids are unique and roughly time-ordered with no network call. Worker ids are leased with a TTL, and the generator refuses to issue ids if the clock steps back.”
Trade-offs
- Snowflake vs UUID v4: 64 bits, local and time-ordered, but needs worker ids and sane clocks; UUID needs neither and is 128 random bits.
- Ticket server vs Snowflake: strictly ordered and simple, but a shared database and a round trip, or a block with gaps.
- Wait vs refuse on clock skew: small backward steps are waited out, but a large step must fail loudly to avoid duplicates.
Bring it up before they ask
Clock going backwards: refuse to issue ids on a big step and fail over, because a duplicate primary key is worse than a short error.
Pack time, a worker id and a counter into 64 bits (a ) so every server makes unique, roughly time-ordered ids locally, with no network call. The risks are a clock that goes backwards and two servers sharing a worker id.
Functional
- Generate globally unique ids across servers and data centers
- Ids are 64-bit numbers
- Ids sort roughly by creation time
- No central lookup on the hot path
Non-functional
- 100K ids a second at peak across the fleet
- Under 1 ms to make an id
- No duplicates, ever, even during failures
- Keeps working if the id registry is briefly down
- Lasts decades
Out of scope
- Gapless sequences (invoice numbers with legal rules)
- Hiding how many ids were issued
- Ids that are hard to guess (use random tokens for that)
Clarifying questions, scale and interface
Read more · 250 words
- How big and how sorted? Must the id fit 64 bits? Must it sort by time, or just be unique?
- How many ids, from how many servers? This sets the bit split.
- How strict is "unique"? One duplicate in a billion is too many for .
Capacity estimate
| Quantity | Calculation | Result |
|---|---|---|
| Demand | 100K ids/s ÷ 20 servers | 5K/s per server |
| One worker's limit | 4,096 per ms × 1,000 ms | 4.096M/s, about 800x headroom |
| Fleet limit | 1,024 workers × 4.096M | 4.19 billion/s |
| Lifetime | 2^41 ms ÷ (1000 × 3600 × 24 × 365.25) | 69.7 years from the epoch |
| Rows | 10B × 8 B (64-bit) | 80 GB of keys |
| Rows as UUIDs | 10B × 16 B | 160 GB, and bigger indexes |
Speed is not the problem. Clock correctness and who owns which worker id are.
API
Usually a library call inside each service: nextId(). A shared service is optional.
| Method | Path | Notes |
|---|---|---|
| POST | /v1/ids?count=100 | Returns ids as strings, in increasing order |
| GET | /v1/ids/:id | Decodes time, worker and sequence (debugging) |
JavaScript numbers are exact only to 2^53 - 1, but these ids reach 2^63: send them as strings.
Data model
Nothing about ids is stored. Only leases are.
| Where | Key | Value |
|---|---|---|
| etcd | /workers/17 | owner and a 30 second TTL |
| Orders table | id bigint PRIMARY KEY | the generated id |
-- the ticket-server alternative: one row per counter
CREATE TABLE tickets (id smallint PRIMARY KEY, next bigint NOT NULL);
UPDATE tickets SET next = next + 1000 WHERE id = 1 RETURNING next - 1000 AS first;Architecture
Each server makes its own ids.
Read more · 33 words
The only shared piece is the lease registry, used at boot and every ten seconds after. Play Boot and first id, 4,097 ids in one millisecond, Clock moves backwards and Lease lost.
Tap any box to see what it is and what it talks to.
- Service
- ID generator
- External API
- Database
Text description
- A server starts. Its generator asks the registry for a worker id lease.
- It gets worker 17 for 30 seconds, and will renew every 10. No other live server can hold 17.
- The app needs an id for a new order.
- The generator reads the clock: 4,512,000 ms since our epoch.
- It returns time (41 bits), worker 17 (10 bits) and sequence 0 (12 bits), packed into one 64-bit number.
- The app inserts the row. No call left the machine to make that id.
The Snowflake layout
Split 64 bits into fields. The top field is time, so ids sort by time.
Read more · 33 words
The worker field stops two servers colliding. The sequence field separates ids made in the same millisecond.
Use a custom epoch (2026 here), not 1970, so the 41 bits last until about 2095.
Clocks, workers and failure
Uniqueness depends on two promises. Time never repeats for one worker: if the clock steps back, wait out a few milliseconds, but refuse and alert on a big step.
Read more · 72 words
No two live servers share a worker id: lease ids with a , and stop issuing a few seconds before the lease ends.
Ids are only roughly ordered across servers, because clocks differ by milliseconds. Do not use them to decide which event happened first.
Choosing a scheme
Pick by constraints, not by habit. Click through the tree.
Do ids need to sort by creation time?
How to approach it
Clarify
64 bits? Sorted? How many ids, how strict?
Estimate
5K ids a second per server against 4.1M of capacity.
Compare schemes
Auto-increment, UUID, , .
Design the bits
41 time, 10 worker, 12 sequence.
Failure
Clock skew, worker leases, sequence overflow.
Read more · 211 words
Step-by-step answer
Rule out the easy answers
A database auto-increment is one machine and a round trip per id. A random UUID is 128 bits, unordered, and scatters inserts across the index.
Spend the bits
1 sign, 41 time, 10 worker, 12 sequence. Say what each buys: order, no collisions, .
Generate locally
Read the clock, bump the sequence in the same millisecond, wait for the next one if it overflows. No network call.
Assign worker ids safely
Lease ids from etcd or ZooKeeper with a . Renew often, and stop early when a renewal fails.
Defend against the clock
Wait out tiny backward steps. Refuse large ones, alert, and fail over. Keep NTP healthy with slewing rather than stepping.
Mention the alternatives
Ticket servers with blocks and odd and even machines; UUID v7 when 128 bits is fine and you want no coordination.
Show the gotchas
Ids as strings in JSON; gaps after crashes are normal; ids are roughly, not strictly, ordered across servers.
Implementation
Tabs: the generator, a with block allocation, UUID v7, and the worker lease registry.
Read more · 56 words
- The Snowflake tests check the layout numbers (69.7 years, 4,096 per ms, 1,024 workers), the sequence overflow wait, and the clock-back cases.
- The ticket test shows 1 million ids cost 1,000 round trips, and a crash leaves gaps, never duplicates.
- The registry test shows an id reused only after its lease expires.
// Snowflake-style 64-bit ids, generated locally with no coordination:
//
// | 1 bit | 41 bits: ms since our epoch | 10 bits: worker | 12 bits: sequence |
// sign time (about 69.7 years) 1,024 workers 4,096 ids per ms
//
// Ids sort by creation time (to the millisecond), fit in a signed 64-bit integer, and two workers can never
// collide because the worker bits differ. The clock is injected so tests can move it.
export const EPOCH = 1_767_225_600_000n; // 2026-01-01T00:00:00Z, our "time zero"
const WORKER_BITS = 10n;
const SEQ_BITS = 12n;
const MAX_SEQ = (1n << SEQ_BITS) - 1n; // 4095
const MAX_WORKER = (1n << WORKER_BITS) - 1n; // 1023
export class Snowflake {
#last = -1n;
#seq = 0n;
// maxBackwardMs: a clock that steps back this far or less is waited out; more than that is an error.
constructor(workerId, { now = () => BigInt(Date.now()), maxBackwardMs = 5n } = {}) {
const w = BigInt(workerId);
if (w < 0n || w > MAX_WORKER) throw new RangeError(`workerId must be 0..${MAX_WORKER}`);
this.worker = w;
this.now = now;
this.maxBackwardMs = maxBackwardMs;
}
nextId() {
let ts = this.now() - EPOCH;
if (ts < this.#last) {
const behind = this.#last - ts;
if (behind > this.maxBackwardMs) throw new Error(`clock moved back ${behind} ms; refusing to issue ids`);
while (ts < this.#last) ts = this.now() - EPOCH; // a tiny step back (NTP slew): wait it out
}
if (ts === this.#last) {
this.#seq = (this.#seq + 1n) & MAX_SEQ;
if (this.#seq === 0n) { // 4,096 ids used up in this millisecond
while (ts <= this.#last) ts = this.now() - EPOCH; // wait for the next one
}
} else {
this.#seq = 0n;
}
this.#last = ts;
return (ts << (WORKER_BITS + SEQ_BITS)) | (this.worker << SEQ_BITS) | this.#seq;
}
}
export function decode(id) {
const n = BigInt(id);
return {
timeMs: Number((n >> (WORKER_BITS + SEQ_BITS)) + EPOCH),
worker: Number((n >> SEQ_BITS) & MAX_WORKER),
seq: Number(n & MAX_SEQ),
};
}
// Handy numbers for the page.
export const limits = {
yearsOfTime: Number(1n << 41n) / (1000 * 3600 * 24 * 365.25),
idsPerMsPerWorker: Number(MAX_SEQ + 1n),
workers: Number(MAX_WORKER + 1n),
};Failure modes and trade-offs
Read more · 155 words
| What breaks | Effect | Mitigation |
|---|---|---|
| Clock steps back | Duplicate ids | Wait out small steps; refuse big ones; alert |
| Two servers share a worker id | Duplicate ids | Leases with TTL; stop before expiry |
| Registry (etcd) is down | New servers cannot start | Running servers keep their lease margin; cache the id on disk |
| Sequence overflow | Would repeat an id | Wait for the next millisecond |
| Server crashes | Unused ids lost | Gaps are fine |
| Epoch runs out | Ids overflow in decades | Choose the epoch; plan a format change |
Trade-offs
| Choice | Gain | Cost |
|---|---|---|
| Snowflake | 64 bits, local, ordered | Needs worker ids and good clocks |
| UUID v4 | No coordination | 128 bits, random inserts |
| UUID v7 | Sortable, no coordination | 128 bits |
| Ticket server | Simple, strictly ordered | Single point of contention, round trips |
Follow-up questions
What if the sequence overflows?
Wait for the next millisecond. 4,096 per ms per worker is far above real demand; if not, add worker bits.
Are the ids strictly ordered?
No. Within one worker yes; across workers only to clock accuracy. Use a log or database sequence if you need strict order.
How do you handle data centers?
Split the 10 worker bits, for example 5 for the data center and 5 for the machine, or just lease from one global registry.
Check yourself
Question 1 of 3Score 2 out of 3 or better to mark this page complete.
Patterns behind this
The coding patterns this design is built on. Practise them and the design questions get easier.