DSA & System Design
Step 9 of 10 · 40 min

Case Study: Notification System

Shows queues, fan-out, user preferences, deduplication, and external providers that fail.

✓ Design a multi-channel, prioritised, deduplicated pipeline✓ Handle provider failover, user preferences and quiet hours
Say it in 30 seconds
  • 5.8K/saverage: 500M notifications a day
  • 50K+/scampaign burst, so throttle the fan-out
  • ~8 min1M campaign draining a shared queue at 2,000/s
Show what to say

“I accept the request with a 202, check preferences, and publish one message per channel onto queues split by channel and priority, so an OTP never waits behind a marketing campaign. Workers dedupe on eventId and channel, then call providers through circuit breakers with failover, and failures end in a dead letter queue with an alert.”

Trade-offs

  • Shared queue vs priority lanes: lanes cost more queues and workers, but an OTP leaves in milliseconds instead of waiting behind the campaign.
  • At-least-once vs exactly-once: exactly once to a phone is impossible, so dedupe on deterministic ids and aim for effectively once.
  • Mark sent early vs after acceptance: marking after the provider accepts allows a rare resend after a crash, but failed sends can still retry.

Bring it up before they ask

A provider that accepts but silently doesn't deliver is worse than one that errors, so track delivery rate per provider.

0 of 0 sections ticked
30-second answer

Accept the request with 202, then put one message per channel on queues split by channel and priority, so a login code never waits behind a marketing blast. each message, and send through providers with .

Why it matters

This system is write-heavy and asynchronous.

Read more · 66 words

One slow channel must not hurt the rest, there must be no duplicates, and you do not control the providers.

Skip all this for a small app: 100K users at 5 messages a day is about 6 sends a second, so one worker and a retry loop do. Play all four scenarios.

Functional

  • Send push, email and SMS
  • Per-user preferences by channel and type
  • Templates with localisation
  • Priorities: an OTP never waits behind marketing
  • Scheduled sends and quiet hours
  • Per-user rate limits and daily caps
  • Delivery tracking and analytics
  • No duplicates

Non-functional

  • OTPs delivered within seconds
  • At-least-once delivery; users see each message once
  • A provider outage doesn't stop delivery
  • Absorb 50K+/s campaign bursts without slowing critical traffic
  • No silent loss: failures end in a DLQ, with an alert

Out of scope

  • Running our own push, email or SMS delivery
  • The campaign editor UI
  • A rich in-app inbox

Ask early which types exist and how urgent each is. That drives priority isolation.

Press Play to watch it run · 7 steps

The notification service reads preferences and publishes one message per channel onto queues split by channel and priority. Workers dedupe, then send through providers behind circuit breakers, with a backup SMS provider and a delayed queue for quiet hours.

Notification pipelineThe notification service reads preferences and publishes one message per channel onto queues split by channel and priority. Workers dedupe, then send through providers behind circuit breakers, with a backup SMS provider and a delayed queue for quiet hours.eventprefspublishquiet hourswhen duefailoversent?Other services emit events; admins and campaigns call POST /v1/notifications.Event or APIMessages that must wait (quiet hours, sendAt). A scheduler moves them on when they're due.Delayed queueuntil notBeforeNotif serviceprefs · priorityPreferences DBcachednotif.sms.critical, notif.push.high, notif.email.low…Channel queueschannel × priorityPush workerEmail workerSMS workerDedupe storeRedis sent:idPush (FCM/APNs)Email (SES)SMS provider AprimaryBackup SMSprovider B

Tap any box to see what it is and what it talks to.

  • Client
  • Queue
  • Service
  • Database
  • Worker
  • Cache
  • External API
Open in playground
Text description
  1. The order service emits order_update for user 42 (event 812), with an idempotency key.
  2. The notification service loads the user's preferences, which are cached in front of the database.
  3. Push, email and SMS are all enabled for order updates. The type maps to priority high.
  4. It publishes one message per channel, with deterministic ids 812:push, 812:email and 812:sms, on the high-priority queues.
  5. Three workers pick up their messages in parallel. Each channel scales and fails on its own, so a slow SMS provider can't delay push.
  6. Each worker renders the template in the user's language and calls its provider.
  7. All three providers accept. Workers record sent:<id> and a delivery row; webhooks report delivered and opened later.

Estimate, API and data model

50M users get 10 messages a day: 500M a day, about 5.8K a second on average.

Read more · 303 words

Campaigns burst to 50K+ a second, far above what providers accept. Queues absorb the burst, and you slow the campaign down on purpose.

A worker with 100 provider calls in flight at 100 ms each sends about 1,000 a second. The average needs about 6 workers and a 50K burst would need 50. You do not buy 50 workers for a burst; you throttle it. The log is 500M rows at about 200 bytes, 100 GB a day, so deliveries is partitioned by day.

MethodPathResult
POST/v1/notifications202 { notificationId }
PUT/v1/users/:id/notification-preferencesChannels, types, quiet hours, time zone
POST/webhooks/:providerDelivery callbacks (delivered, bounced)

202 Accepted means "queued", not "sent". A retry with the same idempotencyKey returns the same notificationId.

Preferences are read for every notification: them. deliveries is the busiest table: partitioned by day, looked up by provider_msg_id when arrive.

Device tokens rot: phones get wiped, apps get uninstalled.

  • Refresh the token on every app start.
  • Invalid or unregistered? Delete it now.
  • Three devices, three pushes: render once, send to each token.

Delivery tracking. Providers report back by webhook (delivered, bounced, opened). Update the delivery row by provider_msg_id.

  • Verify the signature over the raw body first.
  • Webhooks retry, so updates must be .
  • They arrive out of order: never let a late delivered overwrite opened.
Average
5.8K/s
500M a day
Campaign burst
50K+/s
throttle the fan-out
Delivery log
100 GB/day
partition by day
CREATE TABLE notification_prefs (user_id bigint, channel text, type text, enabled boolean NOT NULL,
                                 PRIMARY KEY (user_id, channel, type));
CREATE TABLE device_tokens (token text PRIMARY KEY, user_id bigint NOT NULL, platform text, last_seen timestamptz);
CREATE TABLE deliveries (id text PRIMARY KEY,                       -- `${eventId}:${channel}`: the dedupe key
                         user_id bigint, channel text, provider text, provider_msg_id text,
                         status text, updated_at timestamptz);
CREATE INDEX deliveries_provider_msg ON deliveries (provider_msg_id);  -- webhook lookups

Queues and priorities

One event becomes one message per channel, and the queue layout decides what can hurt what.

Read more · 314 words
LayoutWhat happensUse it when
One queue for everythingOne slow provider or one big campaign delays every message. Workers stuck on 2 s SMS calls leave push waitingA prototype under about 10 messages a second, all equally urgent
One queue per channelA slow provider cannot block the other channels. But an OTP and a promo are both SMS, so the OTP still waits behind the blastAll messages in a channel share one urgency
Per channel and priority (default)Separate queues and separate workers: a campaign can only fill its own laneAny system that sends both codes and marketing
Priority levels in one queueSome brokers order one queue by priority (RabbitMQ yes; Kafka and standard SQS no). Risk is starvation: low messages may never leaveLow volume, and a broker with priorities. Add dedicated workers for the low lane or age messages upward

A 1M campaign on one shared queue at 2,000 a second takes 8 minutes, so a login code expires in line. With lanes it leaves in milliseconds. Try Shared queue, then Two lanes.

QueueMessagesWorkers
notif.sms.criticalOTPs, security alertsDedicated, spare capacity
notif.push.highOrder updatesScaled on queue depth
notif.push.lowMarketing, digestsThrottled, paused first

A separate queue drained by the same busy workers is still a shared bottleneck. Reserve provider capacity: if the provider allows 5,000 SMS a second, cap the low lane at 4,000 and the critical lane keeps 1,000. Keep to two or three lanes: ten lanes are ten things to scale. The cost is idle capacity in the critical lane. A message that keeps failing lands in a , with an alert.

Interactive

Priority lanes

Login codes (OTP) must arrive in seconds; marketing can wait hours. Give them separate queues and workers, or watch one campaign bury every login.

OTP lanecritical · login codesempty2 workers idleMarketing lanelow priorityempty8 workers idle
  • OTP code (one every 0.5 s)
  • marketing (log-scale bar)
  • 1 worker = 20 messages/s
OTP sent now waits
50 ms ✓ on time
Oldest waiting OTP
none (0 delivered)
Marketing backlog
0 keeping up
Simulated time
0 ms (10× real time)

Separate lanes: login codes go out in 50 ms. Launch the campaign and watch them stay fast.

Speed and OTP wait chart
Speed

The OTP latency chart fills in as the simulation runs.

Press Play to watch it run · 5 steps

Each channel has a critical lane and a low lane, with separate workers. A campaign can only fill the low lane, and the low workers are throttled so the provider keeps spare capacity for critical traffic.

One queue per channel and priorityEach channel has a critical lane and a low lane, with separate workers. A campaign can only fill the low lane, and the low workers are throttled so the provider keeps spare capacity for critical traffic.otppromocappedNotif servicetype to prioritysms.criticalsms.lowCritical workersx2, reservedLow workersx8, throttledSMS provider

Tap any box to see what it is and what it talks to.

  • Service
  • Queue
  • Worker
  • External API
Open in playground
Text description
  1. A 1M-message promo lands in sms.low.
  2. Eight low workers chew through it for about 8 minutes.
  3. A login OTP arrives in the middle of the campaign. Type otp means critical.
  4. The critical lane is empty and its two workers are idle, so the OTP is picked up in milliseconds.
  5. The OTP is sent while the campaign is still draining.

Scheduling, users and campaigns

Templates. Copy changes weekly; deploys should not have to.

Read more · 546 words
  • Reference a template id, version and locale, never raw text.
  • Render in the worker, late, so data and language are fresh. Fall back (pt-BR, then pt, then en).
  • One variant per channel. SMS fits 160 characters (70 with an emoji).

Scheduling and quiet hours. A scheduled send and a 3 a.m. marketing push need the same fix: a notBefore time and somewhere to park the message until then.

  • Store an IANA time zone per user (Asia/Kolkata, not +05:30, which breaks with daylight saving).
  • Check quiet hours at send time, in the user's zone. Low priority waits (a Mumbai user's 23:30 promo goes out at 08:00); critical ignores quiet hours.
  • In a zone with daylight saving, 02:30 may not exist on the day the clocks jump, so re-check when the message is due rather than trusting the stored time.
  • A campaign set for 9:00 local time spreads itself out: 10M users over 24 zones is about 417K an hour, roughly 115 a second, instead of 10M at once.
Where to park itHow it worksUse it when
Redis sorted set (default)Scored by notBefore; a scheduler moves what is due onto the real queue. It claims a member by removing it and enqueues only if the removal returned 1, so two schedulers never send the same messageMillions of waiting messages nobody edits. Cost: Redis memory; a restart without persistence loses them
Database jobs tableA send time and a status; claim rows with SKIP LOCKED so pollers do not collideUser-visible schedules that must survive anything ("schedule for Friday 9:00", listable and cancellable), up to a few thousand a second
Broker delayNo scheduler to run. Standard SQS delays at most 15 minutes; RabbitMQ needs a plugin; Kafka has no per-message delayShort retry waits only. Use a sorted set or a table for anything longer

Protecting users. Every extra ping risks an uninstall.

  • Daily caps per type: say 3 marketing pushes a day.
  • Digests: 20 likes become one push.
  • One-click unsubscribe in every marketing email.
  • Suppression lists: never email addresses that bounced or flagged you as spam.

Large campaigns. A 10M-user campaign is a batch job, not 10M API calls, and it runs on its own lanes with its own workers. Play Run in pages, then Pause a typo.

  • Page the audience. Read the segment in pages of about 1,000 and enqueue each page as a batch on the low lanes. Never load 10M ids into memory or one queue write. Keep a cursor so a crashed job resumes at the last page, and deterministic ids make a repeated page harmless.
  • Throttle to the provider limit. The provider sets the speed, not your queue. At 5,000 a second, 10M messages take 2,000 s, about 33 minutes; at 1,000 a second, nearly 3 hours. Throttle the low-lane workers and leave headroom for critical traffic. A time-zone campaign spread over a day (about 115 a second) needs no throttle.
  • Pause and cancel. Workers read the campaign status before each send, so one write stops a typo within a second or two. Pause when the copy is fixable; cancel when it should never finish, and workers drop what is left as they see the status.

Press Play to watch it run · 4 steps

A campaign job reads the audience in pages, enqueues batches on the low lanes, and workers check the campaign status before each send, so a campaign can be paused or cancelled mid-way.

A campaign as a batch jobA campaign job reads the audience in pages, enqueues batches on the low lanes, and workers check the campaign status before each send, so a campaign can be paused or cancelled mid-way.pagebatchrunning?sendCampaign jobSegment storepaged readspush.lowLow workersthrottledStatus flagrunning or pausedPush provider

Tap any box to see what it is and what it talks to.

  • Service
  • Database
  • Queue
  • Worker
  • Cache
  • External API
Open in playground
Text description
  1. The job reads 1,000 user ids at a time from the segment, never all 10M into memory.
  2. It enqueues the batch on the low lane and asks for the next page.
  3. Low workers take messages at the rate the provider allows, 5,000 a second in this example.
  4. 10M messages at 5,000 a second take 2,000 s, about 33 minutes. Critical traffic uses its own lane the whole time.
Minutes to send 10M messages (log scale)
1101001Kmin1671,000 a second335,000 a second3.350,000 a second
Show data table
Minutes to send 10M messages
min
1,000 a second167
5,000 a second33
50,000 a second3.3

Providers and failure modes

Every provider will be slow or go down, so hide each one behind a send(msg) call, a , and a backup.

Read more · 472 words

Without a breaker, each OTP waits out a long timeout on a dead SMS provider, then fails. After about five failures in a row the breaker opens for 30 s and workers send through the next provider. Raise the error rate in the simulator until the breaker opens, then lower it and watch a trial request close it.

is for every channel with a second provider: SMS and email for sure, push rarely (one push service per platform). The cost is a second contract and sender id, and a periodic check that the backup still works. Track delivery rate per provider too (see the reliability lesson).

Not every error deserves a retry. Ask in this order:

QuestionYes means
Permanent (bad number, unregistered token, hard bounce)?Never retry. Mark failed, delete the token or suppress the address. Retrying a dead number wastes quota
A rate limit (429)?Slow the lane down. Honour Retry-After, lower the worker rate, requeue with backoff. Failing over only moves the overload
A backup with a closed breaker?Try the backup now, in the same attempt: a critical message loses milliseconds, not a retry cycle
None of theseRequeue with backoff and jitter; after about 5 attempts move it to the dead-letter queue with an alert

Four things go wrong in production that the happy path never shows.

FailureWhat happensFix
Accepted, not deliveredA provider returns success and still drops messages, so error rates stay green. Say provider A accepts every SMS but only 40% of receipts return where 95% is normalCount delivery receipts per provider over 5 minutes, alert below a threshold, route to the backup. Do this before adding retries: retries cannot fix a quiet drop
Retry stormWhen a provider recovers, every waiting worker retries at once and it falls over againExponential backoff with jitter on every retry; let the breaker send one trial request when half-open
Poison messageOne message that fails every time blocks the laneRetry a few times with growing delays, then move it to the dead-letter queue and page someone. A non-empty DLQ with no alert is a silent loss. Replay is safe: deterministic ids and sent markers skip anything delivered
Everyone opens the appA push to 2M users at 9:00:00; 5% tapping within a minute is 100K opens, about 1,700 a second of extra trafficSpread large sends over minutes or by time zone, and cache what the landing screen loads

Switch the retry policy in the simulator between fixed delays and full , and watch the peak after the 10 s recovery.

A circuit breaker is a fuse box that flips its switch off when too much current surges in, cutting the wire to protect what is behind it.
Interactive

Circuit breaker

Requests (5 per second) pass through a breaker. Raise the dependency's error rate to trip it open, then lower it and watch a trial request close it.

5 failuresin a rowcooldownovertrialfailstrial succeedsClosedfailures 0/5● currentOpenfail fastHalf-openone trial callRecent requests (newest →)

✓ success · ✕ real error (dependency called) · ↩ fast-fail, fallback served

✓ Successes
0
↩ Fast-failures
0
fallback served
✕ Real errors
0
dependency called
Calls spared
0%
never reached it

Closed: calls go to the dependency (0 of 5 failures in a row so far). 0 successes, 0 fast-failures served by the fallback, 0 real errors.

State changes

No state changes yet: raise the error rate until 5 requests in a row fail.

Press Play to watch it run · 3 steps

A provider can return success for a send and still not deliver it. Delivery receipts arrive by webhook, are summed per provider, and an alert fires when a provider's delivery rate drops, so traffic can move to a backup.

Accepted, but not deliveredA provider can return success for a send and still not deliver it. Delivery receipts arrive by webhook, are summed per provider, and an alert fires when a provider's delivery rate drops, so traffic can move to a backup.sendbackuphookwriterateSMS workerSMS provider ASMS provider BWebhooksDeliveriesrate per providerAlert

Tap any box to see what it is and what it talks to.

  • Worker
  • External API
  • Service
  • Database
Open in playground
Text description
  1. Provider A returns 200 for every send, so the breaker stays closed and the error dashboard stays green.
  2. But the delivered receipts that normally come back are missing for most messages. A carrier is filtering the traffic.
  3. Nobody is looking at receipts, so nothing happens. Users wait for codes that never arrive.
Interactive

Retry storm

300 clients fail at once; the server is back at 10 s and answers 20 requests per 100 ms.

Retry policy
Peak after recovery
165 / 100 ms
full jitter: 12
Everyone served by
not by 30 s
280 failing · full jitter: 17.5 s
Requests sent
1,790
6.0 per client
Requests arriving at the server per 100 ms (line: recovery at 10 s)
  • Exponential
  • Full jitter
050100150200req07.51522.429.9Time (s)ExponentialFull jitter
Show data table
Requests arriving at the server per 100 ms (line: recovery at 10 s)
Time (s)Exponential (req)Full jitter (req)
0135143
0.1165184
0.2028
0.3029
0.4037
0.5033
0.6034
0.7048
0.8049
0.9044
113533
1.116528
1.2012
1.3017
1.4015
1.5018
1.6018
1.7018
1.8024
1.9015
2030
2.1025
2.2023
2.3019
2.4016
2.5017
2.6017
2.709
2.8013
2.9012
313512
3.11658
3.2022
3.303
3.4017
3.5010
3.604
3.709
3.8011
3.9012
406
4.1015
4.2010
4.3012
4.409
4.509
4.609
4.7015
4.8011
4.907
5014
5.109
5.2010
5.3011
5.406
5.505
5.6010
5.706
5.805
5.907
608
6.108
6.2012
6.308
6.404
6.505
6.604
6.708
6.806
6.904
71355
7.11652
7.2011
7.306
7.4011
7.504
7.609
7.706
7.808
7.9010
8013
8.105
8.207
8.3011
8.404
8.5010
8.6012
8.707
8.808
8.908
9010
9.109
9.203
9.304
9.407
9.505
9.609
9.707
9.8011
9.909
1004
10.106
10.207
10.3011
10.4012
10.509
10.607
10.708
10.806
10.906
11011
11.109
11.207
11.309
11.405
11.505
11.608
11.707
11.805
11.903
1208
12.105
12.203
12.305
12.406
12.505
12.606
12.705
12.803
12.907
1304
13.105
13.201
13.303
13.402
13.504
13.605
13.701
13.806
13.902
1402
14.103
14.202
14.302
14.404
14.503
14.604
14.703
14.802
14.903
151352
15.11658
15.203
15.300
15.404
15.503
15.603
15.700
15.800
15.901
1602
16.100
16.203
16.300
16.401
16.503
16.600
16.701
16.803
16.901
1702
17.100
17.200
17.300
17.400
17.501
17.600
17.700
17.800
17.900
1800
18.100
18.200
18.300
18.400
18.500
18.600
18.700
18.800
18.900
1900
19.100
19.200
19.300
19.400
19.500
19.600
19.700
19.800
19.900
2000
20.100
20.200
20.300
20.400
20.500
20.600
20.700
20.800
20.900
2100
21.100
21.200
21.300
21.400
21.500
21.600
21.700
21.800
21.900
2200
22.100
22.200
22.300
22.400
22.500
22.600
22.700
22.800
22.900
231300
23.11600
23.200
23.300
23.400
23.500
23.600
23.700
23.800
23.900
2400
24.100
24.200
24.300
24.400
24.500
24.600
24.700
24.800
24.900
2500
25.100
25.200
25.300
25.400
25.500
25.600
25.700
25.800
25.900
2600
26.100
26.200
26.300
26.400
26.500
26.600
26.700
26.800
26.900
2700
27.100
27.200
27.300
27.400
27.500
27.600
27.700
27.800
27.900
2800
28.100
28.200
28.300
28.400
28.500
28.600
28.700
28.800
28.900
2900
29.100
29.200
29.300
29.400
29.500
29.600
29.700
29.800
29.900

Exponential: the retries stay synchronised, so the recovering server's worst 100 ms saw 165 requests (it can answer 20) against 12 with full jitter. At 30 s, 280 clients were still failing; with jitter everyone was served by 17.5 s.

How the model works

Exponential delays are 1, 2, 4, 8 s, then 8 s (capped); full jitter waits a random time between 0 and that. Past its capacity the server wastes time on rejections, so it answers fewer. Each run uses a seeded random stream: reshuffle for another.

Quick checkProvider returns 200 but few users get SMS. What catches it?

Dedupe and idempotency

Duplicates come from client retries and from redelivery after a crash, so defend in three layers, each catching a different source.

Read more · 248 words

Use all three; none is enough alone.

LayerHow it worksUse it when / limit
1. API idempotency keyThe caller sends a key; the API stores it with SET NX, and a retry gets the same notificationId backEvery write endpoint a service may retry. Keep keys about a day. It misses the same event emitted twice with two different keys
2. Deterministic message idsName each message eventId:channel, so event 812 always yields 812:push, 812:email and 812:sms, however often it is processedThe dedupe key for everything downstream: sent markers, the deliveries primary key, the provider's idempotency key. Never a random id or timestamp: those change on each retry
3. Check before sending, mark afterSkip ids already marked sent; write the marker only after the provider acceptsMarking after can resend if the worker crashes in between: a rare duplicate. Marking before can lose a message when the send fails: worse

Pass the id as the provider's own where it has one, and the rare duplicate disappears too. Play mark after, crash before marking and mark before (wrong).

to a phone is impossible. Aim for effectively once (async messaging lesson).

Idempotency is a PAID stamp on a receipt: stamping it twice still leaves it paid once.

Press Play to watch it run · 7 steps

Marking sent after the provider accepts can resend after a crash, but never loses a message. Marking sent before sending can lose a message if the send then fails.

When to write the sent markerMarking sent after the provider accepts can resend after a crash, but never loses a message. Marking sent before sending can lose a message if the send then fails.deliver 812:smssent 812:sms?noalt [mark after (use this)]send, key 812:smsacceptedSET sent 812:smsack
  • Queue
  • Worker
  • Cache
  • External API
Text description
  1. Queue sends (async) to Worker: deliver 812:sms
  2. Worker → Sent markers: sent 812:sms?
  3. Sent markers replies to Worker: no
  4. Worker → Provider: send, key 812:sms
  5. Provider replies to Worker: accepted
  6. Worker → Sent markers: SET sent 812:sms
  7. Worker replies to Queue: ack

Code

deliver.js skips open breakers, fails over, and writes sent:<id> only after acceptance.

Read more · 46 words

Try the notification system challenge: build the pipeline on a canvas and get graded.

// One deterministic message per enabled channel, routed by priority
import { nextAllowedSendTime } from './quiet-hours.js';

const PRIORITY = { otp: 'critical', order_update: 'high', marketing: 'low' };

export async function notify({ eventId, userId, type, data }) {
  const prefs = await getPrefs(userId);                     // cached: { push, email, sms, types, quietHours, tz }
  const priority = PRIORITY[type] ?? 'normal';
  const channels = ['push', 'email', 'sms'].filter(ch => prefs[ch] && prefs.types?.[type]?.[ch] !== false);
  for (const ch of channels) {
    await queue.publish(`notif.${ch}.${priority}`, {
      id: `${eventId}:${ch}`,                               // deterministic → safe to dedupe
      userId, type, ch, data,
      notBefore: priority === 'low' ? nextAllowedSendTime(prefs, new Date()) : Date.now(),
    });
  }
}

Notification system in the interview

Say first that you are separating urgent traffic from bulk. Then five steps.

Read more · 515 words

  1. 0 to 8 min: name the types and their urgency

    Which channels, and which messages are time-boxed? An OTP lives about 5 minutes; a promo can wait.

  2. 8 to 15 min: put numbers on it

    500M a day is about 5,800 a second, bursts above 50K. API: 202 Accepted, an .

  3. 15 to 25 min: draw the pipeline

    Accept, check preferences, one message per channel onto queues split by channel and priority, dedicated workers, providers.

  4. 25 to 40 min: take two deep dives

    (eventId:channel, mark after acceptance) and provider failure (breakers, , delivery rate). Offer campaigns and quiet hours.

  5. 40 to 45 min: own the trade-offs

    with effectively-once results, idle critical capacity, a delayed queue to run.

If the interviewer saysYou raise
"An OTP must arrive in seconds"Priority lanes with dedicated workers and reserved provider rate
"Do not send it twice"Deterministic ids, sent marker after acceptance, provider idempotency key
"The SMS provider is down"Circuit breaker, failover order, delivery rate per provider
"Send to 10M users tomorrow at 9:00 local"Paged batch job, throttle, delayed queue per time zone, pause
"Users complain about too many pings"Daily caps, digests, quiet hours, one-click unsubscribe

Quick checkWhich design decision do you state first in a notification interview?

Interview questions

How do you make sure a notification isn't sent twice?

Queues deliver at least once, so on eventId:channel, marked sent only after the provider accepts.

A 10M-user campaign is running. How do OTPs still arrive in seconds?

Priority lanes with dedicated workers: the campaign can only fill the low lane.

The SMS provider goes down. What happens?

The breaker opens and traffic fails over to the backup.

How do quiet hours work across time zones?

Store a time zone per user and check at send time. Low priority waits in a delayed queue; critical never does.

The SMS provider returns success, but users say no code arrives. What do you check?

Delivery receipts per provider. A falling delivery rate over 5 minutes means it accepts but drops, so alert and route traffic to the backup.

How do you send 10M campaign notifications without hurting the system?

As a throttled batch job on separate workers, with a way to pause.

Check yourself

Question 1 of 5

Score 2 out of 3 or better to mark this page complete.

1. Why use separate queues per channel and priority?

Next: Last stop: how to present all of this under interview pressure.