DSA & System Design
All the basics

What is sharding?

Sharding splits one big dataset into slices by a key, so each machine holds only one slice and does only its share.

One overflowing cabinet is split into three narrower ones, and each holds only one slice of the alphabet.

Press Play to watch it run · 3 steps

One database gets too big or too busy. Splitting the data by a key gives each machine a slice, with two catches.

One database versus slicesOne database gets too big or too busy. Splitting the data by a key gives each machine a slice, with two catches.Your appOne databaseRouterpicks by keySlice 1Slice 2Slice 3

Tap any box to see what it is and what it talks to.

  • Service
  • Database
  • API gateway
Open in playground
Text description
  1. Millions of users all ask one database.
  2. Its disk is nearly full and its processor is maxed out.
  3. Answers slow down. A bigger machine helps for a while, then costs too much.
  • Sharding splits the data by a key across machines.
  • Each machine holds a slice and does only its share.
  • Questions that span slices are harder, and a popular key runs hot.

Reach for it when

When one database cannot take the writes (tens of thousands a second), even with caching and copies. Not for 2K writes.

The catch

A celebrity's posts all hit one slice and it melts. Questions across slices get slower, and moving data later hurts.

Say it in an interview“I would shard only when one machine cannot cope. I would pick a key that spreads load evenly and keeps common queries on a single shard.”