Databases internals · Lesson 12 of 12 · 20 min

Schema Design, Normalization and Joins

Store each fact once, copy it on purpose when reads demand it, and understand how nested-loop, hash and merge joins pay for the match.

✓ Explain update anomalies and the first three normal forms✓ Decide when denormalizing is justified and how to keep copies correct✓ Compare nested-loop, hash and merge joins and choose a join type
Say it in 30 seconds
  • 3NFKey, whole key, nothing but the key
  • 100,000 vs 1,100Nested loop vs hash join steps
  • 3 join algorithmsNested loop, hash, merge
Show what to say

“I normalize first so each fact lives in one place and updates cannot disagree with themselves, up to third normal form. I denormalize only for a measured hot read path and say what keeps the copy correct. Joins run as nested loop, hash or merge, chosen by sizes, indexes and sort order, and at scale I co-locate related rows by the shard key.”

Trade-offs

  • Normalized vs denormalized: normalized changes one place but needs joins; denormalized reads fast but copies can go stale.
  • Hash vs merge join: hash needs memory for a table but takes unsorted input; merge needs sorted inputs and no extra memory.
  • Foreign keys in the database vs in code: the database refuses orphans for free correctness; some huge systems skip them for write speed.

Bring it up before they ask

Whenever you denormalize, say what updates the copy and what happens when that update fails.

0 of 0 sections ticked
30-second answer

stores each fact once so updates cannot disagree with themselves; denormalization copies facts on purpose so hot reads avoid joins. A join matches rows from two tables, and the database picks nested-loop, hash or merge by sizes, indexes and sort order.

Normalization: one fact in one place

Imagine an orders table that repeats each customer's city on every order.

Read more · 110 words

When a customer moves, every order must change. Miss one and the data disagrees with itself: an update anomaly. The same repetition causes insert and delete anomalies.

Normal forms remove such repetition. First: one atomic value per column. Second: no column depends on part of a composite key. Third: no column depends on another non-key column. In short: the key, the whole key, and nothing but the key.

A normalized shop schemaCustomers place orders, an order has many items, and each item points at a product. Every fact lives in one table; the arrows are foreign keys.customersid (PK)namecityordersid (PK)customer_id (FK)created_atorder_itemsorder_id (FK)product_id (FK)quantityunit_priceproductsid (PK)namepricemany to onemany to onemany to one

Customers place orders, an order has many items, and each item points at a product. Every fact lives in one table; the arrows are foreign keys.

  • ◆ composition (owns)
  • ◇ aggregation
  • △ inheritance / implements (dashed)
  • → association · ⇢ dependency
// Denormalized: every order repeats its customer's name and city.
// Normalized: the customer is stored once and orders point at it with customerId.
export function normalize(flatOrders) {
  const customers = new Map();
  const orders = flatOrders.map(({ id, customerId, name, city, total }) => {
    customers.set(customerId, { name, city }); // the last row seen wins, which is exactly the drift problem
    return { id, customerId, total };
  });
  return { customers, orders };
}

// Customers whose orders disagree about their own city: the update anomaly, found after the fact.
export function inconsistentCustomers(flatOrders) {
  const cities = new Map();
  for (const o of flatOrders) {
    if (!cities.has(o.customerId)) cities.set(o.customerId, new Set());
    cities.get(o.customerId).add(o.city);
  }
  return [...cities].filter(([, set]) => set.size > 1).map(([id]) => id);
}

Denormalization: copy on purpose

Joins cost time at read. So you copy: a customer_city column on orders, a precomputed total, a materialized view, or a document embedding its children.

Read more · 91 words
NormalizedDenormalized
ReadsJoin several tablesOne row or document
WritesChange one placeChange every copy
RiskSlow joinsStale or conflicting copies

Keep copies correct with a , a trigger, or an async job that tolerates staleness. Analytics star are denormalized on purpose.

Joins: nested loop, hash and merge

A join pairs rows from two tables that match on a key.

Read more · 90 words

The database has three main ways to do it, and the cost differs enormously. For 100 users and 1,000 orders, a nested loop makes 100,000 comparisons while a makes about 1,100.

EXPLAIN shows which one was chosen (Nested Loop, Hash Join, Merge Join). The planner chooses from row estimates, so stale statistics can lead it to the wrong one.

// Three ways to join users (n rows, unique id) with orders (m rows) on user id.
// Each returns the matching rows and how many comparisons or hash operations it used.
const pair = (u, o) => ({ name: u.name, total: o.total });

// Nested loop: for every user, scan every order. n x m steps (fast only when the inner side has an index).
export function nestedLoop(users, orders) {
  const rows = [];
  let steps = 0;
  for (const u of users) {
    for (const o of orders) {
      steps += 1;
      if (o.userId === u.id) rows.push(pair(u, o));
    }
  }
  return { rows, steps };
}

// Hash join: build a hash table on one side, probe it with the other. n + m steps, needs memory for the table.
export function hashJoin(users, orders) {
  const rows = [];
  const byId = new Map();
  let steps = 0;
  for (const u of users) {
    steps += 1;
    byId.set(u.id, u);
  }
  for (const o of orders) {
    steps += 1;
    const u = byId.get(o.userId);
    if (u) rows.push(pair(u, o));
  }
  return { rows, steps };
}

// Merge join: both inputs already sorted by key, so walk them together once. n + m steps, no extra memory.
export function mergeJoin(users, orders) {
  const rows = [];
  let i = 0;
  let j = 0;
  let steps = 0;
  while (i < users.length && j < orders.length) {
    steps += 1;
    if (users[i].id < orders[j].userId) i += 1;
    else if (users[i].id > orders[j].userId) j += 1;
    else {
      rows.push(pair(users[i], orders[j]));
      j += 1; // ids are unique on the user side, so only the order side advances
    }
  }
  return { rows, steps };
}
Which join algorithm will the planner pick?yesyesnonoNested loop with anindex lookupMerge joinHash joinAre both inputs alreadysorted on the join key?Few outer rows, and anindex on the innertable's join column?

Few outer rows, and an index on the inner table's join column?

Join types

Read more · 72 words
JoinReturns
INNEROnly rows that match on both sides
LEFTEvery left row, with NULLs where the right side has no match
FULLEvery row from both, matched where possible
CROSSEvery pair (n times m rows)

An anti-join finds rows with no match: customers who never ordered.

NOT EXISTS expresses the same thing. Do not write NOT IN over a column that can be NULL: it returns nothing at all.

SELECT c.id, c.name
FROM customers c
LEFT JOIN orders o ON o.customer_id = c.id
WHERE o.id IS NULL;

Keys and constraints

Read more · 143 words
  • . A surrogate key (an integer or UUID) survives changes to the real-world data. A random UUID inserts all over the , so time-ordered IDs (UUIDv7, ) keep inserts local and indexes small.
  • Foreign keys make the database refuse orphans and cascade deletes. They cost a little on writes, which is why some large systems enforce them in code instead.
  • Unique and check constraints are the cheapest correctness you can buy.

Joins at scale

When tables are , a join may need rows from different machines.

Read more · 90 words
  • Co-locate. orders and order items by the same key (customer id), so a customer's rows share a shard.
  • Broadcast a small table (countries, products) to every shard.
  • Join in the application. Fetch from each side and match in code, batching lookups (see the N+1 problem in HTTP and API styles).
  • Denormalize the column you would have joined for.

Shard choice and its costs are in sharding strategies.

Common modeling decisions

Read more · 105 words
  • Many-to-many needs a join table with two foreign keys (students, courses, enrolments).
  • Money as integer cents or a decimal type, never a float.
  • Timestamps in UTC, with the zone only at display.
  • Soft deletes (a deleted_at column) keep history but must be in every query and unique index.
  • JSON columns are fine for data you do not filter on; if you query a field often, make it a column.

Follow-up questions

What is normalization and what problem does it solve?

It stores each fact once, up to third normal form, which prevents update, insert and delete anomalies where copies of a fact disagree.

When would you denormalize?

When a measured hot read path is slow because of joins and you can keep the copies correct. Analytics and document stores do it by design.

How does a database execute a join?

By nested loop (best with an index), (build a table on the smaller input) or merge join (sorted inputs), chosen from row estimates.

Check yourself

Question 1 of 7

Score 2 out of 3 or better to mark this page complete.

1. Which phrase sums up third normal form?