Operating systems and concurrency · Lesson 1 of 12 · 20 min

Processes, Threads and Context Switches

What a process owns, what threads share, and what the CPU actually does when it switches between them.

✓ Say what a process owns and what its threads share✓ Explain what a context switch saves and why it costs✓ Choose processes, threads or an event loop for a workload
Say it in 30 seconds
  • a few µsDirect cost of one context switch
  • ~1 MBStack per thread on a typical JVM
  • ~9%CPU lost to 50 µs slices
Show what to say

“A process owns an address space; threads are paths of execution inside it that share the heap and code but keep their own stack and registers. A context switch saves and restores registers, and the real cost is the cold cache and TLB afterwards. I pick processes for isolation, threads for shared state, and an event loop for many idle connections.”

Trade-offs

  • Process vs thread: a process contains crashes but is heavy to start; a thread is cheap and shares memory, but one bug kills all.
  • Thread per request vs loop: threads are simple; an event loop holds thousands of idle sockets on one stack, but CPU loops block it.
  • Short vs long time slice: short slices respond faster, long slices lose less time to switching and cache refill.

Bring it up before they ask

Say that counter++ is load, add, store, so two threads can lose an update, before the interviewer asks about locks.

0 of 0 sections ticked
30-second answer

A process is a running program with its own private memory. Threads are the paths of execution inside it, and they share that memory, which is why they are cheap to switch and easy to break.

Process or thread: what is the difference?

A process is a house; threads are the people living in it.

Read more · 88 words

The house has its own address and front door. The people share the kitchen but each keeps their own bed.

ProcessThread
OwnsAddress space, open files, PIDStack, registers, program counter
SharesNothing by defaultCode, heap, globals, open files
CrashOthers are safeTakes the whole process down

What a context switch saves, and why it costs

When a thread's time slice ends, the kernel : it saves the running thread's registers (including the program counter and stack pointer) and loads the next thread's.

Read more · 108 words

The switch takes a few microseconds. The hidden cost is bigger: the next thread meets and a (the cache of address translations) full of someone else's data. A switch between processes also changes the address space, so it costs more.

A 50 microsecond slice loses about a tenth of the CPU, which is why a thread per request melts down under load.

One core, two threads, and the switch between them0 ms2.7 ms5.4 ms8.0 ms10.7 ms13.4 msThread AA runsA runs: 0–4 msA resumesA resumes: 9.4–13.4 msKernelsave A, load B: 4–4.7 mssave B, load A: 8.7–9.4 msThread BB runsB runs: 4.7–8.7 mstimer interrupttime slice ends

A timer interrupt stops thread A. The kernel saves A's registers, loads B's, and B runs. The switch itself is drawn much wider than it really is.

Text description
  • Thread A: A runs (0–4 ms), A resumes (9.4–13.4 ms)
  • Kernel: save A, load B (4–4.7 ms), save B, load A (8.7–9.4 ms)
  • Thread B: B runs (4.7–8.7 ms)
// What share of the CPU goes to switching instead of working?
// All times are in microseconds. `refill` is the extra time the next thread spends
// re-warming the cache and TLB; it often costs more than the switch itself.
export function overhead(sliceUs, switchUs, refillUs = 0) {
  const lost = switchUs + refillUs;
  return lost / (sliceUs + lost);
}

Sharing memory is the danger

Threads share memory, so two can touch one variable at once. counter++ is not one step: the CPU does LOAD, ADD, STORE, and another thread can slip in between.

Read more · 57 words

If both load 0 before either stores, both store 1 and one update vanishes: a . A lock makes the three steps one critical section. The next lessons cover locks and alternatives.

// Two "threads" each run counter++, which the CPU does as LOAD, ADD, STORE.
// The schedule says who moves next, so we can replay any interleaving.
export function run(schedule, { locked = false } = {}) {
  const mem = { counter: 0 };
  const t = [{ pc: 0, reg: 0 }, { pc: 0, reg: 0 }];
  const ops = [(th) => { th.reg = mem.counter; }, (th) => { th.reg += 1; }, (th) => { mem.counter = th.reg; }];
  let owner = null;

  const step = (id) => {
    const th = t[id];
    if (th.pc === 3) return false;
    if (locked && th.pc === 0) {
      if (owner !== null && owner !== id) return false; // the lock is held: wait
      owner = id;
    }
    ops[th.pc](th);
    th.pc += 1;
    if (locked && th.pc === 3) owner = null; // unlock after STORE
    return true;
  };

  for (const id of schedule) step(id);
  while (t.some((th) => th.pc < 3)) if (!step(0) && !step(1)) throw new Error("stuck");
  return mem.counter;
}

fork, exec and starting a thread

fork() clones the calling process. The child gets a copy of the address space, which the kernel shares copy-on-write: pages are copied only when someone writes.

Read more · 65 words

exec() then replaces the child's code with a new program. Shells do exactly this for every command you run.

Starting a thread (pthread_create, new Thread) allocates a stack and a control block and shares everything else. It is much cheaper than a fork, but a thread's stack is real memory you pay for: about a megabyte of address space each on a typical JVM.

Press Play to watch it run · 7 steps

The parent asks the kernel to clone it, the child replaces its own code with a new program, and the parent waits for the exit status.

Starting a program: fork, exec, waitThe parent asks the kernel to clone it, the child replaces its own code with a new program, and the parent waits for the exit status.fork()clone: same memory, copy-on-writechild's PIDexec("ls"): replace code and datawait(pid)Skip wait() and the exited child stays in the process table as a zombieexit(0)exit status 0
  • Service
  • API gateway
  • Worker
Text description
  1. Parent → Kernel: fork()
  2. Kernel sends (async) to Child: clone: same memory, copy-on-write
  3. Kernel replies to Parent: child's PID
  4. Child → Kernel: exec("ls"): replace code and data
  5. Parent → Kernel: wait(pid)
  6. Child → Kernel: exit(0)
  7. Kernel replies to Parent: exit status 0

Kernel threads, green threads and the event loop

Most languages map one language thread to one kernel thread (1:1), so the OS schedules it.

Read more · 110 words

Go and Erlang run many cheap green threads (goroutines, processes) on a few kernel threads (M:N), so a goroutine costs a few kilobytes, not a megabyte.

Node.js uses neither: one thread runs your JavaScript and an hands it each finished I/O event. It handles thousands of idle sockets with one stack, but a long CPU loop blocks everyone. CPython's default build has a global interpreter lock, so threads help with waiting on I/O but not with CPU-bound Python code.

Pick one: process, thread or event loop

Ask in this order: does a failure need to stay contained, is the work mostly waiting, and does it share state?

Read more · 27 words

Real systems mix them. Chrome gives each site its own process, each process runs several threads, and a Node deployment often runs one process per CPU core.

Process, thread or event loop?yesyesnoyesyesnononoSeparate processesEvent loop (async I/O)ThreadsThousands of connectionsat once?ThreadsSeparate processesDoes the work need toshare memory?Does the work mostlywait on the network ordisk?Must a crash or a bugstay out of the otherwork?

Must a crash or a bug stay out of the other work?

Where this goes wrong

Read more · 122 words
  • Zombies. A child that exits but is never wait()ed for stays in the process table. Orphans are reparented to init and are harmless.
  • One thread can kill the process. A segfault or an uncaught out-of-memory in any thread ends all of them.
  • Fork in a threaded program. The child gets only the calling thread, but copies the locks other threads held. It can instantly.

Follow-up questions

What is the difference between a process and a thread?

A process owns an address space and resources. Its threads share the heap and code but keep their own stack and registers.

What happens during a context switch?

The kernel saves the running thread's registers, picks another thread and loads its registers. A process switch also changes the page tables.

How many threads should a server have?

About one per core for CPU-bound work, more for work that waits. Measure as you raise it.

Check yourself

Question 1 of 7

Score 2 out of 3 or better to mark this page complete.

1. Two threads of one process are running. Which of these does each thread have its own copy of?

Next: Threads share memory, and memory is virtual. Next: how the OS schedules them and maps their addresses.