Operating systems and concurrency · Lesson 2 of 12 · 25 min

Scheduling and Memory: Virtual Memory, Paging and the TLB

How the OS picks who runs next, and how every process gets its own private address space through paging.

✓ Compare FCFS, shortest-job and round robin scheduling✓ Walk an address through the page table and the TLB✓ Explain a page fault, thrashing, and stack versus heap
Say it in 30 seconds
  • 17 ms vs 3 msAverage wait, first come first served vs shortest first
  • 4 KiBUsual page size; low 12 bits are the offset
  • ~2% slowerMemory access with a 99% TLB hit rate
Show what to say

“The scheduler trades average wait against fairness and switching cost: shortest job first minimises waiting but starves long jobs, round robin keeps everyone moving. Virtual memory gives each process a private address space through page tables, a TLB caches translations, and a page fault is the kernel loading a page that is not in RAM.”

Trade-offs

  • Shortest job first vs round robin: lowest average wait but needs burst times and starves long jobs, against fairness with no knowledge.
  • Small vs huge pages: 4 KiB pages waste little memory; 2 MiB pages need far fewer TLB entries for big heaps.
  • Stack vs heap: the stack is fast, freed on return, but small and per thread; the heap is large and shared but needs an allocator.

Bring it up before they ask

Virtual memory is not swap: even with no swap a process still has translated addresses, and thrashing means the working set exceeds RAM.

0 of 0 sections ticked
30-second answer

The scheduler decides who runs next; gives each process private addresses, translated page by page. The keeps that fast, and a is the OS fetching a page not yet in RAM.

Scheduling: who runs next

The scheduler picks which ready thread gets a core, and for how long. Each policy trades average wait against fairness and overhead.

Read more · 125 words
PolicyIdeaCatch
First come, first servedRun in arrival orderA long job blocks the short ones
Shortest job firstRun the shortest nextNeeds burst times; long jobs starve
Round robinA quantum each, then the back of the queueTiny quantum wastes time switching

Take jobs of 24, 3 and 3 ms. Average wait is 17 ms first come first served, 3 ms shortest first, and about 5.7 ms round robin, which needs no burst times. Real schedulers blend these.

Round robin with a 4 ms quantum: A needs 24 ms, B and C need 3 ms each0 ms6 ms12 ms18 ms24 ms30 msJob A (24)AA: 0–4 msA runs aloneA runs alone: 10–30 msJob B (3)B doneB done: 4–7 msJob C (3)C doneC done: 7–10 msquantum ends

A runs for 4 ms, then B and C each finish within their first turn, then A runs alone to the end. B and C wait 4 and 7 ms instead of 24 and 27.

Text description
  • Job A (24): A (0–4 ms), A runs alone (10–30 ms)
  • Job B (3): B done (4–7 ms)
  • Job C (3): C done (7–10 ms)
// Jobs arrive together at time 0; each is { name, burst } in milliseconds.
// Wait time = finish - burst; turnaround = finish.
function summarize(finish, jobs) {
  const avg = (xs) => xs.reduce((a, b) => a + b, 0) / xs.length;
  return {
    finish,
    avgWait: avg(jobs.map((j) => finish[j.name] - j.burst)),
    avgTurnaround: avg(jobs.map((j) => finish[j.name])),
  };
}

export function fcfs(jobs) {
  const finish = {};
  let t = 0;
  for (const j of jobs) finish[j.name] = t += j.burst;
  return summarize(finish, jobs);
}

// Shortest job first: the best average wait, but it needs to know the burst and can starve long jobs.
export const sjf = (jobs) => fcfs([...jobs].sort((a, b) => a.burst - b.burst));

// Round robin: each job runs for at most `quantum`, then goes to the back of the queue.
export function roundRobin(jobs, quantum) {
  const left = Object.fromEntries(jobs.map((j) => [j.name, j.burst]));
  const queue = jobs.map((j) => j.name);
  const finish = {};
  let t = 0;
  while (queue.length) {
    const name = queue.shift();
    const ran = Math.min(quantum, left[name]);
    t += ran;
    left[name] -= ran;
    if (left[name] > 0) queue.push(name);
    else finish[name] = t;
  }
  return summarize(finish, jobs);
}

Virtual memory gives each process a private address space

Every process sees addresses starting at zero, as if it owned all memory. Hardware and kernel translate them to real RAM, which gives isolation.

Read more · 32 words

Memory is cut into pages (usually 4 KiB) and RAM into frames. A per-process page table maps page numbers to frame numbers; the low 12 bits are the offset and pass through.

Press Play to watch it run · 3 steps

The CPU asks the TLB first. On a hit the physical address is ready at once. On a miss the hardware walks the page table. If the page is not in RAM, the kernel handles a page fault.

Translating one virtual addressThe CPU asks the TLB first. On a hit the physical address is ready at once. On a miss the hardware walks the page table. If the page is not in RAM, the kernel handles a page fault.page 3, offset 0xA7C?alt [TLB hit]frame 7 (about 1 ns)read 0x7A7C
  • Client
  • Cache
  • Database
  • Object storage
  • External API
Text description
  1. CPU → TLB: page 3, offset 0xA7C?
  2. TLB replies to CPU: frame 7 (about 1 ns)
  3. CPU → RAM: read 0x7A7C

Walk an address through the TLB and page table

Click the three tabs. Address 0x3A7C is page 3, offset 0xA7C; if page 3 is in frame 7, the physical address is 0x7A7C.

Read more · 96 words

The page table lives in memory, so the CPU recent translations in the : a hit takes about a nanosecond, and are typically above 99%.

export const PAGE_SIZE = 4096;

export class PageFault extends Error {
  constructor(vpn) {
    super(`page fault on page ${vpn}`);
    this.vpn = vpn;
  }
}

// A virtual address is (page number, offset). The TLB remembers a few recent page -> frame answers.
export class Mmu {
  constructor(pageTable, tlbSize = 2) {
    this.pageTable = pageTable; // Map: page number -> frame number
    this.tlbSize = tlbSize;
    this.tlb = new Map(); // insertion order = least recently used first
    this.stats = { tlbHits: 0, walks: 0, faults: 0 };
  }

  translate(vaddr) {
    const vpn = Math.floor(vaddr / PAGE_SIZE);
    const offset = vaddr % PAGE_SIZE;
    let frame = this.tlb.get(vpn);
    if (frame !== undefined) {
      this.stats.tlbHits += 1;
      this.tlb.delete(vpn); // re-insert below to mark it most recent
    } else {
      this.stats.walks += 1; // TLB miss: walk the page table in memory
      frame = this.pageTable.get(vpn);
      if (frame === undefined) {
        this.stats.faults += 1; // not mapped: the kernel must step in
        throw new PageFault(vpn);
      }
      if (this.tlb.size >= this.tlbSize) this.tlb.delete(this.tlb.keys().next().value);
    }
    this.tlb.set(vpn, frame);
    return frame * PAGE_SIZE + offset;
  }
}

Page faults, replacement and thrashing

A means "this page is not mapped or not present".

Read more · 124 words

Often that is normal.

FaultCauseCost
MinorPage is in RAM, mapping missing (new or copy-on-write page)Microseconds
MajorPage must be read from diskMilliseconds
InvalidAddress was never validSegmentation fault

When RAM is full the kernel a page, ideally the least recently used ().

Thrashing is when the active pages no longer fit in RAM: a loop over five pages with four frames faults on every access, and the machine waits on disk.

// How many page faults does a reference string cause with `frames` physical frames?
// FIFO evicts the page that has been in memory longest; LRU the one not used for longest.
export function faults(refs, frames, policy = 'lru') {
  const mem = []; // front = next victim
  let count = 0;
  for (const page of refs) {
    const at = mem.indexOf(page);
    if (at >= 0) {
      if (policy === 'lru') mem.push(...mem.splice(at, 1)); // a use makes it the newest
      continue;
    }
    count += 1;
    if (mem.length === frames) mem.shift();
    mem.push(page);
  }
  return count;
}

Stack versus heap

Read more · 59 words
StackHeap
HoldsCall frames: locals, return addressObjects that outlive a call
OwnerOne per threadShared by all threads
FreedOn returnBy you or a garbage collector
FailureStack overflowOut of memory, leaks

The stack is fast: allocating is moving a pointer. In JavaScript or Java, objects live on the heap and frames hold references.

The cost of translation

A flat page table is only affordable for small address spaces.

Read more · 127 words

Real tables are trees: x86-64 uses four levels of nine bits, so a miss costs up to four memory reads.

With memory at 100 ns, a TLB at 1 ns and a single-level table, a 99% gives 0.99 × 101 + 0.01 × 201 = 102 ns, only 2% slower than raw memory. At a 90% hit rate it is 111 ns. Huge pages (2 MiB) need far fewer TLB entries, so databases and JVMs with big heaps often use them.

Scheduling traps

Read more · 112 words
  • Starvation. Pure priority or shortest-job policies can leave a task waiting forever. Aging raises the priority of waiting tasks.
  • is not swap. Swap is only where evicted pages can go. A server without swap still uses virtual memory.
  • fork is cheap. Copy-on-write shares pages until one side writes.

Follow-up questions

What is virtual memory and why do we need it?

Each process gets its own address space, translated page by page to physical frames. It isolates processes and lets programs use more address space than RAM.

What happens on a TLB miss, and on a page fault?

A miss makes the hardware walk the page table. A fault means the page is absent: the kernel loads it and retries.

What is thrashing?

Working sets exceed RAM, so pages are evicted just before they are needed again.

Check yourself

Question 1 of 7

Score 2 out of 3 or better to mark this page complete.

1. Jobs of 24, 3 and 3 ms arrive together. Which policy gives the lowest average wait?

Next: Many threads can now run at once. Next: what goes wrong when they touch the same data.