Errors, testing and debugging

01 · THREE KINDS OF ERROR

Three kinds of error

Errors come in three kinds, and they show up at different times. On the stage, three machines: one that never starts, one that starts and then stops dead, and one that runs perfectly and gives the wrong answer.

A syntax error means the text isn't valid code: a bracket missing, a typo in a keyword. The parser can't build a tree, so nothing runs at all, not even the first line.

console.log("Does this line run?");
​
const scores = [4, 5, 6;
console.log(scores);

A runtime error happens while the program is running, when it asks for something impossible: a property of undefined, a function that doesn't exist. The lines before it ran; the lines after never will.

console.log("start");
​
const player = { name: "Ada" };        // no scores yet
console.log(player.scores.length);     // asks undefined for .length
​
console.log("never printed");

A logic error is the quiet one. The program runs to the end and prints a confident, wrong answer. No message, no line number. This is the bug the rest of this chapter hunts:

function average(xs) {
  let sum = 0;
  for (let i = 1; i < xs.length; i++) {
    sum += xs[i];
  }
  return sum / xs.length;
}
​
console.log("average:", average([4, 5, 6]));   // should be 5

02 · EXCEPTIONS

Exceptions: throw, and catch

When a function hits a problem it can't handle itself, it can throw an exception: an object that describes what went wrong. Throwing stops the function at once. The exception travels down the call stack, abandoning each function in turn (this is called unwinding), until it reaches code that said it can deal with it: a try block with a catch.

On the stage, main calls loadScores, which calls parseLine, which calls toNumber("x"). The exception drops through three frames and lands in main's catch, the green ring. The program carries on.

function toNumber(text) {
  const n = Number(text);
  if (Number.isNaN(n)) {
    throw new Error(`"${text}" is not a number`);
  }
  return n;
}
function parseLine(line) {
  return line.split(",").map(toNumber);
}
function loadScores(lines) {
  return lines.map(parseLine);
}
​
try {
  console.log(loadScores(["4,5,6", "7,8,9"]));
  console.log(loadScores(["4,5,6", "7,x,9"]));
  console.log("not reached");
} catch (err) {
  console.log("Caught:", err.message);
} finally {
  console.log("finally always runs");
}
console.log("and the program carries on");

finally runs whether the code succeeded, threw, or returned early: it is where clean-up goes (closing a file, releasing a lock). Without any catch, an exception goes all the way down, and the program, or in a browser this one task, stops with "Uncaught".

Errors can carry more than a message. Make your own kind by extending Error, catch only what you understand, and pass the rest on. When you add context, keep the original as the cause, so nothing is lost:

class ParseError extends Error {
  constructor(message, line) {
    super(message);
    this.name = "ParseError";
    this.line = line;
  }
}
​
function parse(lines) {
  return lines.map((text, i) => {
    const n = Number(text);
    if (Number.isNaN(n)) throw new ParseError(`bad number "${text}"`, i + 1);
    return n;
  });
}
​
function load(lines) {
  try {
    return parse(lines);
  } catch (err) {
    if (err instanceof ParseError) {
      // add context, keep the original as the cause
      throw new Error(`scores.txt line ${err.line} is broken`, { cause: err });
    }
    throw err;                 // not ours: let it keep going up
  }
}
​
try {
  load(["4", "5", "six"]);
} catch (err) {
  console.log(err.message);
  console.log("  caused by", err.cause.name + ":", err.cause.message);
}

Asynchronous code throws too. With await, a failed promise becomes an ordinary exception you can catch with the same try. (Chapter 9 is about async.)

async function fetchScores() {
  // pretend this waited on the network, then failed
  throw new Error("server said 503");
}
​
try {
  const scores = await fetchScores();
  console.log(scores);
} catch (err) {
  console.log("Caught an async error:", err.message);
}

03 · ERRORS AS VALUES

Errors as values

Throwing is one way to report a problem. The other is to return it, as an ordinary value. JavaScript does both. Some built-ins answer "no" with a special value: indexOf returns -1, Number("x7") returns NaN, a missing key gives undefined. They never crash, which is exactly the danger: NaN quietly spreads through every sum it touches.

The stage shows the deliberate version. parse never throws. Every input comes out as a result with either a value or an error in it, and the code that called it has to look before it uses it: green to one bin, red to the other.

// 1. Values that quietly mean "no answer"
console.log([4, 5, 6].indexOf(9));       // -1: not found
console.log(Number("x7"));               // NaN: not a number
console.log(new Map().get("missing"));   // undefined
console.log(0 / 0, NaN + 1, NaN === NaN);// NaN spreads, and isn't equal to itself
​
// 2. Return the error on purpose, Go and Rust style
function parse(text) {
  const n = Number(text);
  if (Number.isNaN(n)) return { ok: false, error: `"${text}" is not a number` };
  return { ok: true, value: n };
}
​
for (const t of ["42", "x7", "19"]) {
  const r = parse(t);
  if (r.ok) console.log("got", r.value);
  else console.log("skipped:", r.error);
}

Some languages are built on this idea. Go functions return the error as a second value, and checking it is a habit the whole language leans on. Rust returns a Result that is either Ok or Err, and the compiler refuses to let you use the value without deciding what to do with the error.

n, err := strconv.Atoi("x7")
if err != nil {
    // the error is an ordinary value: you must look at it
    return fmt.Errorf("reading score: %w", err)
}
fmt.Println(n)

fn read_score(text: &str) -> Result<i32, ParseIntError> {
    let n = text.parse::<i32>()?;   // ? : on Err, return it to the caller
    Ok(n)
}
 
match read_score("x7") {
    Ok(n) => println!("got {n}"),
    Err(e) => println!("skipped: {e}"),   // the compiler makes you handle both
}

Which to use? A rough rule: return errors that are a normal part of the job (bad user input, a file that may not exist), and throw for things that shouldn't happen if the program is correct. Whichever you choose, never let an error vanish: a catch that does nothing, or a NaN nobody checks, turns a loud bug into a silent one.

04 · STACK TRACES

Reading a stack trace

When an exception isn't caught, it arrives with a stack trace: a snapshot of the call stack at the moment it was thrown. It looks like noise until you know how to read it. On the stage, each line of the trace is wired to the frame it came from.

• Top line: the kind of error and its message. TypeError: xs is not iterable means something tried to loop over a value that can't be looped over.
• Next line: where it was thrown, as file:line:column. Here, line 3, inside average.
• Each line below: who called that. report on line 7, called by main on line 11, called from line 14.

function average(xs) {
  let sum = 0;
  for (const x of xs) sum += x;
  return sum / xs.length;
}
function report(player) {
  return player.name + ": " + average(player.scores).toFixed(1);
}
function main() {
  const players = [{ name: "Ada", scores: [4, 5, 6] }, { name: "Grace" }];
  for (const p of players) console.log(report(p));
}
​
main();

The trace is a value you can read from code, too: it lives in err.stack. (It isn't part of the ECMAScript standard yet, but every major engine has it; each formats it slightly differently. Here the sandbox names your code your-code.js, and the bottom frame says eval because that is how the sandbox starts your file.)

function average(xs) {
  let sum = 0;
  for (const x of xs) sum += x;
  return sum / xs.length;
}
function report(player) {
  return player.name + ": " + average(player.scores).toFixed(1);
}
function main() {
  const players = [{ name: "Ada", scores: [4, 5, 6] }, { name: "Grace" }];
  for (const p of players) console.log(report(p));
}
​
try {
  main();
} catch (err) {
  console.log(err.name + ": " + err.message);
  // err.stack is the trace. Keep the lines about this file:
  for (const line of err.stack.split("\n")) {
    if (line.includes("your-code")) console.log(line.trim());
  }
}

Python prints its trace the other way up, oldest call first, with the error at the bottom, and points at the exact expression that failed:

Traceback (most recent call last):
  File "stats.py", line 14, in <module>
    main()
  File "stats.py", line 12, in main
    print(report(p))
  File "stats.py", line 8, in report
    return p["name"] + ": " + str(average(p["scores"]))
                                          ~^^^^^^^^^^
KeyError: 'scores'

Two habits make traces useful. First, read the whole message: it usually says exactly what is wrong. Second, the top frame is where the program noticed; the cause is often further down, in your code, where a bad value was made. Skip frames inside libraries until you reach a file you wrote.

05 · THE BUG HUNT

The bug hunt: a method, not luck

Debugging is the scientific method applied to a program. Beginners often change things at random until the bug goes away; studies of novice programmers find exactly that, and it is slow. The loop on the stage is faster:

1. Reproduce it. Find input that makes the bug happen every time. A bug you can't reproduce, you can't confirm you fixed.
2. Guess why. Form one specific hypothesis: "the sum is wrong".
3. Predict. If that's true, what should you see? "The sum should reach 15."
4. Test it, with a print, a breakpoint or an experiment, and let the result narrow the search.

The simplest test is a print statement: a probe that reports what is true at one moment. Logging an object, { i, x: xs[i], sum }, labels every value for free:

function average(xs) {
  let sum = 0;
  for (let i = 1; i < xs.length; i++) {
    console.log({ i, x: xs[i], sum });     // a probe: what is true right now?
    sum += xs[i];
  }
  console.log("final sum:", sum, "count:", xs.length);
  return sum / xs.length;
}
​
// Hypothesis: the sum is wrong. Prediction: it should reach 15.
average([4, 5, 6]);

A good reproduction is also a small one. Shrink the failing input until the bug has nowhere to hide. Here, one item is enough, and the answer 0 almost names the bug for you:

function average(xs) {
  let sum = 0;
  for (let i = 1; i < xs.length; i++) sum += xs[i];
  return sum / xs.length;
}
​
// Shrink the failing input until the bug has nowhere to hide
console.log(average([4, 5, 6]));   // 3.67, expected 5
console.log(average([4, 5]));      // 2.5,  expected 4.5
console.log(average([4]));         // 0,    expected 4  ← the first item vanishes

06 · BISECT

Bisect: halve the haystack

Sometimes you know the code worked last week and is broken today, but not which of the changes since broke it. Don't read every change. Test the one halfway between the good and the bad: whichever way it goes, half the suspects are cleared. This is binary search, and it is called bisecting.

On the stage, 16 commits: the bracket closes from 0 to 15, then 7 to 15, 7 to 11, 9 to 11, 10 to 11. Four tests, and commit 11 is the culprit. This program does exactly the same:

// 16 commits. Commit 0 worked; commit 15 is broken.
// One commit in between broke it. Which?
const FIRST_BAD = 11;                      // the secret we are hunting for
const works = (commit) => commit < FIRST_BAD;
​
let good = 0, bad = 15, tests = 0;
while (bad - good > 1) {
  const mid = Math.floor((good + bad) / 2);
  tests++;
  const ok = works(mid);
  if (ok) good = mid; else bad = mid;
  console.log(`test ${tests}: commit ${mid} is ${ok ? "good" : "bad "} → look in ${good}..${bad}`);
}
console.log(`First bad commit: ${bad}, found in ${tests} tests`);

Git does this for you. Mark one commit good and one bad, and git bisect checks out the middle commit for you to test; give it a test script and it runs the whole search alone:

$ git bisect start
$ git bisect bad                 # the commit you're on is broken
$ git bisect good v1.4           # this old release worked
Bisecting: 7 revisions left to test after this (roughly 3 steps)
$ npm test                       # ...test the commit git checked out
$ git bisect good                # or: git bisect bad
  ...
a1c9e2f is the first bad commit
$ git bisect reset
 
# or let a script decide every step for you:
$ git bisect run npm test

The same trick works on anything you can cut in half: comment out half of a function, delete half of an input file, turn off half of the settings. Whatever still fails holds the bug.

07 · THE DEBUGGER

The debugger, and the duck

A debugger lets you pause a running program and look around. You set a breakpoint on a line (the red dot on the stage); when execution reaches it, everything freezes. Now you can read every variable, see the call stack, and step: run one line, step into a function call, or step out of the current function.

On the stage the debugger pauses on line 4 each time round the loop. The watch panel shows i = 1 on the first stop, and sum ends at 11 instead of 15. The Step through button on every code block in this course is a small debugger. Try it here:

function average(xs) {
  let sum = 0;
  for (let i = 1; i < xs.length; i++) {
    sum += xs[i];
  }
  return sum / xs.length;
}
​
const result = average([4, 5, 6]);
console.log(result);

Every browser has a full debugger built in (open DevTools, go to Sources, or Debugger in Firefox, and click a line number), and so do VS Code, Python (pdb) and C (gdb). Worth learning early: conditional breakpoints (pause only when i === 500), logpoints (print without editing the code), and pausing on any exception. Writing debugger; in JavaScript pauses there when DevTools is open.

08 · KINDS OF TEST

Unit, integration and end-to-end tests

Debugging finds a bug once. A test is a small program that checks your program, so the bug can never come back unnoticed. At its heart a test is just: run the code, compare the result with what you expected, complain loudly if they differ. That comparison is an assertion.

Here is a whole test runner in ten lines, testing our buggy average. Real ones (Jest, Vitest, Node's built-in node:test, Python's pytest) add nicer reports, but do the same:

// A tiny test runner: the same idea as Jest, Vitest or node:test.
function test(name, fn) {
  try {
    fn();
    console.log("✓", name);
  } catch (err) {
    console.log("✗", name, "→", err.message);
  }
}
function assertEqual(actual, expected) {
  if (!Object.is(actual, expected)) {
    throw new Error(`expected ${expected}, got ${actual}`);
  }
}
​
// The code under test (still buggy)
function average(xs) {
  let sum = 0;
  for (let i = 1; i < xs.length; i++) sum += xs[i];
  return sum / xs.length;
}
​
test("one number is its own average", () => assertEqual(average([7]), 7));
test("the average of 4, 5, 6 is 5", () => assertEqual(average([4, 5, 6]), 5));
test("an empty list averages to 0", () => assertEqual(average([]), 0));

Tests come in sizes, and the stage stacks them as a pyramid. At the bottom, many unit tests: one function, no network, no disk, milliseconds each. In the middle, fewer integration tests: several parts together, often with a real database or file. At the top, a few end-to-end tests that drive the real app like a user, through a browser. Higher tests catch more kinds of bug but are slower and break for more reasons, so you want fewer of them.

Integration tests find bugs that live between units. Both functions below are correct on their own; real input, with a blank line in it, breaks them together:

// Two units that each pass their own tests...
function parseScores(text) {           // "4,5,6" -> [4, 5, 6]
  return text.split(",").map(Number);
}
function average(xs) {
  if (xs.length === 0) return 0;
  return xs.reduce((a, b) => a + b, 0) / xs.length;
}
​
// ...and an integration test that runs them together, on real input
function report(fileText) {
  return fileText.trim().split("\n").map((line) => average(parseScores(line)));
}
​
const file = "4,5,6\n10,20\n\n7";      // a blank line, as real files have
const got = report(file);
console.log(got);
console.assert(got.length === 3, "expected 3 rows, got", got.length);

// tests/scores.spec.js · Playwright drives a real browser
import { test, expect } from "@playwright/test";
 
test("a teacher sees the class average", async ({ page }) => {
  await page.goto("http://localhost:3000/class/7b");
  await page.getByLabel("Upload scores").setInputFiles("fixtures/7b.csv");
  await expect(page.getByRole("cell", { name: "Average" })).toContainText("5.0");
});

Assertions aren't only for tests. Put them in your code to state what must be true (an invariant), so a wrong assumption fails loudly and early, near its cause:

function average(xs) {
  console.assert(Array.isArray(xs), "average() needs an array, got", xs);
  if (xs.length === 0) return 0;
  const result = xs.reduce((a, b) => a + b, 0) / xs.length;
  // an invariant: the mean always lies between the smallest and largest value
  console.assert(result >= Math.min(...xs) && result <= Math.max(...xs), "mean out of range:", result);
  return result;
}
​
console.log(average([4, 5, 6]));
console.log(average("456"));    // the assertion names the real problem before the crash

09 · TEST FIRST

Test first: red, green, refactor

Test-driven development (TDD) turns the order round: write the test before the code. Kent Beck set it out as a cycle of three short steps, the three lamps on the stage:

1. Red. Write one small test for behaviour that doesn't exist yet. Run it and watch it fail, which proves the test can fail.
2. Green. Write the simplest code that makes it pass. Nothing more.
3. Refactor. Now clean up the code (and the tests), running them after every change. They stay green, or you know at once what you broke.

function test(name, fn) {
  try { fn(); console.log("✓", name); }
  catch (err) { console.log("✗", name, "→", err.message); }
}
function assertEqual(a, b) {
  if (!Object.is(a, b)) throw new Error(`expected ${b}, got ${a}`);
}
​
// RED: the tests come first. median() doesn't exist yet, so they fail.
function median(xs) {
  throw new Error("not written yet");
}
​
test("odd count: the middle one", () => assertEqual(median([3, 1, 2]), 2));
test("even count: halfway between the middle two", () => assertEqual(median([4, 1, 3, 2]), 2.5));
test("one item", () => assertEqual(median([9]), 9));

Why bother? Writing the test first makes you decide what the code should do before you are attached to how it does it, keeps every step small, and leaves a full test suite behind as a side effect. It suits code with clear inputs and outputs best. When exploring an idea you don't understand yet, many programmers sketch first, then write the tests once the shape is clear. Either way, when you fix a bug, write the test that reproduces it first: a regression test, which makes sure that bug stays fixed.

10 · PROPERTY TESTS

Property-based testing

An example test checks one input you thought of. A property test states a rule that must hold for every input, then lets the computer try hundreds of random ones. For sorting: the output is in order, and it has the same items as the input.

When an input breaks the rule, the tester shrinks it: it keeps trying smaller versions while they still fail, so you get the simplest example of the bug, not a messy one. On the stage, 13 random lists pass, the 14th fails, and it shrinks from [13, 13, 11, 13, 2] to [10, 2]. Run the real thing:

// A seeded random generator, so a failure can be replayed exactly.
let seed = 2;
function random() {
  seed = (seed * 1103515245 + 12345) % 2147483648;
  return seed / 2147483648;
}
const randomInt = (lo, hi) => lo + Math.floor(random() * (hi - lo + 1));
// Like QuickCheck, start with small inputs and let them grow run by run.
const randomList = (size) =>
  Array.from({ length: randomInt(0, Math.min(size, 8)) }, () => randomInt(0, size));
​
// The code under test. Looks right. Isn't.
function sortNumbers(xs) {
  return xs.slice().sort();
}
​
// The property: the result is in order, and nothing is lost.
function holds(xs) {
  const out = sortNumbers(xs);
  const ordered = out.every((x, i) => i === 0 || out[i - 1] <= x);
  return ordered && out.length === xs.length;
}
​
// Shrink: keep making the failing input smaller while it still fails.
function shrink(xs) {
  let changed = true;
  while (changed) {
    changed = false;
    for (let i = 0; i < xs.length && !changed; i++) {
      const fewer = xs.filter((_, j) => j !== i);           // drop one item
      if (!holds(fewer)) { xs = fewer; changed = true; }
    }
    for (let i = 0; i < xs.length && !changed; i++) {
      for (const smaller of [Math.trunc(xs[i] / 2), xs[i] - Math.sign(xs[i])]) {
        const closer = xs.slice();
        closer[i] = smaller;                                 // move one toward 0
        if (!changed && smaller !== xs[i] && !holds(closer)) { xs = closer; changed = true; }
      }
    }
  }
  return xs;
}
​
for (let run = 1; run <= 200; run++) {
  const input = randomList(run);
  if (!holds(input)) {
    console.log(`Run ${run} failed on`, input);
    console.log("sortNumbers gave", sortNumbers(input));
    const small = shrink(input);
    console.log("Shrunk to", small, "→", sortNumbers(small));
    break;
  }
  if (run === 200) console.log("200 random lists passed");
}

The idea comes from QuickCheck (Claessen and Hughes, 2000) for Haskell. Today there is a library for most languages: fast-check for JavaScript, Hypothesis for Python. Good properties to look for: a round trip (decode what you encode and get it back), an invariant (sorting never changes the length), or a slow but obviously right version to compare against.

from hypothesis import given, strategies as st
 
@given(st.lists(st.integers()))
def test_sort_keeps_order_and_items(xs):
    out = my_sort(xs)
    assert all(a <= b for a, b in zip(out, out[1:]))
    assert sorted(out) == sorted(xs)
 
# Falsifying example: test_sort_keeps_order_and_items(xs=[10, 2])

Its wilder cousin, fuzzing, throws huge numbers of random or mutated inputs at a program looking for crashes. Google's OSS-Fuzz runs it nonstop on open-source projects. It has found bugs in some of the most-used code there is.

11 · COVERAGE

Code coverage, and its limits

Code coverage measures which parts of your code ran while the tests ran: the share of lines (line coverage), or of branches, the paths through each if (branch coverage). Tools such as Istanbul (behind Jest's and Vitest's --coverage) and Python's coverage.py add counters to your code to measure it.

Coverage is good at one thing: showing code that no test touches, so you know where to look. It cannot tell you the tests are good. On the stage the gauge reaches 100%, and the bug is still sitting on its line:

const ran = new Set();                 // which lines ran: a home-made coverage tool
​
function average(xs) {
  ran.add("sum = 0");     let sum = 0;
  ran.add("loop");        for (let i = 1; i < xs.length; i++) {
  ran.add("sum += x");      sum += xs[i];
                          }
  ran.add("return");      return sum / xs.length;
}
​
// A "test" that runs every line, and checks almost nothing
const result = average([4, 5, 6]);
console.log("test passed:", typeof result === "number");
console.log("line coverage:", ran.size + "/4 =", (ran.size / 4) * 100 + "%");
console.log("actual answer:", result, "(should be 5)");

Coverage also can't see code that isn't there. The missing if (xs.length === 0) on the stage is a line no tool can report as uncovered. Studies agree: once you account for how many tests a suite has, coverage is only weakly linked to how many bugs it catches. Google's engineers suggest roughly 60% as acceptable, 75% as commendable and 90% as exemplary, and warn against chasing 100%.

12 · TYPES, LINT, LAYERS

Types, linting, and layers of defence

The cheapest bug is the one caught before the code even runs. Two kinds of tool read your code without running it (static analysis) and work like tests you never had to write.

A type checker knows what kind of value every variable can hold and refuses code that mixes them up. TypeScript would have caught Grace's missing scores from the stack-trace act before anyone ran it:

type Player = { name: string; scores?: number[] };   // scores may be missing
 
function report(p: Player) {
  return p.name + ": " + average(p.scores);
  //                             ~~~~~~~~
  // error TS2345: Argument of type 'number[] | undefined' is not
  // assignable to parameter of type 'number[]'.
}

A linter (ESLint for JavaScript, Ruff or Pylint for Python) looks for patterns that are legal but usually wrong: == where you meant ===, a variable you never use, code that can never run. Here is a toy one; real linters read the parse tree, not raw text:

// A ten-line linter: real ones (ESLint) read the parse tree, not raw text,
// but the idea is the same: spot risky patterns without running the code.
const source = `
let total = 0
for (let i = 1; i < scores.length; i++) total += scores[i]
if (total == "0") console.log("nothing")
const unused = 42
`;
​
const rules = [
  [/==(?!=)/, "use === : == converts types first (0 == '0' is true)"],
  [/for \(let \w+ = 1;/, "loop starts at 1: is skipping the first item on purpose?"],
  [/const (\w+) = /, "check this name is used somewhere"],
];
source.split("\n").forEach((line, n) => {
  for (const [pattern, message] of rules) {
    if (pattern.test(line)) console.log(`line ${n}: ${message}`);
  }
});

No single check catches everything, and that is the point of the last station. Each layer is a sheet with holes in it: types, lint, unit tests, code review, then a canary release that shows new code to a small share of real users first. Most bugs hit a solid part of some sheet. A bug only reaches everyone when the holes line up.

The Pipeline page lets you ship a bad commit through all of these layers and watch a canary catch it. Bugs That Changed History replays what happened when the holes did line up: in July 2024, a CrowdStrike content update whose template expected 21 input fields, while the sensor supplied 20, crashed about 8.5 million Windows machines at once, and a staged rollout was one of the fixes promised afterwards.