Tooling · Case Study

Worksheet Forge

How do you generate endless math practice without ever printing a wrong answer key?

I tutor middle and high-school math, and the same two problems came up every week: writing fresh practice takes longer than teaching it, and a hand-written answer key is wrong often enough to cost a student real confidence. Worksheet Forge is the tool I built to remove both. It turns a list of topics into a typeset PDF (problems first, answer key attached), and it is built around two invariants that the test suite treats as regressions if violated.

Invariant 1

Problems are never hardcoded.

No generator may return a problem from a literal list. Every number a student sees is drawn from a seeded RNG inside difficulty-scaled ranges, which is what stops -8 + 5 from being question 1 of every worksheet forever.

Invariant 2

No PDF is emitted with an unverified key.

Every answer is re-derived independently of how the generator produced it: the printed question is parsed back into sympy and re-solved. Any mismatch raises and fails the build loudly.

Build pipeline

A build runs topics through a catalog that decides section grouping and directions, draws each problem from a seeded RNG, then refuses to render anything until every answer has survived verification. The catalog chooses which generators run, never which problems. That distinction is what keeps the first invariant enforceable.

Topic list

negatives fractions slope

›

or YAML spec

hand-authored sections

↓

Catalog

sections · directions · subskill progression

↓

Seeded draw

random.Random(seed)

↓

Generators

backwards from the answer

›

Anti-repeat ledger

blocks the last 5 runs

↓

verify_all()

every key re-derived, or the build stops

↓

LaTeX

Jinja template

↓

PDF

problems, then the answer key

The verification step

Verification is the part I care most about. The naive version, having the generator store the answer it computed, then check it against it check the answer against itself, catches nothing, because a generator with a sign error will confidently agree with its own mistake. So the checker never sees the generator's arithmetic. It reads the rendered LaTeX string, the same one the student will read on paper, parses it back into sympy, and solves it from scratch. Roughly 120 strategies cover the topic catalog, and a separate check enforces that the string being verified literally appears in the printed question, since otherwise nothing guarantees the verified expression is the one that got typeset.

Problem

question_latex + answer_expr

↓

Fragment check

verified string must appear in the question

›

Strategy lookup

~120 kinds

↓

LaTeX → sympy

parsed from the printed string

↓

Re-solve

independent of the generator

↓

Match

→ render

›

Mismatch

→ VerificationError

Watch it run

Pick a topic and step through a real draw. Stage 1 is what the generator sampled; stage 2 is the string that would be typeset; stage 3 re-reads that string with no knowledge of stage 1; stage 4 compares. Then tick the box at the bottom to corrupt the answer key and watch the build refuse it.

1. Generator draws

sample

x_sol = 5, m = 6, b = 8

solution picked first, then the right-hand side computed from it, so no degenerate cases are possible

↓

2. Rendered to LaTeX

render

6x + 8 = 38

this exact string is what gets typeset onto the sheet

↓

3. Parsed back to an expression

re-parse

6x + 8 = 38

read from the printed LaTeX alone; the generator's own arithmetic is never consulted

↓

4. Re-solved and compared

re-solve

5 = 5

match: this problem may be rendered

✓ verify_all() passed: the worksheet may now be rendered

Corrupt the answer key. Nudges the printed answer by one, the way an off-by-one in a generator would. Stages 1 to 3 are untouched, so the mismatch is caught purely by re-deriving the answer from the printed question.

Generate a worksheet

The same generators, wired to the controls the CLI exposes. Reroll as many times as you like: no problem here came from a list, so the numbers keep changing while the skill being drilled stays put. Difficulty is not just bigger numbers: in linear equations it is the probability that the solution itself lands on a fraction, which is the step where students actually stall. Versions A/B/C are fully independent draws, for handing neighbours different sheets.

Topics

Difficulty

Per topic: 5

Versions

seedshow answer key

$ python -m forge quick negatives:5:medium linear_equations:5:medium --seed 42

Practice Set

Name: ______

Part A: Negative Numbers

Evaluate each expression completely.

1. $10 + 14$
2. $-56 \div (-8)$
3. $-18 + 7$
4. $-56 \div 8$
5. $2 + 2$

Part B: Linear Equations

Solve each equation for x.

1. $4x + 7 = -29$
2. $5x - 1 = 3x - 17$
3. $6x + 5 = -7$
4. $-2x + 4 = -10x - 24$
5. $3x - 8 = -9$

10 problems generated · every key re-derived and verified ✓

Notes on this demo

This page is a browser reimplementation of the parts of the tool it shows. The real one is Python, uses sympy for verification, and emits LaTeX compiled to PDF. Seeds here reproduce this page, not a CLI worksheet.
Five topics are ported for the demo. The real catalog covers 14 topics and roughly 60 subskills, from signed arithmetic through trigonometry, sequences, and probability.
The verifier running above is genuine, not a mock: it tokenizes the printed LaTeX, rebuilds the expression, and re-solves it. It clears 24,000 generated problems with no failures, and it is what rejects the corrupted key.