Skip to main content

Generator Determinism

See also: Wiki Publishing, Generated Docs, Testing

Every generated artifact in this repo (the wiki/generated/ pages, the ERD, the story catalog, the Remote Config templates, the token baseline) is gated byte-exact: CI and the pre-push hook regenerate it and fail on any difference. That contract has a hard consequence:

A gated generator's output must be a pure function of repo content. Nothing from the environment may leak in — not filesystem iteration order, not OS or filesystem, not locale, not clocks, not run-to-run measurement noise. If it does, the gate fails on machines that didn't cause the difference, the printed remedy regenerates a no-op, and the author is stuck.

This page exists because we shipped that bug class three times in the wiki system's first week. Read the incident log before writing a generator.

Incident log​

DateIncidentFix
2026-08-02Prettier ping-pong. The pre-push hook's prettier --write realigned generated markdown tables; the staleness gate then saw the reformatted page as stale, and regenerating un-formatted it again — an unwinnable loop caught before the first commit.wiki/generated is prettier-ignored (.prettierignore).
2026-08-03Coverage measurement jitter. v8 attributes a few lazily-run function bodies non-deterministically across worker shards, so back-to-back runs of identical code wobble by hundredths of a point. The hook's own coverage sweep then failed wiki:check against pure noise, mid-push.0.25pp hysteresis: rows within epsilon keep the committed value (isWithinJitter, a1dd060).
2026-08-03Filesystem tie order. Two bundle outputs are byte-equal in size; the sort had no tiebreak past size, so tied rows kept readdir order — which differs across Windows/macOS/Linux. A page regenerated on one OS could never satisfy CI's Linux check (PR #97).Name as the final sort key in the bundle and Storybook pages (PR #98); the same rule audited into all sorts.

The third incident was only diagnosable because the second one prompted printing committed-vs-fresh diff lines on staleness (PR #96) — with control characters escaped, so a CRLF-only difference can't masquerade as "identical". Keep that property too.

Audit (2026-08-03)​

SurfaceOrdering / environment sourceVerdict
wiki/generated/* sortsfully keyed, name as final tiebreak✅ fixed in PR #98
tools/build-stories.ts catalogreaddirSync(...).sort()✅ safe
tools/check-design-tokens.ts baselineentries sorted before write✅ safe
tools/generate-types-erd.tsentities/edges sorted by name✅ safe
tools/generate-remote-config.tsflags sorted by key✅ safe
Line endings.gitattributes pins wiki/** to LF✅ guarded (was a latent trap for autocrlf=true checkouts)
localeCompare collationimplicit runtime locale⚠️ watch — fine while keys are ASCII and runtimes agree; prefer code-unit compares for new gated output

Checklist for a new gated generator​

  1. Every sort ends in a unique key (name/path). Never rely on readdir, glob, Map/Set iteration, or "stable sort keeps input order" — the input order is the environment.
  2. No clocks, no randomness, no locale formatting — toFixed, never toLocaleString; no dates in output.
  3. Normalize path separators to / before they reach output.
  4. Noisy measurements get tolerance, not byte-gating — if the input itself is nondeterministic (coverage, timing), absorb the noise explicitly the way the coverage page does, and gate on meaning.
  5. Machine output is prettier-ignored and written with \n.
  6. Staleness errors must show the difference — build-wiki's diffPreview does this for the wiki; match it if you build a new gate.
  7. Prove it: regenerate twice locally and let CI's Linux runner be the second platform. The spec file (tools/build-wiki.spec.ts) pins each incident's exact reproduction — add yours there.