Testing

See also: Local Development · CI CD and Deploys · Tooling and Scripts

Three layers: unit tests (Vitest), Storybook interaction tests, and Cypress e2e (web + mobile).

Reproducing CI locally (read this before chasing a "CI-only" failure)

CI is not a different test suite — it is the SAME commands on a smaller machine. The variable that changes behavior is total resident load on the box. Vitest 4 with isolate: true (our default) forks a FRESH process per test file, but three things still accumulate machine-wide: the MAIN vitest process holds the whole project's v8 coverage in memory for the entire run (open vitest issue #4476) — multiplied by however many projects nx runs concurrently; DYING forks pile up (terminations are "started but not awaited until the end of the run", each lingering up to 60s with open handles, so resident processes exceed maxWorkers); and any leaked handle keeps its dying fork alive longer. Oversubscribe a small box with that and it GC/swap-thrashes wholesale — at which point trivial synchronous tests time out (that tell means the MACHINE is frozen, not that the test is slow; no timeout bump fixes it). Slow lanes make it worse, which is why capping workers backfired. (Historical note: the vitest-3 era really did reuse one worker across many files — the mock-bleed sagas below date from then, and void-ed rejections / leaked timers remain fail-the-file bugs today.)

Because of that, CI (ci.yml test job) runs FULL vitest workers (fewest files per fork) but NX_PARALLEL=1 (one project owns the 4-vCPU box at a time) with NODE_OPTIONS=--max-old-space-size=3072 so a runaway process dies fast with a V8 trace instead of swap-thrashing the runner into the "lost communication" abyss. Never cap VITEST_MAX_WORKERS on CI — a 2026-07-31 attempt at '50%' packed ~85 files into each fork and produced exactly the frozen-fork wedge, three runs in a row.

To reproduce a CI-shaped failure locally, the worker cap is the STRESS AMPLIFIER — it forces the many-files-per-fork world on any machine:

VITEST_MAX_WORKERS='50%' bunx nx vite:test <project> --coverage --skip-nx-cache
# or the full CI sweep shape, cross-project parallelism included:
VITEST_MAX_WORKERS='50%' bunx nx affected -t vite:test --base=origin/main --coverage

Run it a few times — worker packing is order-dependent. Known CI-death signatures:

• "Worker exited unexpectedly", no V8 heap trace, ~last files missing → the OS OOM-killed a fork (capacity). The dead worker's in-flight files are the suite files that never printed a ✓ line — if they're all trivial, suspect capacity or a cumulative leak from files that DID run, not the missing files themselves.
• "The hosted runner lost communication" at ~49 min, no logs at all → the runner agent itself starved: either a synchronous spin (hung while-loop, wedged scheduler) or whole-box memory thrash from bloated forks. Every job now carries timeout-minutes: 30, which bounds the loss and (when the agent is still breathing) preserves logs.
• Trivial sync tests timing out (even at 15s) → the fork is FROZEN (GC death spiral), not slow. Reduce files-per-fork / process count; never raise timeouts for this.
• A spec FILE fails while all its tests pass → an unhandled rejection or leaked handle detonating after the last test. Rules that prevent the class: every void someAsync() must be unable to reject (catch inside, or fail-closed like AuthService.accountClaims); every setTimeout/interval/ rAF a component starts is cleared via DestroyRef; specs use mockReturnValueOnce for poisoned values (vi.clearAllMocks does NOT clear implementations — a persistent never-settling promise once cost a file 12 silent seconds).

Never retry past a pre-push hook flake — the hook runs the CI commands, so an intermittent hook failure IS the CI failure with better odds. Chase it with the env knob above.

Unit tests (Vitest + Angular TestBed)

Projects use the vite:test nx target (Vitest with @analogjs/vitest-angular / @analogjs/vite-plugin-angular, jsdom), so Angular TestBed works inside Vitest specs.

bunx nx vite:test stretched-components                              # one project
bunx nx affected --target=vite:test --base=origin/main --head=HEAD  # what CI / pre-push run
bun run test:functions                                              # Firebase Functions (own vitest setup in apps/firebase-functions)

The tools/generate-types-erd.ts generator has its own spec run via bun run types:erd:test (see Tooling and Scripts).

Storybook play / interaction tests

Stories in apps/storybook double as tests. bunx nx e2e storybook-e2e (or bun run storybook:e2e) builds Storybook and runs tools/storybook-e2e.ts, which serves dist/storybook on port 6199 and runs @storybook/test-runner (Playwright, chromium) against it:

• Every story is smoke-tested in a real browser — any render or console error fails.
• Stories with a play function get their interactions asserted.

The script installs the required chromium build itself (idempotent). Storybook provides DI-level mocks by default, so no backend is needed. See Component Library Guide for writing stories and play functions.

Visual regression (Storybook) — base vs branch

bun run storybook:visual                  # diff main → working tree, full report
bun run storybook:visual -- --filter card # only stories whose id/title matches
bun run storybook:visual -- --all         # ignore change detection, shoot everything
bun run storybook:visual -- --base <ref> --reuse-builds --themes light,dark

Parallelism follows the machine: the two storybook builds and the two screenshot sides run concurrently, and each side's page pool defaults to half the repo-wide NX_PARALLEL budget (the same machine-derived knob pre-push and CI export), capped at 8 — override per-run with --concurrency N.

tools/storybook-visual-diff.ts answers "what am I visually accepting on this branch?":

1. Builds the base ref's storybook in a detached git worktree (tmp/visual-diff-base, cached incl. its node_modules) and the working tree's storybook.
2. Screenshots every story on both sides with Playwright (light + dark via prefers-color-scheme, animations disabled, fullPage). Stories are enumerated from index.json, and the shot waits for Storybook's render lifecycle to settle (phase finished) — so play functions have already run and screenshots capture post-interaction end states. Adding a story (with or without play) adds its visual coverage automatically; nobody writes screenshot tests. Speed comes from in-place story switching (each worker page boots Angular once, then setCurrentStory — the test-runner's own mechanism) and from skipping networkidle in favor of the render-phase wait; a stability loop (two consecutive identical captures, 3 s cap) makes JS-driven animations (d3 transitions, count-ups) deterministic — full suite ≈ 790 shots + diffs in under 2 minutes with builds reused. Known noise floor: infinitely-animating stories (acrostics, tickers) never settle and can show sub-0.25 % ratios at the bottom of the report. The Playwright driving runs in a Node worker (shoot-worker.ts, bundled at runtime) — Playwright cannot control its browser from Bun on Windows.
3. Pixel-diffs the pairs (pixelmatch, images padded to union size so a size change counts) and writes dist/storybook-visual-diff/report.html: added/removed stories pinned to the top, then changed stories sorted most-different-first, identical/skipped collapsed below.
4. Change detection ("TurboSnap at home", tools/visual-diff/affected.ts): a story is skipped when its transitive TS import closure (plus sibling .scss/.html in closure directories) doesn't intersect the changed files. Changes to global surfaces — foundations/, apps/storybook/, package.json, bun.lock — invalidate every story. Unmappable stories are never skipped silently. Unit specs: bun run storybook:visual:test.

Both sides are screenshotted on the same machine in the same browser build, so there is no cross-platform baseline problem — the diff is pure code-change signal. CI/branch-deployment wiring is future work.

Visual regression (apps + extension) — base vs branch

bun run app:visual       # web app, all routes, vs a fresh seeded emulator
bun run mobile:visual    # mobile app at 390×844 (browser build of the Ionic shell)
bun run extension:visual # extension popup + annotated fixture pages
# all take --base <ref> and --reuse-builds; app runners take --filter/--themes

tools/app-visual-diff.ts builds both sides with --configuration=emulated, starts a fresh auth+firestore emulator (same no-import/no-export contract as e2e — refuses to run while the dev emulator is up), and screenshots every route in light + dark. Reports land in dist/app-visual-diff/<app>/report.html and dist/extension-visual-diff/report.html, most-different-first.

⚠️ The rendered world is deliberately fully unlocked, and every report carries a banner saying so:

• Every feature flag is ON — emulated builds run @stretched/feature-flags in local mode with forceAllEnabled: true (no Firebase contact, nothing to seed), so flag-gated pages render enabled even though they ship default-OFF. New registered flags join automatically. The --flags-off variant shots neutralize the force via the __STRETCHED_FLAGS_DEFAULTS_ONLY__ global (set by the shoot worker before app boot) to capture the customer-default view.
• The emulated account test@test.com is signed in with the admin: true custom claim (granted via the emulator's privileged endpoint), so /admin/* pages render. The Firebase session is injected into IndexedDB before app boot — no UI login flow.
• The extension runner forces all extension feature toggles ON with a deterministic $25/h rate, loads the unpacked MV3 build via a persistent Chromium context (channel: 'chromium' — the headless shell doesn't support extensions), and shots the popup plus local fixture pages with fixed prices (tools/visual-diff/fixtures/) that the content script annotates. Extension shots never touch live retailer pages.

Routes listed in the manifests' *_FLAGS_OFF_ROUTES get a second, flags-off shot (report rows suffixed --flags-off): after the all-on pass the seeded doc is cleared and the page re-shot on the compiled defaults — the customer-reality view of pages a partial-page flag trims (today: /account, which offers only Minutes + Hours while web_section_upper_tiers_enabled is OFF). Add a route there whenever a partial-page flag lands.

Route coverage is drift-gated: the manifests in tools/visual-diff/route-manifests.ts must cover every path in the apps' route tables — route-manifests.spec.ts (part of storybook:visual:test, pre-push + CI) fails when a new route lacks a manifest entry or an explicit exclusion. Serving uses the CSR index for all routes so guards and flags evaluate against live emulator state, never a stale prerendered page. Mobile note: home/dashboard sit behind incomeStoryGuard and deterministically render the onboarding redirect for a fresh account; native-shell truth (safe areas, native fonts) is a future Android-emulator phase.

E2E (Playwright) — web, mobile, and extension

bun run e2e:playwright                    # all three surfaces (~15 s + builds)
bun run e2e:playwright -- --project=web   # web | mobile | extension
bun run e2e:playwright -- --grep layaway  # any Playwright Test flag

tools/e2e-playwright.ts builds the emulated apps (reusing existing dists) plus the Firebase Functions (built + npm-installed into dist/apps/firebase-functions), starts a fresh auth+firestore+functions emulator (same refuse-to-reuse contract as the Cypress runner), seeds the unlocked world (all feature flags ON, test@test.com with the admin claim; per-spec accounts that need paid content grant themselves a tier claim via grantClaims, because the admin role no longer opens member content — see Auth and Users), serves the builds + price fixtures, and runs Playwright Test under Node. Running the functions emulator means flows like account deletion and the GDPR data export (account-deletion.e2e.ts) exercise the REAL users function — the e2e asserts via the emulator's owner-bypass REST API that the Auth account and every uid-keyed Firestore doc are actually gone. Specs live in apps/playwright-e2e (see its README): web auth/navigation/admin-gate, mobile shell + guest journey, and the extension's first-ever e2e — annotation, tooltip, multi-currency, popup persistence, layaway sign-in gating — driving the unpacked MV3 build in a persistent full-Chromium context against local fixtures. Signed-in states are injected via the Firebase IndexedDB session (no UI login round-trip) except in specs where the login flow itself is the subject. Runs in CI's e2e job.

Policy: the Cypress suites below are frozen — keep them green, invest nothing new. New e2e coverage goes to apps/playwright-e2e; migrate the remaining Cypress specs opportunistically, then retire Cypress and its Windows workarounds.

E2E (Cypress) — web and mobile (frozen)

bun run e2e:emulated          # web suite (apps/stretched-e2e) on :4200
bun run e2e:mobile:emulated   # mobile suite (apps/stretched-mobile-e2e) on :4300

Both run tools/e2e-emulate.ts, which:

1. Refuses to start if anything is listening on :9099 — running against the dev emulator would pollute its persisted emulator-data/ with test users. Stop bun run emulate first.
2. Starts a fresh Firebase emulator (--only auth,firestore) with no import and no export — e2e state never touches emulator-data/ and nothing persists after the run.
3. Runs nx e2e <project> --configuration=emulated with CYPRESS_EMULATED=1.
4. Force-kills the emulator (nothing to export) and exits with Cypress's exit code.

CYPRESS_EMULATED gating

Emulator-only specs — e.g. apps/stretched-e2e/src/e2e/auth.cy.ts (creates/deletes accounts) and story-builder.cy.ts — gate themselves with:

const emulatedDescribe = Cypress.env('EMULATED') ? describe : describe.skip;

So a plain bunx nx e2e stretched-e2e or a CI run against deployed dev can never create test users in a real Firebase project; those suites simply skip.

How the e2e targets work (important gotcha)

The e2e targets in apps/stretched-e2e/project.json and apps/stretched-mobile-e2e/project.json use nx:run-commands + tools/cypress-e2e.ts, not the @nx/cypress:cypress executor. That executor starts the dev server and then reports success without ever launching Cypress (observed on nx 22.7.5 + @nx/cypress 22.7.1 with continuous serve targets). Do not "simplify" the targets back to it.

tools/cypress-e2e.ts <serveProject> <serveConfiguration> <e2eDir> <baseUrl> does serve → wait → run:

1. Serves the app (bunx nx serve <project> --configuration=<config>) — unless CYPRESS_BASE_URL is already set (the deploy-preview CI runs Cypress against a deployed channel URL; see CI CD and Deploys), in which case no dev server is started.
2. Polls the base URL until it answers (up to ~2 minutes).
3. Runs bunx cypress run in the e2e project dir with CYPRESS_BASE_URL set to the target URL, passing CYPRESS_EMULATED through.

It also strips NX_WORKSPACE_ROOT_PATH and ELECTRON_RUN_AS_NODE from the environment — the VS Code terminal quirks described on Tooling and Scripts.

What runs where

Layer	Local command	CI
Unit (affected)	bunx nx affected --target=vite:test ...	ci.yml PR job + pre-push hook
Functions	bun run test:functions:coverage	ci.yml + pre-push hook (with --coverage)
Storybook e2e	bun run storybook:e2e	— (build verified in deploys)
Web e2e	bun run e2e:emulated	ci.yml e2e job; deploy-preview against the channel URL
Mobile e2e	bun run e2e:mobile:emulated	—

Coverage thresholds (enforced)

Every project pins coverage.thresholds in its vitest config, floored at the level its suite actually achieves — so any --coverage run fails if coverage regresses. CI's affected vite:test --coverage sweep and the dedicated functions step both run with coverage, which is what makes the thresholds a real gate. Current floors (statements/branches/functions/lines):

Project	Thresholds
stretched-types	100 / 100 / 100 / 100
firebase-functions	100 / 98 / 100 / 100
stretched-chrome-extension	100 / 98 / 100 / 100
stretched (web)	98 / 85 / 99 / 99
stretched-mobile	99 / 95 / 100 / 99
stretched-components	96 / 88 / 96 / 98

Raise a floor when you add tests; never lower one to make a change pass — uncoverable code (bootstrap/wiring, Storybook demos) belongs in that project's commented coverage.exclude list instead. The remaining uncovered branches are documented per-project in the config comments (SSR typeof window guards, WebView-only wiring, jsdom Selection/Range limits in the page builder).

See CI CD and Deploys for the full pipeline.