Testing

The reality (as of 2026-06-22)

The project now has a full automated test suite built on Vitest (Node + jsdom) and Playwright (browser). It covers the active engine and the SPA across every standard test type; the legacy classic server (src/server.js, src/web/frontend/**, src/prompt-modules/**) is intentionally out of scope — it is being phased out, so only the pure stages the active core engine still imports (cleanup.js, prompt-salt.js) are tested from that folder.

Layout

tests/                         # Node-side suite (Vitest, environment: node)
  helpers/                     # seededRandom.js (mulberry32 + withSeed), fakeLoader.js
  unit/                        # pure-module unit tests
  integration/                 # engine pipeline over a fake loader
  contract/                    # (provider/API contracts live in web-app — see below)
  snapshot/                    # seeded, reproducible output snapshots
  regression/                  # bug-regression guards (one per fixed defect)
  e2e/                         # Playwright specs: home.spec, visual.spec, accessibility.spec
                               #   + visual.spec.js-snapshots/ (committed visual baselines)
vitest.config.js               # root (node) config
playwright.config.js           # builds the SPA + serves dist via `vite preview`

web-app/
  tests/                       # SPA suite (Vitest, environment: jsdom)
    *.test.js                  # unit (share, settings, customStore) + contract (providers)
    *.test.jsx                 # component/UI (Field, TokenPicker) via Testing Library
    promptEngine.integration.test.js  # real browser engine over the bundled data
    setup.js                   # jest-dom matchers + localStorage reset + RTL cleanup
  vitest.config.js             # jsdom config (reuses vite.config: react plugin, lodash alias)

Test types covered

Type	Where	Notes
Unit	`tests/unit/*`, `web-app/tests/.test.js`	contentSafety, diffSettings, keywordRepeater, gatedLists, listManifest, DPL compiler, cleanup, prompt-salt; SPA share/settings/customStore
Component / UI	`web-app/tests/*.test.jsx`	Field controls, TokenPicker — React Testing Library + jsdom
Integration	`tests/integration/**`, `web-app/tests/promptEngine.integration.test.js`	full stage pipeline via a fake loader (Node) and via the real bundled-data browser loader (SPA)
E2E	`tests/e2e/home.spec.js`	Playwright drives the built SPA: type → generate → results; block search
Visual regression	`tests/e2e/visual.spec.js`	`toHaveScreenshot` of stable chrome (topbar, sidebar, full page with the random suggestion masked)
Accessibility	`tests/e2e/accessibility.spec.js`	`@axe-core/playwright`, WCAG 2 A/AA, fails on serious/critical (color-contrast excluded — tracked)
Snapshot	`tests/snapshot/**`	seeded (Math.random) DPL + pipeline output
Contract / API	`web-app/tests/providers.test.js`	SD WebUI `txt2img` request/response contract, `fetch` mocked
Smoke	`scripts/smoke-test.mjs` (`npm run smoke`)	the original import-graph smoke, still the fast gate
Bug regression	`tests/regression/bugRegressions.test.js`	one guard per fixed defect / documented landmine

Running

npm test            # lint + smoke + Node unit/integration/snapshot/regression + SPA suite
npm run test:unit   # Node-side Vitest only
npm run test:web    # SPA Vitest only (jsdom)
npm run test:e2e    # Playwright E2E + visual + a11y (builds the SPA, serves dist)
npm run test:all    # everything, including E2E
npm run test:coverage / test:web:coverage   # with coverage
npm run test:e2e:update                      # refresh committed visual baselines

npm run test:e2e needs the Playwright browser once: npx playwright install chromium. The config sets channel: "chromium" so the full chromium build is used (no separate chromium-headless-shell download).

Windows runtime prerequisite: Chrome-for-Testing needs the Microsoft Visual C++ Redistributable. Without it, the browser fails to launch with spawn UNKNOWN → "side-by-side configuration is incorrect". On this machine the bundled Chrome-for-Testing build hit the SxS error even with the VC++ runtime present, so the config uses channel: "chrome" (the system-installed Google Chrome, version-matched to the Chromium Playwright targets). CI can drop that channel to use the bundled browser. The Vitest suites (npm test) have no browser dependency. First test:e2e run writes the visual baselines under tests/e2e/visual.spec.js-snapshots/ (committed).

Gotchas baked into the suite

lodash captures Math.random at import, so _.random / _.sample / _.shuffle cannot be stubbed by overriding Math.random. Tests that touch lodash randomness assert invariants (token counts, value shape) or use single-entry lists; only the DPL renderer (its own Math.random-based RNG) is made deterministic with withSeed.
The SPA Vitest config reuses vite.config.js so import.meta.glob (the browser loader's data bundle) and the lodash alias resolve exactly as in the real build.
Visual baselines are committed under tests/e2e/visual.spec.js-snapshots/; regenerate them on a deliberate UI change with npm run test:e2e:update.

Adding a bug-regression test

When you fix a bug, add an it("regression: …") to tests/regression/bugRegressions.test.js that fails on the old behaviour and passes on the fix, with a one-line note on the original symptom. That permanently locks the fix.