Performance, E2E & coverage

End-to-end browser tests of the wasm subsystems, a hard JS bundle-size budget, advisory Lighthouse CI, and coverage-threshold ratchets across JS and Python suites.

This layer sits on top of the unit tests, parity contract, identity invariant, and byte oracle it closes the envelope-hardening gap for a heavily client-side app with wasm subsystems that unit tests can’t exercise.

Playwright E2E

Four specs in app/e2e/ drive the production bundle in headless Chromium and prove the three heavy client subsystems boot and produce output.

Spec	Route	Proves
duckdb-query.e2e.ts	/data-quality	DuckDB-wasm instantiates and a real parquet query resolves and renders
onnx-inference.e2e.ts	/ml/simulator	onnxruntime-web creates a session from the live CDN model and serves a finite inference
identity.e2e.ts	/lap-decomposition/waterfall	Seven-term identity UI boots and renders end-to-end
smoke.e2e.ts	/, /data-quality	SPA shell mounts with no uncaught error; client-side routing works

The signal. The observability layer keeps an always-on in-memory ring of timing marks (duckdb_init / first_query / onnx_warmup), recorded regardless of whether a RUM backend is configured, mirrored read-only to window.__OTP_PERF__. The E2E waits on those marks (app/e2e/_helpers.ts) rather than scraping fragile DOM, so a green run means the subsystem actually executed.

Run modes.

Local / CI (default)

# No E2E_BASE_URL → Playwright builds-then-serves via vite preview on :4173
# App fetches data + models from live prod GCS CDN. Needs no warehouse.
make app-e2e-install   # one-time: download Playwright Chromium
make app-e2e           # build + run specs

Chromium only DuckDB-wasm and onnxruntime-web rely on cross-origin isolation + wasm threads, and Chromium is the reference engine prod is verified against. vite preview serves with the required COOP/COEP headers (see the preview.headers block in vite.config.ts).

Bundle-size budget (blocking)

app/scripts/check_bundle_size.mjs sums the JavaScript bytes in dist/ (raw + gzip + largest single chunk; .wasm binaries excluded) and fails if any metric exceeds app/perf-budget.json.

Current baseline: ~3.0 MB raw / ~0.9 MB gzip of JS across ~70 assets; the largest single chunk is the self-hosted DuckDB worker (~0.75 MB). The committed budget carries 8% headroom so ordinary churn doesn’t flake the gate.

make app-build && make app-bundle         # check budget against current build
make app-bundle-budget-update             # regenerate budget after intentional size change

Never widen the budget to paper over an unexplained regression lazy-load the offending module instead. Run make app-bundle-budget-update and commit the new perf-budget.json in the same PR as the intentional size change.

Lighthouse CI (advisory)

app/lighthouserc.json collects Lighthouse on the data-engine-free shell routes (/, /roadmap) so scores reflect bundle/shell health rather than wasm-init variance. All assertions are warn and the step is continue-on-error it surfaces CWV/a11y regressions for review without flaking the build red. The bundle-size script is the hard byte gate.

make app-build && make app-lighthouse   # advisory; non-blocking

Coverage thresholds

Floors set just below the current baseline they guard against losing coverage (deleting tests, landing large untested modules), not as an absolute quality bar. Raise them when coverage rises; never lower without cause.

Suite	Enforced in	Floor	Baseline
vitest (app)	vitest.config.ts thresholds, run via --coverage	stmts/lines 32 · fns 33 · branches 78	~33 / 34 / 79%
ingestion (Python)	make cov-python	--cov-fail-under=55	~60%
transform coefficients (Python)	dbt-ci.yml pytest + make cov-python	--cov-fail-under=60	~68%
ml (Python)	make cov-python	--cov-fail-under=25	~28%

The ML floor (25%) reflects only the warehouse-free tests that always run. Data-dependent paths (train/predict/evaluate/export_onnx) need a populated mart and are enforced locally via make cov-python, not in ml-ci.yml’s fixture-gated subset.

Local quick reference

make app-e2e-install              # one-time: download Playwright Chromium
make app-e2e                      # build + Playwright specs vs live CDN
make app-build && make app-bundle # bundle-size budget gate
make app-lighthouse               # advisory Lighthouse (after a build)
make app-coverage                 # vitest coverage-threshold gate
make cov-python                   # Python ratchets (ingestion + coefficients + ml)

Operator setup

• Add App E2E and App Performance → perf to branch protection required status checks once they’ve run green on a PR.
• After a deliberate bundle change, run make app-bundle-budget-update and commit the new perf-budget.json in the same PR.
• (Optional) Wire E2E_BASE_URL to the preview.yml channel URL to smoke the actual deployed artefact instead of a local vite preview.

Observability

The window.__OTP_PERF__ ring the E2E tests wait on, and the SLOs the synthetic monitor enforces.

Security

Required branch protection status checks including App E2E and App CI.