Skip to main content
These decisions were made to minimise infrastructure cost, eliminate server-side compute, and keep the entire analytical pipeline reproducible from a single clone. They are recorded here so new contributors can understand why the stack looks the way it does.
Decision: Use DuckDB (via dbt-duckdb) as the core transform and analytical engine for local development, CI/CD, and production parquet compilation.Rationale: Low-overhead columnar engine, zero setup, optimised execution, and direct integration with Parquet files without requiring an active SQL database process.Tradeoff: Single-connection write limit fully sufficient for the batch transformation DAG.
Decision: Run SQL queries directly in the user’s browser using DuckDB-Wasm, querying Gold-layer Parquet files served from a CDN.Rationale: Eliminates server runtime hosting costs, scales with client bandwidth, and executes complex analytics queries in under 10 ms.Tradeoff: Initial payload includes downloading the WASM engine and base parquets mitigated via background pre-warming.
Decision: Serve Gold-layer analytics outputs as compressed Parquet files from a CDN (gs://off-the-pace-cdn) rather than running an active cloud database server.Rationale: Dramatically reduces costs, simplifies infrastructure maintenance, and enables the zero-compute DuckDB-Wasm architecture.Tradeoff: Real-time write/update capabilities are not natively supported handled by compiling static database files via offline/CI batch builds.
Decision: Train XGBoost models locally/in CI and compile them to .onnx files for direct browser-side evaluation.Rationale: Avoids server-side ML hosting costs, preserves data privacy, and enables instantaneous in-browser stint-life and tyre cliff classification.Tradeoff: Target model complexity is bounded by browser memory XGBoost fits this constraint comfortably.
Decision: Build replay_simulator.py to stream historical data to EventHub/Eventstream endpoints.Rationale: Allows 24/7 integration testing and simulation of live race streams outside actual race weekends.Tradeoff: Simulated lap interval (e.g. 90 s) is a flat approximation of real-world timing flow.
Decision: Use OLS with event dummy variables from dim_events.csv.Rationale: Interpretable and sufficient for the 22-race sample size.Tradeoff: Assumes linear degradation across the stint.
Decision: Use TimeSeriesSplit for model validation.Rationale: Guarantees temporal integrity training always precedes validation in seasonal data.Tradeoff: Fewer validation folds than random cross-validation.
Decision: Implement the public-facing dashboard as a React application styled with Tailwind CSS instead of Streamlit.Rationale: Professional visual design, deep layout customisation, and smooth integration of DuckDB-Wasm and ONNX runtimes.Tradeoff: More complex development setup and boilerplate than a Python-based Streamlit dashboard.
Decision: Include all 21 races from 2018 but accept missing race telemetry for Rd1/Rd2. F1’s archive has the Rd1 (Australia) race’s car-data stream (speed, throttle, brake) but no position stream, and FastF1 merges per-lap telemetry only when it has both; for the Rd2 (Bahrain) race it has neither stream. Qualifying at both weekends has both streams, but the pipeline ingests race telemetry only. verify_bronze.py reports the two races as known gaps (ingestion/src/bronze_checks.py, KNOWN_TELEMETRY_GAPS).Rationale: Preserves the full dataset for trend analysis; lap-time degradation signals are available as a fallback; clean to document and test.Tradeoff: Feature inconsistency for 2018 Rd1/Rd2 (laps-only vs. full telemetry). The ML layer handles this via conditional feature selection fct_lap_degradation includes both lap_time_delta and speed_delta where available. Since CO-04 (2026-10-15) the missing gap is recorded as unknown (NULL), not clear air, so both races are excluded from the field base and lose their residuals and training labels. The owner chose to keep them out rather than guess their air state.
Decision: Promote exported Gold-layer parquet + ONNX artefacts to the live CDN bucket via a dedicated, idempotent script (scripts/publish_cdn.sh), and cache-bust every parquet URL with the manifest version hash (?v=<version>).Rationale: The app reads all data from the CDN even in local dev, so make app-data alone never reaches the running app a publish step is mandatory. CDN parquet paths are not content-hashed, so a schema/data change reuses the same URL; the GCS public edge cache (Cache-Control: public, max-age=3600) would then serve stale data to all clients for up to an hour. Stamping URLs with the manifest version gives each data revision a unique key that always misses the stale cache.Tradeoff: A redundant ?v= query param on every data request, and a manual publish step. Models are exempt their filenames already carry a _vN version.
This decision was triggered by the 2026-06-11 incident where a rebuilt mart was correct on disk but the public HTTP endpoint served a months-old cached generation with a pre-expansion schema.

Deployment

How releases are cut, promoted staging→prod, and rolled back.

Conventions

Writing rules and the layer README contract.