Skip to main content
A production data platform is judged on standards, not features. This page reframes the same material that lives in the Data, Transform, ML, and Platform tabs as the checklist a data-engineering reviewer actually runs down, and links each standard to where it is met in the repo.

​
Idempotent, partitioned ingestion

Bronze stores source data exactly as the API returns it, Hive-partitioned by season and race, with each writer isolated by its own try/except so one failing dataset never blocks the others. Re-running a season overwrites its partitions deterministically rather than appending duplicates, and a data-quality gate runs on write. A schema fingerprint is recorded on every run so source-side drift becomes a queryable event instead of a downstream surprise.
Met by: Ingestion architecture (per-writer isolation, fingerprinting) and Data quality (the write-time gate).

​
Data contracts and a testing pyramid

The warehouse is governed by 443 dbt tests: schema tests (not-null, unique, accepted-values, relationships) at the base, singular SQL tests for business rules in the middle, and at the top a hand-written invariant, the additive identity, that fails the build if the seven decomposition terms miss closure on any lap. Tests are the contract between layers, not an afterthought.
Met by: The CI Contract, Structural tests, and Identity closure.

​
Drift and quality gates

Silent regressions are the failure mode that matters most for an analytics warehouse, and three independent gates guard against them:
  • Byte-stability oracle: every model output is hashed build-over-build; a changed hash without a code change fails CI.
  • Build-over-build profile diff: table-level statistics are compared against a committed baseline to catch distributional drift.
  • Post-publish verify with auto-rollback: after publishing to the CDN, a verify step confirms the live files match what was built and rolls back automatically if they do not.
Met by: Regression gates and Data pipeline.

​
Lineage

Lineage is not drawn after the fact: dbt resolves the {{ ref() }} graph and builds models in dependency order, and the full DAG is generated as a browsable lineage graph. Every model and every column is documented in the Model Reference, generated from the dbt manifest so it cannot drift from the code.

​
Orchestration

The pipeline runs as a staged DAG in CI: ingest, build, test, train, export, publish, verify, each stage gated on the last, with retries and alerting on failure. There is no hidden manual step between a code change and a published, verified artifact.
Met by: Data pipeline and Deployment.

​
Observability and SLOs

Production is monitored, not hoped about: Sentry captures errors, web-vitals RUM captures real-user performance, synthetic monitors exercise the live app on a schedule, and the targets are written down as SLOs rather than left implicit.
Met by: Observability and Performance.

​
Cost model

Because all compute is build-time or client-side, serving has no per-user cost: the CDN ships static files and the browser does the work. The cloud footprint is codified in Terraform, including a billing budget with alert thresholds, so cost is a guarded number rather than a monthly surprise.
Met by: Architecture decisions (the IaC) and Performance (the zero-server serving model).

​
The scorecard in one view

See the architecture

The end-to-end system and the build-time vs request-time split.

See it answer a question

The São Paulo 2021 case study: strategy vs driver skill, in seconds.