Manifest report: every ingestion run, queryable

Turn the per-run manifest Parquet files into a status summary what’s ok, skipped, or errored, and whether FastF1’s schema drifted between runs.

Every ingestion attempt ok, skip, or error is logged as one row in data/bronze/manifests/run_<run_id>.parquet, written once per ingest.py invocation (see Architecture). manifest_report.py reads every manifest file back and answers the two questions the raw Parquet doesn’t answer on its own: what’s the latest status per session, and did FastF1’s schema change underneath you.

make manifest-report

========================================================================
INGESTION MANIFEST REPORT
========================================================================
Runs on record:  3
Sessions tracked:  47  (ok=45, skip=1, error=1)

Latest-run ERRORS (need attention):
  ✗ 2019 Rd9 R    austrian_grand_prix

Schema drift:
  ✓ no fingerprint changes detected   FastF1 schema stable across all runs

​

Last-run status

For every (season, round_number, session_type) combination that’s ever been attempted, the report keeps only the most recent manifest row so re-running make ingest-recent after a transient error supersedes the earlier failure rather than reporting it twice. Anything still showing error or thin after the latest run is printed by name and needs attention.

run_id · string

UTC timestamp the run started (YYYYMMDDTHHMMSSZ) sortable by construction, used to order runs chronologically.

season / round_number / session_type · int / int / 'R' | 'Q'

The session this row describes.

status · 'ok' | 'skip' | 'error' | 'thin'

ok written this run. skip already existed on disk, untouched. error failed the DQ gate or raised during load. thin loaded, but key columns came back missing even after one retry, so nothing was written and the next run pulls it again. Sessions that have not been run yet get no row at all.

row_count / dq_passed / duplicate_lap_keys · int / boolean / int

Row count of the laps written, whether the schema check passed, and how many duplicate (race_id, DriverNumber, LapNumber) keys were found.

schema_fingerprint · string

12-character SHA-1 of the sorted column names empty for skip/error/thin rows, since no write happened.

Runs from 2026-09-25 on add the columns below. Manifests written earlier lack them, and the report treats a missing column as “not recorded”.

thin_reasons / known_gap_note · string / string

The completeness checks the session failed (results.Points 0% non-null (< 90%), …), and the documented source gap that explained any of them (ingestion/src/bronze_checks.py). On an ok row a non-empty thin_reasons means it was written with --accept-thin.

telemetry_laps_attempted / telemetry_laps_merged / telemetry_coverage · int / int / float

Race only: laps per-lap telemetry was tried on, laps it came back for, and the share. Healthy races are above 99.7%; below 95% is flagged.

telemetry_car_drivers / telemetry_pos_drivers / telemetry_note · int / int / string

How many drivers the session had car-data (speed/throttle) and position (X/Y/Z) samples for, and the commonest per-lap errors or low coverage / known gap.

circuit_info_status · string

Race only: ok: N corners, none, or failed: <error> (e.g. a MultiViewer 404). A failure is tolerated but recorded here, not only in the log.

venue_note · string

Set when the event’s slug named a different venue in earlier seasons (2026: spanish_grand_prix is Madrid, not Barcelona), or a new slug appears at a known venue.

​

Schema drift detection

How fingerprint comparison works

Within each (season, session_type) group, manifest rows are walked in chronological order. Every time schema_fingerprint changes from one successful write to the next, that’s a drift event FastF1 added, removed, or renamed a column between the two runs. Each transition is reported with the from/to fingerprint and the timestamp it changed, so a silent upstream schema change can’t slip into the warehouse unnoticed.

Schema drift is compared within a season, not across seasons. A 2018 fingerprint differing from a 2024 fingerprint isn’t drift different seasons are expected to have different schemas. Drift means the same (season, session_type) produced a different fingerprint on a later run, which only happens on a re-ingest with --force.

​

Flags on the last write

For each session the report also reads its most recent non-skip row (a later skip row carries none of the columns above) and lists anything written but worth a look: low telemetry coverage, an accepted thin load, no circuit info, a venue change. These are informational.

The report’s exit code is non-zero whenever any session shows error or thin in its latest run, or any drift was detected making make manifest-report usable as a CI gate, not just an interactive tool.

​

Next

Replay

Step through one race’s laps to spot-check what a manifest row’s data actually looks like.