Section 7 · Learn · Data
Provenance and reproducibility
Recording where every document and every number came from, so a stranger can verify a claim without trusting the person making it.
Recording where every document and every number came from, so a stranger can verify a claim without trusting the person making it.
What it is
Provenance is the chain from a published figure back to the raw input that produced it: source URL, checksum, script, commit.
Why it is needed
A benchmark nobody can re-run is a marketing claim. The corpus is also too large to redistribute, so the manifest has to stand in for the data.
How it works
Each document records source, identifier, URL, checksum, page count, tenant and licence. Each derived dataset records the script that produced it and the command that regenerates it.
In this project
corpus_manifest.json is committed; the 719 MB of raw documents are not. Replay the URL, verify the checksum. Every precomputed interactive dataset carries describes, generated_by.script, generated_by.regenerate and generated_by.commit — 23 of 23 datasets verified as carrying full provenance. Page counts distinguish real page breaks from estimated ones, because a synthesised number presented as measured is a fabrication regardless of intent.
Trade-offs
Advantages
- Claims are checkable independently
- Large corpora need not be redistributed
- Attribution and licensing are recorded per document
Limitations
- Sources can disappear or change
- Checksums pin bytes, not meaning
Common mistakes
- Committing derived data without the script that made it
- Prose quoting figures that drift from the data beside it
When to use it
Reach for it when
- Any published measurement
How it interacts with the rest of the system
Deduplication by containmentCorpus acquisition and rate limiting