Zeroth

A reproducible benchmark of end-to-end pipeline quality, measured on a public corpus.

Anant Sharma · 2026

Abstract

Zeroth is an open reconstruction of a production confidential-document retrieval platform, rebuilt from scratch over public documents so the design can be inspected and argued with. Every number published here was measured on the corpus described in the methodology, and applies only to it.

The benchmark measures a full pipeline rather than a retriever in isolation: hybrid retrieval, cross-encoder reranking, constrained generation, citation resolution, quote verification, and abstention. Access control is enforced inside the retrieval query itself, and its effect on measured recall is reported separately rather than folded into headline numbers.

Point at a stage for what it does. Solid = implemented; dashed = planned, with the phase that delivers it.

Figure 1. The pipeline under measurement. Solid stages are implemented; dashed stages are planned, marked with the phase that delivers them.

What is measured

Retrieval quality by and ; grounding by and ; and , which is the one most benchmarks leave out — whether the system declines when the evidence is not there.

Figure 7. Corpus composition, read from the committed manifest rather than hardcoded.

Results

In progress — Phase 5

Nine configurations, each differing from the baseline by exactly one factor, so every difference is attributable to a single change.

Waiting on the retrieval platform (Phase 2) and the evaluation harness (Phase 4)