Section 10
Learn
Every concept this project is built on, explained in depth and tied to where it actually appears in the implementation.
26 topics across 7 areas. Each opens in place Each has its own page covering the concept, why it exists, how it works, how it is used here, the trade-offs, and the mistakes that are easy to make. Nothing is included that this project does not actually use.
Retrieval
How a question becomes a ranked list of passages.
- BM25 and lexical searchA ranking function that scores documents by term overlap with the query, weighted by how rare each term is and normalised for document length.
- Cross-encoder rerankingA second-stage model that reads the query and a passage together and scores their relevance directly, reordering the shortlist retrieval produced.
- Embeddings and dense retrievalText mapped to a vector so that passages with similar meaning land near each other, letting search work by meaning rather than by shared words.
- Hybrid retrievalRunning lexical and dense retrieval independently and merging their results, because each finds passages the other misses.
- Reciprocal Rank FusionMerging ranked lists using each item's rank position rather than its score, so incomparable scoring scales never have to be reconciled.
- Retrieval-Augmented GenerationAnswer questions from a document corpus by retrieving relevant passages first, then generating an answer grounded in them.
Vector search
How nearest-neighbour search works, and what it costs in exactness.
Data
How the corpus is built, split and kept reproducible.
- Chunking strategiesSplitting documents into passages small enough to embed and retrieve, without destroying the structure that makes them meaningful.
- Corpus acquisition and rate limitingFetching a corpus from public sources politely, resumably, and in a way a stranger can repeat.
- Deduplication by containmentDetecting when one document is wholly contained inside another, which Jaccard similarity cannot do.
- Multi-tenancy and tenant assignmentPartitioning a corpus into isolated groups so access control and leakage tests are meaningful rather than theatrical.
- Provenance and reproducibilityRecording where every document and every number came from, so a stranger can verify a claim without trusting the person making it.
Security
How access control is enforced, and what it does to retrieval.
- ANN post-filtering under access controlAn approximate index picks its nearest neighbours first and the access policy removes forbidden ones afterwards — leaving gaps nothing refills.
- Partitioning by tenantSplitting a table so each tenant's rows live in their own partition with their own index, removing the post-filtering problem rather than mitigating it.
- PostgreSQL Row-Level SecurityAccess rules enforced by the database inside the query itself, so a row a role may not see is never returned regardless of how the query was written.
Evaluation
How the numbers are produced and what they depend on.
- Abstention and knowing when not to answerDeclining to answer when the retrieved evidence does not support one — and measuring whether the system does it for the right reason.
- Bootstrap confidence intervalsEstimating how much a measured score would move if you had drawn a different query set of the same size.
- Golden sets and graded relevanceThe labelled query set a retrieval system is scored against, and the grades that define what counts as a correct result.
- LLM-as-judge and blind judgingUsing a language model to grade relevance or faithfulness against a published rubric — and keeping it blind to the answers you want.
- NDCG and rank-aware scoringA metric that rewards putting the most relevant passages highest, not merely including them somewhere.
- Recall@k and retrieval metricsWhether the passages needed to answer a query made it into the top k results.
Generation
How an answer is produced and checked against its sources.
- Citation resolution and quote verificationChecking that every citation points at a real passage and that every quoted span actually appears in it.
- Constrained decoding and structured outputForcing a model's output to match a schema during generation by masking invalid tokens, rather than validating and retrying afterwards.
- Model pinning and reproducibilityRecording the exact dated model snapshot used, rather than a floating alias that silently changes.
Engineering
How the site itself is built and why it works without a backend.
- Hydration and HTML nestingThe step where server-rendered HTML is adopted by client JavaScript — and the invalid nesting that silently breaks it.
- Static export and precomputed interactionShipping a site as plain files with no server, and making it interactive by committing captured state instead of querying a backend.
Cross-reference index
What connects to what, across metrics, failure modes and concepts. Every relationship here is one the content already declares — nothing is inferred, and a reference that does not resolve fails the build.
Metrics
- Abstention (correct)
- ANN post-filtering under Row-Level SecurityThe judge sees the drafter's labelsFaithfulnessRecall@10
- Answer correctness
- The judge sees the drafter's labelsUnpinned model referenceAnswer relevanceFaithfulness
- Answer relevance
- The judge sees the drafter's labelsUnpinned model referenceAnswer correctness
- Citation accuracy
- MAX_TOKENS truncation parsed as a recordThe judge sees the drafter's labelsCitation coverageFaithfulness
- Citation coverage
- MAX_TOKENS truncation parsed as a recordCitation accuracyFaithfulness
- Context precision
- The judge sees the drafter's labelsANN post-filtering under Row-Level SecurityRecall@10Faithfulness
- Cost per query
- Unpinned model referencep95 latency
- Faithfulness
- The judge sees the drafter's labelsUnpinned model referenceMAX_TOKENS truncation parsed as a recordCitation accuracyCitation coverageAnswer correctness
- MRR@10
- ANN post-filtering under Row-Level SecurityThe planner flipRecall@10NDCG@10
- NDCG@10
- ANN post-filtering under Row-Level SecurityThe judge sees the drafter's labelsThe planner flipRecall@10MRR@10
- p95 latency
- The planner flipCost per query
- Recall@10
- ANN post-filtering under Row-Level SecurityThe planner flipThe judge sees the drafter's labelsRecall@5NDCG@10MRR@10Context precision
- Recall@5
- ANN post-filtering under Row-Level SecurityThe planner flipRecall@10NDCG@10
Failure modes
- ANN post-filtering under Row-Level Security
- Recall@10Abstention (correct)Context precisionMRR@10NDCG@10Recall@5ANN post-filtering under access control
- Element identity reuse during parsing
- no cross-references
- MAX_TOKENS truncation parsed as a record
- Citation accuracyCitation coverageFaithfulness
- Superuser bypasses Row-Level Security
- no cross-references
- Table owner bypasses Row-Level Security
- no cross-references
- The judge sees the drafter's labels
- Recall@10Abstention (correct)Answer correctnessAnswer relevanceCitation accuracyContext precisionFaithfulnessNDCG@10
- The planner flip
- Recall@10p95 latencyMRR@10NDCG@10Recall@5
- Truncated identifier collision
- no cross-references
- Unpinned model reference
- Answer correctnessAnswer relevanceCost per queryFaithfulness
Concepts
- Abstention and knowing when not to answer
- ANN post-filtering under access controlCitation resolution and quote verificationRecall@k and retrieval metrics
- ANN post-filtering under access control
- PostgreSQL Row-Level SecurityPartitioning by tenantHNSW and approximate nearest neighbour searchAbstention and knowing when not to answerRecall@k and retrieval metricsANN post-filtering under Row-Level Security
- Bootstrap confidence intervals
- Recall@k and retrieval metricsNDCG and rank-aware scoringGolden sets and graded relevance
- Chunking strategies
- Embeddings and dense retrievalProvenance and reproducibilityRetrieval-Augmented Generation
- Citation resolution and quote verification
- Constrained decoding and structured outputAbstention and knowing when not to answerRetrieval-Augmented Generation
- Constrained decoding and structured output
- Citation resolution and quote verificationModel pinning and reproducibilityRetrieval-Augmented Generation
- Embeddings and dense retrieval
- BM25 and lexical searchHNSW and approximate nearest neighbour searchReciprocal Rank FusionHybrid retrievalChunking strategies
- Golden sets and graded relevance
- LLM-as-judge and blind judgingRecall@k and retrieval metricsNDCG and rank-aware scoringAbstention and knowing when not to answer
- HNSW and approximate nearest neighbour search
- Embeddings and dense retrievalANN post-filtering under access controlPartitioning by tenantPostgreSQL Row-Level Security
- Hybrid retrieval
- Reciprocal Rank FusionCross-encoder rerankingBM25 and lexical searchEmbeddings and dense retrieval
- Model pinning and reproducibility
- LLM-as-judge and blind judgingConstrained decoding and structured output
- Multi-tenancy and tenant assignment
- PostgreSQL Row-Level SecurityPartitioning by tenantANN post-filtering under access control
- NDCG and rank-aware scoring
- Recall@k and retrieval metricsGolden sets and graded relevanceCross-encoder reranking
- Partitioning by tenant
- PostgreSQL Row-Level SecurityANN post-filtering under access controlMulti-tenancy and tenant assignmentHNSW and approximate nearest neighbour search
- PostgreSQL Row-Level Security
- ANN post-filtering under access controlPartitioning by tenantMulti-tenancy and tenant assignment
- Recall@k and retrieval metrics
- NDCG and rank-aware scoringGolden sets and graded relevanceBootstrap confidence intervalsANN post-filtering under access control
- Reciprocal Rank Fusion
- Hybrid retrievalCross-encoder rerankingBM25 and lexical searchEmbeddings and dense retrieval