Section 7 · Learn · Retrieval
Cross-encoder reranking
A second-stage model that reads the query and a passage together and scores their relevance directly, reordering the shortlist retrieval produced.
A second-stage model that reads the query and a passage together and scores their relevance directly, reordering the shortlist retrieval produced.
What it is
A bi-encoder embeds query and passage separately and compares vectors. A cross-encoder feeds both into one model at once, so every query token can attend to every passage token.
Why it is needed
Separate encoding forces each passage into a single vector before it knows what was asked. A cross-encoder decides relevance with the question in hand, which is strictly more informative and measurably more accurate.
How it works
Too expensive to run over a whole corpus — it is one forward pass per query-passage pair. So it runs only over the shortlist retrieval already produced, typically the top 50, and reorders it.
Flow
- 151,310 chunks
- 2retrieve top 50
- 3cross-encode 50 pairs
- 4reordered top 10
Cost grows with shortlist depth, not corpus size. That is what makes the stage affordable.
In this project
bge-reranker-base runs over the top 50 fused candidates. The walkthrough shows movement per result, so the reader can see how much reordering the reranker actually does and judge whether it earned its latency.
Trade-offs
Advantages
- Substantially more accurate than vector similarity
- Needs no index of its own
- Drops in without changing retrieval
Limitations
- One model forward pass per candidate
- Latency scales with shortlist depth
- Cannot recover a passage retrieval never returned
Common mistakes
- Reranking a shortlist so shallow the right passage was never in it
- Ignoring the latency it adds to p95
When to use it
Reach for it when
- Precision at the top of the list matters
Avoid it when
- Latency budgets that cannot absorb a second model
How it interacts with the rest of the system
Reciprocal Rank FusionHybrid retrieval