Chunking strategies

What it is

A 200-page filing cannot be one vector. It is split into passages — here targeting 512 tokens — each embedded and indexed independently.

Why it is needed

Chunking decides what is findable. A fact split across two chunks may be retrievable from neither. This is a real design decision, not a parameter.

How it works

Fixed-token windows slide through the token stream with overlap so a fact near a boundary appears whole in at least one chunk. Section-aware packing instead respects document structure and never lets a chunk span two sections.

At a glance

In this project

Both strategies are implemented behind one interface and run over the same corpus: 51,310 fixed-512 chunks and 59,579 section-aware. Page and section provenance is preserved through both, which is what lets a citation resolve to a location a reader can check.

A correction to a common claim

Section-aware chunking is often described as avoiding cuts mid-sentence. Measured here, it does not: it splits long sections internally using the same overlap, so it starts mid-sentence about as often as fixed windows do. What it actually guarantees is that no chunk spans two sections. Measured over one document per source, fixed-512 produced 4, 17 and 32 chunks crossing a section boundary; section-aware produced zero in every case, by construction.

Why the overlap exists

15% overlap means roughly 77 tokens repeat between consecutive chunks. Without it, a sentence spanning a boundary is truncated in both chunks and retrievable from neither. The cost is index size and some duplicate results.

Trade-offs

Common mistakes

• Chunking before parsing structure, losing page and section provenance
• Assuming section-aware prevents mid-sentence starts

When to use it

How it interacts with the rest of the system

Embeddings and dense retrievalProvenance and reproducibilityRetrieval-Augmented Generation