Section 7 · Learn · Retrieval

Embeddings and dense retrieval

Text mapped to a vector so that passages with similar meaning land near each other, letting search work by meaning rather than by shared words.

What it is

An embedding model converts a passage into a fixed-length list of numbers — here 384 of them. Similar meanings produce nearby vectors, so retrieval becomes a nearest-neighbour search in that space.

Why it is needed

Users do not use the document's vocabulary. Someone asks about 'ending a contract early' when the document says 'termination for convenience'. Lexical search scores that zero; dense retrieval finds it.

How it works

The model is a small transformer. The vector for the CLS token is taken as the passage representation and normalised to unit length, which makes the dot product of two vectors equal to their cosine similarity — so similarity is one multiply-accumulate rather than a division.

Formula

cos(a,b) = (a . b) / (|a| |b|)

With both vectors L2-normalised, |a| = |b| = 1, so:

cos(a,b) = a . b

And pgvector's cosine distance operator <=> returns 1 - cos(a,b).

a . b: dot product, the sum of elementwise products
384: the dimensionality bge-small produces
<=>: pgvector's cosine distance operator; smaller is more similar

In this project

bge-small-en-v1.5 embeds all 51,310 chunks in about 1.5 minutes on an RTX 5060 at 558 chunks/second, producing a 79 MB float32 matrix. Verified after generation: all L2 norms within 0.0006 of 1.0, no NaN rows, no zero rows. The embeddings are not committed — they are reproducible from the committed corpus manifest.

Trade-offs

Advantages

Finds paraphrase and synonymy
One vector per passage regardless of length
Similarity is cheap once vectors exist

Limitations

Rare identifiers can be washed out by surrounding prose
Embedding quality is domain-dependent
Recomputing the whole corpus is required when the model changes

Common mistakes

Forgetting to normalise, then using dot product as if it were cosine
Mixing vectors from two different models in one index
Assuming a high cosine means relevance rather than surface similarity

When to use it

Reach for it when

Users phrase questions in their own words
As one half of a hybrid retriever

Avoid it when

Alone, when exact identifiers must match

How it interacts with the rest of the system

HNSW and approximate nearest neighbour searchReciprocal Rank FusionHybrid retrievalChunking strategies

← All topics