The retrieval ladder
Most RAG systems are built flat: embed the documents, search the vectors, hand chunks to the model, ship. When the answers disappoint, the usual response is to swap models or write longer prompts. The better response is to see retrieval as a ladder of four rungs, each one fixing a specific blindness of the rung below, and to climb it with measurements instead of vibes.
Measure first or you are decorating
Before touching the stack, build the yardstick: twenty five questions that represent real usage, and a written rubric for what counts as relevant context. Then score every configuration against the same questions. The metric that matters at this layer is context precision: of the chunks you retrieved, how many actually belong? Without this, every change is decoration. With it, every rung of the ladder becomes a number you can defend.
This essay is the method, not a benchmark. It carries no chart on purpose: the only retrieval numbers worth publishing are the ones you ran on your own corpus, against your own questions. Build the yardstick first, then the four rungs below turn into numbers you can defend.
Rung one: dense retrieval, and its blindness
Embedding search is the right foundation: it finds meaning, not just words, so a question phrased nothing like the document still lands. Its blindness is the exact term. Product codes, function names, statute numbers, rare identifiers: embeddings smear these into semantic soup precisely because they are unusual. Dense-only systems fail most on queries that a keyword search would have aced, which is a maddening thing to discover in production.
Rung two: hybrid, the blindness patched
Run a keyword scorer alongside the dense search and merge the two rankings. Reciprocal rank fusion is the standard merge: each document scores by the reciprocal of its rank in each list, summed. It needs no tuning and rewards showing up in both lists. Hybrid fixes the exact-term hole while keeping the semantic reach, and it is the single highest-value change most flat RAG systems can make.
Rung three: reranking, precision purchased
The retrievers so far scored the query and each document independently, which is cheap but shallow. A cross-encoder reads the query and a candidate together and produces a much sharper judgment, at a price that only works on a shortlist. So the shape is: cheap search finds fifty, expensive reranker orders the top ten. Two-stage designs like this recur everywhere in retrieval because they let you spend your latency budget where it changes the answer.
Rung four: filters, the unglamorous rung
Most real queries carry constraints that are not semantic at all: this version, this section, this date range. Metadata filtering enforces them structurally instead of hoping the embedding space notices. It is unglamorous and it routinely rescues correctness, with one caveat worth respecting: filtered recall degrades differently than unfiltered recall, so measure them separately.
The takeaway
Climb in order, measure at every rung, and stop when the numbers say you are done rather than when the stack looks impressive. The ladder is boring by design. Boring, measured retrieval is what reliable AI systems stand on.
One level down. The deep version of this lives in the private atlas: 004, the approximate answer. What rung one is standing on: IVF cells, product quantization, HNSW descent, and the recall-latency-cost triangle. Private atlas, not deployed with this site.