Skip to content
derpx06Notes on systems, models & learning
5. Retrieval & RAG Patterns · lesson 45 of 68 · 1 min · January 10, 2026

Post-Retrieval Re-Ranking

The Cross-Encoder. Fixing the mistakes of the vector store.

Standard Vector Search uses Bi-Encoders. It encodes the Query independently. It encodes the Document independently. Then it measures the angle. Pros: Fast (millions of docs). Cons: Lossy. It misses subtle interactions between query and doc.

A Cross-Encoder takes both the query and the document as a single input: Model(Query, Doc). It outputs a score (0 to 1) of "How relevant is this?" Pros: Extremely accurate. Cons: Slow. Expensive. You cannot index this ahead of time.

We combine them.

  1. Stage 1 (Retrieval): Use Vector Search (Bi-Encoder) to get the top 50 candidates. (Fast, coarse).
  2. Stage 2 (Re-Ranking): Use a Cross-Encoder (like BGE-Reranker or Cohere Rerank) to score those 50.
  3. Result: Take the top 5 from the Cross-Encoder.

"Apple" means fruit. "Apple" means company. A Vector store puts them close. A Cross-Encoder sees the full sentence "I want to buy a Granny Smith Apple" and pushes the "Company" documents to the bottom.

Re-ranking is the single most effective way to boost precision in a RAG pipeline. It acts as a "Quality Filter" before the expensive LLM generation step.

Key Intuition: "Index fast. Rank slow."