architecture
Hybrid Retrieval
Combining BM25 keyword search with dense vector similarity to surface documents that match both the vocabulary and the meaning of a query — the retrieval architecture at the core of my rag-engine-rs project.
Why Neither Alone Is Enough
BM25 (keyword search) excels at exact term matches and rare proper nouns. It fails when the user paraphrases or uses synonyms.
Dense vector search (semantic similarity) excels at paraphrase and concept matching. It can miss documents containing a critical exact term that the query also uses literally.
Hybrid retrieval runs both in parallel and merges the results, capturing the strengths of each.
The Two-Stage Pattern
Query
├─ BM25 → top-K keyword candidates
└─ pgvector ANN → top-K semantic candidates
↓
Reciprocal Rank Fusion (or weighted score merge)
↓
Re-ranker (cross-encoder or score threshold)
↓
Final top-N documents → LLM context window
Reciprocal Rank Fusion (RRF): combines ranked lists without needing score normalization.
Grounded in rag-engine-rs
In my rag-engine-rs project, documents are chunked, embedded, and stored in pgvector. At query time, the system runs a BM25 pass over a Postgres full-text index and a pgvector ANN pass in parallel, then merges with RRF before streaming the grounded answer to the client. The two-stage design means the re-ranker only sees a small merged candidate set, keeping latency low.
When Not to Use Hybrid Retrieval
Hybrid retrieval doubles the moving parts: two indexes to build and keep in sync, two queries per request, and a fusion step to tune. That cost is only justified when your queries genuinely span both modes. A corpus queried almost entirely by exact identifiers (error codes, SKUs) may do fine on BM25 alone; a corpus of conceptual prose queried in natural language may do fine on vectors alone. Measure recall on real queries before adding the second index — the simplest retrieval that hits your recall target is the right one, and hybrid is a deliberate step up, not a default.