Hybrid Search: Combining BM25 and Vector Retrieval for RAG
Vector search alone is not enough for enterprise RAG, and the gap shows up predictably: a user searches for part number SL-4471-B or error code E-2203, and the embedding model, trained to capture meaning rather than exact tokens, returns semantically related chunks that never contain that exact string. Hybrid search fixes this by running a lexical keyword search like BM25 alongside vector search and combining the two ranked result lists into one, typically with reciprocal rank fusion. This is not a niche technique anymore; by 2026 hybrid retrieval with fusion is close to the default architecture for any enterprise RAG system that has to handle exact identifiers, acronyms, and domain jargon alongside natural-language questions.
Why Vector-Only Retrieval Fails on Exact Terms
Dense embeddings compress a chunk of text into a few hundred numbers that capture semantic meaning, which is exactly the property that makes them blur exact identifiers together. Two part numbers that differ by one character, SL-4471-B and SL-4471-C, can embed to nearly identical vectors because their surrounding context is nearly identical, so a vector search for one can easily surface a chunk about the other. The same failure hits acronyms, SKUs, order numbers, and error codes, the exact vocabulary that manufacturing and ERP-adjacent queries are full of. A support engineer searching for a specific fault code needs the chunk that contains that code, not the chunk that is merely about similar faults.
BM25: The Lexical Half of Hybrid Search
BM25 is a decades-old term-frequency and inverse-document-frequency ranking function, and it remains genuinely relevant in 2026 precisely because it is exact: it scores a chunk higher when it contains the query's literal terms, weighted by how rare and how frequent those terms are. It does not understand synonyms or paraphrase, which is its known weakness, but that weakness is exactly what vector search covers. Running BM25 is cheap and fast, available natively in Postgres full-text search, Elasticsearch, OpenSearch, and as a built-in option in most dedicated vector databases, so adding it alongside an existing vector index is rarely the expensive part of a hybrid retrieval build.
Reciprocal Rank Fusion: Combining Two Ranked Lists
The hard part of hybrid search is not running two retrievers, it is combining their results, because a BM25 score and a cosine similarity score are not on comparable scales and cannot simply be averaged or weighted without constant retuning. Reciprocal rank fusion sidesteps the problem entirely by ignoring the raw scores and using only each result's rank position: a chunk's fused score is the sum of 1 divided by (a constant k plus its rank) across both result lists, with k typically set to 60. A chunk that ranks highly in either list, or moderately in both, rises to the top, and because the formula only needs rank order, it works identically regardless of what scoring function either retriever used underneath.
- RRF uses rank position only, so it never needs recalibration when either retriever's scoring changes
- A typical k value of 60 dampens the influence of any single very high rank from one retriever
- A chunk that both retrievers agree on, even moderately, consistently outranks a chunk only one flagged highly
Implementation Patterns and What They Cost
The simplest production pattern pairs Postgres full-text search with pgvector in the same database, running both queries and fusing results in application code, which adds no new infrastructure for teams already on Postgres. Elasticsearch or OpenSearch with a vector plugin is a common choice for teams that already run one for logging or search and want BM25 and vector search unified in a single system. Qdrant and Weaviate both offer native hybrid search as a first-class feature, which shortens implementation time at the cost of committing to that vector database. Expect hybrid retrieval to roughly double query-time latency compared to vector search alone, since you are running two searches and a fusion step, though the accuracy gain on exact-match-heavy corpora is almost always worth it.
How Netray Implements Hybrid Retrieval
Netray defaults every DataRay deployment to hybrid retrieval with reciprocal rank fusion rather than treating it as an upgrade path, because the exact-match failure mode is common enough in manufacturing and ERP data that skipping it almost guarantees a support ticket in the first month. We tune the BM25 and vector fusion balance against each client's own golden evaluation set rather than shipping a fixed default, since a corpus dense with part numbers benefits from lexical search more than a corpus of narrative reports does. The result is measured directly in recall@k before and after hybrid is enabled, so the improvement is a number, not an assumption.
Frequently Asked Questions
What is reciprocal rank fusion in hybrid search?
Reciprocal rank fusion combines two ranked result lists, typically from a lexical search and a vector search, by scoring each result using only its rank position rather than its raw score. A result's fused score is the sum of 1 divided by a constant (commonly 60) plus its rank across both lists. This avoids the problem of combining scores from two systems with incomparable scales, and it consistently favors results both retrievers agree on.
Is hybrid search always better than vector-only retrieval?
For enterprise corpora containing exact identifiers, part numbers, error codes, or acronyms, yes, almost always. Vector-only retrieval blurs exact terms together in embedding space, which hybrid search corrects with a lexical BM25 pass. For narrative-only corpora with no exact-match vocabulary, the improvement is smaller, but the added latency cost is low enough that hybrid is a reasonable default in most production systems.
Why does vector search struggle with part numbers and error codes?
Dense embeddings compress text into vectors that capture semantic meaning, which causes visually or contextually similar identifiers, like two part numbers differing by one character, to embed nearly identically. A query for one exact identifier can surface a chunk about a different but related identifier instead. BM25 lexical search does not have this problem because it matches literal terms, which is why pairing it with vector search closes the gap.
How much latency does hybrid search add to a RAG query?
Expect roughly double the retrieval-stage latency compared to vector search alone, since hybrid search runs both a lexical and a vector query and then fuses the results. In absolute terms this is usually tens of milliseconds, not a user-perceptible delay, especially since the generation stage that follows typically takes far longer than the retrieval and fusion steps combined.
Key Takeaways
- 1Why Vector-Only Retrieval Fails on Exact Terms: Dense embeddings compress a chunk of text into a few hundred numbers that capture semantic meaning, which is exactly the property that makes them blur exact identifiers together. Two part numbers that differ by one character, SL-4471-B and SL-4471-C, can embed to nearly identical vectors because their surrounding context is nearly identical, so a vector search for one can easily surface a chunk about the other.
- 2BM25: The Lexical Half of Hybrid Search: BM25 is a decades-old term-frequency and inverse-document-frequency ranking function, and it remains genuinely relevant in 2026 precisely because it is exact: it scores a chunk higher when it contains the query's literal terms, weighted by how rare and how frequent those terms are. It does not understand synonyms or paraphrase, which is its known weakness, but that weakness is exactly what vector search covers.
- 3Reciprocal Rank Fusion: Combining Two Ranked Lists: The hard part of hybrid search is not running two retrievers, it is combining their results, because a BM25 score and a cosine similarity score are not on comparable scales and cannot simply be averaged or weighted without constant retuning. Reciprocal rank fusion sidesteps the problem entirely by ignoring the raw scores and using only each result's rank position: a chunk's fused score is the sum of 1 divided by (a constant k plus its rank) across both result lists, with k typically set to 60.
Put this into numbers
Free interactive tools for exactly this problem. No signup to use them.
Hybrid Search Cost Calculator
Estimate the monthly infrastructure cost of running vector search and keyword search together, including an optional reranker, based on your node counts and query volume.
Free ToolPrivate RAG Corpus Sizing Calculator
Estimate chunk counts, vector index storage, raw text volume, and embedding compute time before you build a private retrieval system over your document estate.
Free ToolRAG Chunking Strategy Calculator
Turn corpus size, chunk length, and overlap into a concrete chunk count, embedding cost, and vector storage footprint before you build the ingestion pipeline.
Terms used in this article
Seeing your RAG system miss exact part numbers or error codes? Netray will add hybrid retrieval with fusion and show you the recall improvement against your own queries.
Related Resources
Enterprise RAG Architecture: The Full 2026 Blueprint
A practitioner's blueprint for enterprise RAG in 2026: ingestion, chunking, embedding, retrieval, rerank, generation, and the eval loop that keeps it honest.
AI & Automationpgvector vs Dedicated Vector Databases: An Honest Comparison
pgvector vs Milvus, Qdrant, and Weaviate for enterprise RAG: real tradeoffs on scale, latency, operational overhead, and when Postgres is genuinely enough.
AI & AutomationRAG Evaluation Metrics: Recall@k, MRR, Faithfulness, and More
The RAG evaluation metrics that matter: recall@k and MRR for retrieval, faithfulness and answer relevance for generation, and how to build the eval loop.