> ## Documentation Index
> Fetch the complete documentation index at: https://docs.truvec.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Hybrid search and reranking

> The two strategies you can add around an embedding model, what they change, and what they cost.

The embedding model isn't the only part of retrieval you can change. Two strategies are common in production RAG systems, and Truvec lets you test both on your own questions: **hybrid search**, which adds keyword search to vector search, and **reranking**, which re-scores the best candidates with a second model.

Together with the embedding model, they form the **retrieval pipeline** of a run. A model with its pipeline is called a **configuration**, for example `text-embedding-3-small - hybrid - rerank cohere/rerank-v3.5`.

```mermaid theme={null}
flowchart LR
    Q["Question"] --> D["Vector search<br/>(embedding model)"]
    Q -.->|hybrid| K["Keyword search<br/>(BM25)"]
    D --> F["Fusion<br/>(hybrid only)"]
    K -.-> F
    F --> C["First-stage candidates"]
    C -.->|reranker| R["Reranker re-scores<br/>the candidates"]
    C --> T["Top k sent to the LLM"]
    R -.-> T
```

## Dense search (the default)

Every run starts with **dense search**: the question is embedded and the vector database returns the chunks whose vectors are closest. This is what [How RAG retrieval works](https://docs.truvec.dev/concepts/rag-and-embeddings.md) describes, and it's what a run does when you leave the pipeline settings untouched.

Dense search is good at meaning ("I forgot my login" finds the password-reset article) and weaker at exact strings: product names, error codes, version numbers, part references. To an embedding model, `ERR-4012` and `ERR-4021` look almost the same.

## Hybrid search

**Hybrid search** runs a keyword search next to the vector search and merges the two rankings.

* The keyword search is **BM25**, the classic ranking used by search engines: chunks that contain the question's rarer words, more often, rank higher. Compound terms such as `ERR-404`, `v2.1.3` or `gpt-4o` are indexed whole and as their parts, so they match exactly.
* The two rankings are merged with **reciprocal rank fusion (RRF)**: each chunk gets points for its rank in each list, so a chunk that is high in both ends up first. Fusion uses ranks rather than scores, because cosine similarities and BM25 scores aren't on the same scale.
* The **Keyword / Dense** slider sets the weight of each ranking. 50% gives them equal say; move it toward **Dense** if your questions are mostly paraphrases, toward **Keyword** if they quote exact terms.

Hybrid search costs nothing extra: the keyword index is built by Truvec and doesn't call any provider. It adds a few milliseconds of search time.

> [!TIP]
>
> Hybrid search tends to help when users type identifiers, product or feature names, legal references or code symbols. It can hurt on conversational questions, where a frequent word pulls in the wrong chunks. That's why it's worth measuring rather than switching on by default.

## Reranking

A **reranker** is a second model that reads the question and each candidate chunk **together** and scores how well the chunk answers it. Embedding models compare two vectors computed separately; a reranker (a *cross-encoder*) sees both texts at once, so it judges relevance more precisely, at the price of one call per question.

How it works in Truvec:

1. The first stage (dense or hybrid search) returns a pool of **first-stage candidates**: 30 by default, from 1 to 100, and never fewer than your top k.
2. The reranker scores every candidate.
3. The top k by reranker score become the result.

A reranker can only reorder what the first stage found. If the right chunk isn't among the candidates, no reranker can bring it back: raise the number of candidates, or improve the first stage.

### Available rerankers

| Reranker | Billed | List price |
| - | - | - |
| **Cohere Rerank 3.5** (`cohere/rerank-v3.5`) | Per search: one question with up to 100 candidates | \$2.00 per 1,000 searches |
| **Voyage rerank-2.5** (`voyage/rerank-2.5`) | Per token: the question once per candidate, plus every candidate | \$0.05 per 1M tokens |
| **Voyage rerank-2.5-lite** (`voyage/rerank-2.5-lite`) | Per token, like rerank-2.5 | \$0.02 per 1M tokens |

Rerankers are included in every plan, like the embedding models: no provider API key needed.

### What a reranker costs

A reranker usually costs more per query than embedding the question. With Cohere, every question is one search, so 1,000 queries cost \$2.00 whatever the length of your chunks. With Voyage, the cost grows with the number of candidates and their length: 30 candidates of about 250 tokens is roughly 8,000 to 10,000 tokens per question, or about \$0.40 to \$0.50 per 1,000 queries with rerank-2.5.

For comparison, embedding 1,000 short questions with `text-embedding-3-small` costs well under a cent. The **cost per 1,000 queries** of a run includes the reranker, so the trade-off is visible in the results. See [Cost, speed and storage](https://docs.truvec.dev/concepts/cost-and-performance.md).

> [!NOTE]
>
> Cohere counts a candidate longer than about 500 tokens (question included) as several documents, and a search over more than 100 documents as several searches. Truvec's estimate assumes one search per question. It's accurate at the default 30 candidates and 1,000-character chunks, and an underestimate with close to 100 candidates and very long chunks.

Reranking also adds latency: one API round trip per question, typically a few hundred milliseconds. The results show it separately (**rerank** time and its p95) and in the total time per query.

## Reading the results of a pipeline

In the [Retrieval Inspector](https://docs.truvec.dev/guides/run-evaluation.md#the-retrieval-inspector), the score next to each retrieved chunk depends on the pipeline:

| Pipeline | Score shown |
| - | - |
| Dense | **Similarity**: the cosine similarity of the chunk and the question |
| Hybrid | **Fused rank score**: the RRF score. Each chunk also shows its rank in the dense and in the keyword ranking |
| With a reranker | **Reranker relevance**: the reranker's score. Each chunk also shows its rank before reranking (`was #12 before rerank`) |

The movement is the most useful part: it tells you whether the right chunk was rescued by the keyword ranking, or pulled up from deep in the candidates by the reranker.

## And chunking?

Chunking isn't part of the pipeline, because it changes the chunks themselves. A golden set points at specific chunks, so a set built on one chunking can't grade another. To compare chunkings, upload the documents with each setting and build a golden set on each. See [Comparing chunking strategies](https://docs.truvec.dev/concepts/chunking.md#comparing-chunking-strategies).

## When to try what

| What you see in the results | What to try |
| - | - |
| The right chunk is retrieved but rarely at rank 1 (low hit rate @1, good hit rate @k) | A **reranker** |
| Misses on questions with names, codes or identifiers | **Hybrid search** |
| The right chunk is missing from the top k entirely, for many questions | A **better embedding model**, more candidates, or another **chunking** |
| A cheap model is close to an expensive one | The cheap model **with a reranker**, against the expensive one without |

To find out what each change is worth, use a [strategy comparison](https://docs.truvec.dev/guides/compare-models.md#compare-strategies): one baseline, one change per configuration.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.