text-embedding-3-small - hybrid - rerank cohere/rerank-v3.5.
Dense search (the default)
Every run starts with dense search: the question is embedded and the vector database returns the chunks whose vectors are closest. This is what How RAG retrieval works describes, and it’s what a run does when you leave the pipeline settings untouched. Dense search is good at meaning (“I forgot my login” finds the password-reset article) and weaker at exact strings: product names, error codes, version numbers, part references. To an embedding model,ERR-4012 and ERR-4021 look almost the same.
Hybrid search
Hybrid search runs a keyword search next to the vector search and merges the two rankings.- The keyword search is BM25, the classic ranking used by search engines: chunks that contain the question’s rarer words, more often, rank higher. Compound terms such as
ERR-404,v2.1.3orgpt-4oare indexed whole and as their parts, so they match exactly. - The two rankings are merged with reciprocal rank fusion (RRF): each chunk gets points for its rank in each list, so a chunk that is high in both ends up first. Fusion uses ranks rather than scores, because cosine similarities and BM25 scores aren’t on the same scale.
- The Keyword / Dense slider sets the weight of each ranking. 50% gives them equal say; move it toward Dense if your questions are mostly paraphrases, toward Keyword if they quote exact terms.
Reranking
A reranker is a second model that reads the question and each candidate chunk together and scores how well the chunk answers it. Embedding models compare two vectors computed separately; a reranker (a cross-encoder) sees both texts at once, so it judges relevance more precisely, at the price of one call per question. How it works in Truvec:- The first stage (dense or hybrid search) returns a pool of first-stage candidates: 30 by default, from 1 to 100, and never fewer than your top k.
- The reranker scores every candidate.
- The top k by reranker score become the result.
Available rerankers
Rerankers are included in every plan, like the embedding models: no provider API key needed.
What a reranker costs
A reranker usually costs more per query than embedding the question. With Cohere, every question is one search, so 1,000 queries cost $2.00 whatever the length of your chunks. With Voyage, the cost grows with the number of candidates and their length: 30 candidates of about 250 tokens is roughly 8,000 to 10,000 tokens per question, or about $0.40 to $0.50 per 1,000 queries with rerank-2.5. For comparison, embedding 1,000 short questions withtext-embedding-3-small costs well under a cent. The cost per 1,000 queries of a run includes the reranker, so the trade-off is visible in the results. See Cost, speed and storage.
Cohere counts a candidate longer than about 500 tokens (question included) as several documents, and a search over more than 100 documents as several searches. Truvec’s estimate assumes one search per question. It’s accurate at the default 30 candidates and 1,000-character chunks, and an underestimate with close to 100 candidates and very long chunks.
Reading the results of a pipeline
In the Retrieval Inspector, the score next to each retrieved chunk depends on the pipeline:
The movement is the most useful part: it tells you whether the right chunk was rescued by the keyword ranking, or pulled up from deep in the candidates by the reranker.
And chunking?
Chunking isn’t part of the pipeline, because it changes the chunks themselves. A golden set points at specific chunks, so a set built on one chunking can’t grade another. To compare chunkings, upload the documents with each setting and build a golden set on each. See Comparing chunking strategies.When to try what
To find out what each change is worth, use a strategy comparison: one baseline, one change per configuration.