Run an evaluation
1
Open Evaluation
2
Select a golden dataset
Only committed datasets are listed. See Build a golden dataset.
3
Select an embedding model
Each model shows its provider, number of dimensions and price per million tokens.
4
Set the retrieval pipeline (optional)
Under Retrieval pipeline, keep Dense and No reranker for plain vector search, or:
- choose Hybrid (BM25 + dense) to fuse keyword search with the vector search, and set the Keyword / Dense weight,
- choose a Reranker to re-score the first-stage candidates, and set how many First-stage candidates it reads (30 by default).
5
Set Top K
The number of chunks retrieved per question, from 1 to 50 (5 by default). Match what your RAG system sends to the LLM.
6
Click Run Evaluation
The evaluation runs in the background and appears in Evaluation History with a Running status, which updates automatically.
The first evaluation of a model in a project embeds all your chunks, which takes longer and costs more. Later evaluations with the same model reuse those embeddings and only embed the questions.
Evaluation history
Every evaluation of the project is listed, newest first, with its configuration (for exampletext-embedding-3-small - hybrid - rerank cohere/rerank-v3.5), dataset, top k, status, and for completed runs the hit rate, MRR and cost. Click an evaluation to open its results below the list. Delete an evaluation with the trash icon.
A Corpus changed badge means documents were added or removed after the evaluation ran, so its results may not reflect the current documents.
If an evaluation Failed, open it to read the error. See Troubleshooting for common causes.
The Retrieval Inspector
The results of a completed evaluation open in the Retrieval Inspector.Metric cards
Hit rate, recall, MRR and nDCG at your top k, latency per query, and cost per 1,000 queries. With a reranker, latency is the whole query (embedding, search and reranking) and the hint shows the reranking time; the cost includes the reranker. See Metrics and Cost, speed and storage.Charts and tables
- Hit rate by cut-off: how quickly the right chunk shows up in the ranking.
- Rank of the first relevant chunk: how many questions were answered at rank 1, rank 2, and so on, and how many were missed.
- Cost & footprint: price, index cost, cost per 1,000 queries, what this run billed, corpus size in tokens, vector storage and dimensions.
- Separation & latency: score gap between the right chunks and the best wrong one, and embedding and search times.
Question by question
This is where you learn the most. Use the All questions / Hits / Misses filter, the question list and the arrows to move through questions. For each one you see:- the question and its target chunks,
- the retrieved chunks in rank order, marked Ground truth or Distractor, with their score: the similarity for dense search, the fused rank score for hybrid search (with the chunk’s dense and keyword ranks), or the reranker relevance with a reranker (with the chunk’s rank before reranking),
- Expected but not retrieved: target chunks that didn’t make the top k,
- whether the ground truth was found, and at which rank.