No. You need documents and an idea of the questions your users ask. Truvec handles the embedding, search and scoring. The concepts pages explain everything you need in plain language.
No. Truvec evaluates retrieval: whether the right passages are found and ranked high. It’s the foundation of answer quality. If the right passages aren’t retrieved, no LLM can answer correctly. Evaluating the generated answers themselves is a separate step.
OpenAI (text-embedding-3-small, 3-large, ada-002), Cohere (Embed v4, English v3, Multilingual v3), Voyage AI (voyage-4-large, voyage-4, voyage-4-lite), Google (Gemini Embedding 2 and 001), and open-source models: BGE, Nomic Embed, mxbai, GTE and Qwen3 Embedding. See Model pricing.
A representative sample is enough: 5 to 50 documents for a first evaluation. What matters is that they cover the topics and document types your users ask about.
20 to 30 for a first look, 50 to 100 for confident decisions between close models. See Golden datasets.
Nothing beyond your plan. Truvec runs the embedding models and the golden dataset generation on its own accounts, so you don’t need provider API keys or pay providers separately. Each run uses one of your plan’s evaluation runs (a comparison of several models counts as one) and some of its embedding volume. The first run with a model embeds all your chunks; later runs reuse those embeddings and only embed the questions. A run that fails doesn’t count against your runs.The costs shown in your results (index cost, cost per 1,000 queries, Billed by this run) are what the same usage would cost you with the provider in production, to help you compare models. See Model pricing.
Only what’s needed to run the models you choose. The text of your chunks and questions is sent to the provider of each embedding model you evaluate: OpenAI, Cohere, Voyage AI, Google (Gemini API) or Fireworks AI, which hosts the open-source models. To generate a golden dataset, chunk text is sent to OpenAI. Truvec uses these providers’ paid APIs, whose terms restrict how they may use the data, and doesn’t use your content to train models.Your chunks, golden datasets and results are stored on Truvec’s servers in the EU and deleted with your projects. To delete your account and all its data, email support@truvec.dev. See the Privacy Policy.
Quality metrics are stable for the same documents, questions and model: chunk embeddings are reused between runs, so differences are rare and tiny. If scores changed noticeably, the documents or the dataset likely changed (look for the Corpus changed badge). Latency varies from run to run with network conditions and provider load.
Yes. Upload them as pre-chunked data and Truvec evaluates exactly those chunks.
Yes, with one project per strategy, because golden datasets point to specific chunks. See Comparing chunking strategies.
No. Committing locks it so that results stay comparable. Create a new dataset instead. The old one keeps your earlier results meaningful.
Yes. Everything you do inside a project (documents, golden datasets, evaluations, comparisons and the playground) is available through the REST API. Billing and team management are done in the interface.