API introduction

Everything you do inside a project in the Truvec interface is available through a REST API: documents, golden datasets, evaluations, comparisons and the playground. Billing and team management stay in the interface. Use it to automate evaluations, for example re-running a comparison every time your documents change, or to pull results into your own dashboards.

Base URL

https://app.truvec.dev

Every endpoint path starts with /api, for example https://app.truvec.dev/api/v1/projects. API calls use the same plan and allowances as the interface.

Authentication

Requests are authenticated with a bearer token.

Use your Truvec account

Sign in with the email and password of your Truvec account. A teammate’s account works too: API calls act on the projects of the account’s workspace.

Get a token

The login endpoint takes a form with your email as username:

curl -X POST https://app.truvec.dev/api/auth/login \
  -d "username=you@example.com" \
  -d "password=a-strong-password"
# {"access_token": "eyJhbGciOi...", "token_type": "bearer"}

Send it with every request

curl https://app.truvec.dev/api/v1/projects \
  -H "Authorization: Bearer $TOKEN"

Tokens expire after 2 hours. Request a new one when you get a 401 response.

End-to-end example

This script runs the whole workflow: create a project, upload a document, generate and commit a golden dataset, compare two models and print the ranking.

evaluate.sh

API=https://app.truvec.dev
TOKEN=$(curl -s -X POST $API/api/auth/login -d "username=you@example.com" -d "password=a-strong-password" | jq -r .access_token)
AUTH="Authorization: Bearer $TOKEN"

# 1. Create a project
PROJECT=$(curl -s -X POST $API/api/v1/projects -H "$AUTH" -H "Content-Type: application/json" \
  -d '{"name": "Support assistant"}' | jq .id)

# 2. Upload a document with the default chunking
curl -s -X POST "$API/api/v1/documents/upload?project_id=$PROJECT" -H "$AUTH" \
  -F "file=@help-center.pdf" -F "chunking_strategy=recursive" -F "chunk_size=1000" -F "chunk_overlap=200" > /dev/null

# 3. Generate a golden dataset, then commit it (review it in the interface first!)
DATASET=$(curl -s -X POST $API/api/v1/golden-datasets -H "$AUTH" -H "Content-Type: application/json" \
  -d "{\"project_id\": $PROJECT, \"name\": \"Generated v1\", \"number_of_questions\": 30}" | jq .id)
curl -s -X POST $API/api/v1/golden-datasets/$DATASET/commit -H "$AUTH" > /dev/null

# 4. Compare two models (ids from GET /api/v1/embedding-models)
BENCHMARK=$(curl -s -X POST $API/api/v1/benchmarks -H "$AUTH" -H "Content-Type: application/json" \
  -d "{\"project_id\": $PROJECT, \"golden_dataset_id\": $DATASET, \"name\": \"small vs large\", \"model_ids\": [1, 2], \"top_k\": 5}" | jq .id)

# 5. Wait for the comparison to finish
until [ "$(curl -s $API/api/v1/benchmarks/$BENCHMARK -H "$AUTH" | jq -r .status)" != "running" ] && \
      [ "$(curl -s $API/api/v1/benchmarks/$BENCHMARK -H "$AUTH" | jq -r .status)" != "pending" ]; do sleep 5; done

# 6. Print the ranking
curl -s $API/api/v1/benchmarks/$BENCHMARK -H "$AUTH" \
  | jq '.results.comparison[] | {model: .model_name, mrr, recall, cost_per_1k_queries: .cost.cost_per_1k_queries_usd}'

Committing a generated dataset without reviewing it is fine for a quick test, but not for decisions. See Build a golden dataset.

Key resources

Resource	What it is
Projects	Containers for documents, datasets and results.
Documents	Uploaded files and their chunks. `GET /api/v1/documents/chunking-strategies` lists the chunking options.
Golden datasets	Test questions and their expected chunks. Generated on creation; entries can be edited until the dataset is committed.
Embedding models	The models you can evaluate, with their price per 1M tokens.
Evaluations	One model on one golden dataset. `GET /api/v1/evaluations/{id}` returns the per-question results.
Benchmarks	A comparison of several models; each model’s run is an evaluation linked to the benchmark.
Playground	Ad-hoc queries against a model.

Errors

Errors return a JSON body with a detail field:

Status	Meaning
`400`	The request can’t be processed, for example an unsupported file or a file without text.
`401`	Missing or expired token.
`402`	Your plan doesn’t allow it: no active plan, or a limit or monthly allowance is reached. `detail` says which.
`403`	The resource belongs to another workspace.
`404`	The resource doesn’t exist.
`409`	Conflict. On upload, `detail.code` is `mixed_chunking` when documents of the same type were chunked with other settings; resend with `confirm_mixed_chunking=true` to proceed.
`422`	Invalid parameters; `detail` lists each problem.
`429`	Too many requests. Wait for the number of seconds in the `Retry-After` header.

Overview

projects

documents

golden-datasets

embedding-models

evaluations

benchmarks

reports

playground

auth

API introduction

Base URL

Authentication

End-to-end example

Key resources

Errors

​Base URL

​Authentication

​End-to-end example

​Key resources

​Errors

Base URL

Authentication

End-to-end example

Key resources

Errors