The framework is free. The history is what you pay for.
vizra/evals is MIT licensed and always will be.
Vizra Cloud is for when baselines need to outlive one laptop.
For trying it out on one project.
- ✓ Unlimited eval runs
- ✓ 1 project
- ✓ 2 seats
- ✓ 14 days of per-sample detail
$290 billed yearly
For a team shipping agents together.
- ✓ Unlimited eval runs
- ✓ 5 projects
- ✓ Unlimited seats
- ✓ 90 days of per-sample detail
$990 billed yearly
For agencies running evals across many clients.
- ✓ Everything in Pro
- ✓ Unlimited projects
- ✓ 12 months of per-sample detail
- ✓ Priority support
Model spend is not ours to charge. Running an eval costs tokens at your own provider, on your own key. Vizra never runs the model, so nothing you see here scales with how much you evaluate — the price is the price.
What you get without paying anything
The whole eval engine. This is not a trial or a crippled tier — it is the same code the paid product runs on, and it works forever offline.
- ✓Evals as Pest tests, sampled and scored
- ✓LLM judging with reasoning kept
- ✓Baselines and CI gates
- ✓The local dashboard, in your own app
- ✓Cost and token tracking
What the cloud adds
Everything that stops being true the moment more than one person, or more than one machine, is involved.
- ✓History that survives a laptop, and CI runs that keep theirs
- ✓One baseline the whole team shares
- ✓Run an eval from the browser — no terminal
- ✓Pull request checks on every branch
- ✓Drill into any sample: response, tool calls, judge reasoning
Questions worth asking first
Doesn't Pest 5 already do this? +
Pest's own eval expectations are quick, unrecorded checks — good for a one-off "does this look right" in a test file. toPassEval() is for everything you want kept: datasets, sampling, real tool-call assertions, multi-turn conversations, and a result that persists so today's run can be compared against yesterday's. The two coexist in the same file; reach for Pest's when you want a fast local check, and toPassEval() when the answer needs a history.
Does Vizra run my agent? +
No. Your evals run in your own application, against your own database and your own model keys. The cloud stores results and, when you click Run, queues a request your app collects on its next check. We hold no code, no credentials and no model keys.
What actually leaves my network? +
Scores, timings, cost and git metadata always. Per-sample detail — the prompt, the response, tool calls and judge reasoning — is what powers the drill-down and is sent by default. Set VIZRA_CLOUD_SAMPLES=false and you keep everything except the ability to click into a row.
What counts as a project? +
One codebase. Most teams need one; an agency running evals for several clients needs several, which is what the Agency tier is for. Eval runs are never metered — evals are bursty and CI-shaped, and a bill that spikes on the day you evaluate hardest is a bill that stops you evaluating.
Do you charge per seat? +
No. Every paid tier has unlimited seats. Per-seat pricing punishes exactly the thing this product is for — getting more of the team looking at quality.
Can I cancel? +
Any time, from the billing screen. You keep access until the end of the period you have paid for, then drop to the free tier rather than losing your account.
Can I self-host the cloud? +
Not today. The framework and its local dashboard are open source and run entirely on your infrastructure, which covers most of what self-hosting is usually asked for. Get in touch if you need more than that.
Can you write our evals for us? +
Yes. The person who wrote the framework takes on a small number of fixed-price engagements: a half-day review, a four-week sprint that leaves a suite passing in your CI, or a monthly retainer. That is a separate offer from the cloud, priced on the services page, and it never changes what the product itself can see.
Want it done for you? See services →
Start free, upgrade when it hurts.
Install the package, write one eval, push a run. You will know within an afternoon whether you want the history.