I retired a Laravel package with 38,000 installs. Here's what I built instead.

On 24 August I marked vizra/vizra-adk as abandoned on Packagist. It had 38,924 installs, 296 stars, and it was still pulling roughly 146 downloads a day at the moment I retired it. v0.0.48 is the final release. There will not be another one.

This is the post explaining why, and what replaced it. The short version: Laravel shipped an official AI SDK that does the ADK's job better than the ADK did, and the interesting problem turned out to be somewhere else entirely.

What the ADK was for

Vizra ADK was an agent-building kit for Laravel. Agents as classes, tools, memory, multi-step orchestration, streaming — the plumbing you need before an LLM can do anything useful inside a real application. I started it in mid-2025 because nothing in the PHP ecosystem did that job and I was tired of writing the same glue in every project.

Then laravel/ai arrived. It passed five million installs in six months. It is maintained by the framework team, it ships with the framework's conventions, and it will be tested against every Laravel release for as long as Laravel exists. I maintained the ADK alone, in evenings.

There is a version of this post where I explain why my package was still worth using. I do not believe that version. When the framework ships the thing your package exists to provide, competing with it is a way to spend two years losing slowly. The honest move is to stop.

The thing nobody had built

Building an agent was the part that got solved. Knowing whether the agent was any good was not.

I spent the last few years building agent systems in regulated environments — RAG pipelines, LLM-as-judge evaluation, multi-agent orchestration — where "it seemed fine when I tried it" is not an acceptable answer to a compliance question. The failure mode I saw over and over was not a crash. It was a prompt tweak on a Tuesday that quietly made the answers worse, discovered three weeks later by a customer.

Traditional tests do not catch that. An agent is nondeterministic: the same input produces a different response every time, so a single assertion is a coin flip dressed up as a verdict. Fakes prove your wiring works — they do not prove the answer is still good. Those are different failure modes and you want to check both.

In Python and JavaScript there is a whole category of tooling for this. In PHP, there was Pest 5's evals plugin, one experiment by Freek, and a handful of small repos. Pest put evals on the Laracon US stage in July. The problem was on the ecosystem's radar and the answer was mostly missing.

Vizra Evals

vizra/evals runs your agent against a dataset, scores every response, and keeps the result — so two runs can be compared. It is MIT licensed, built on the official Laravel AI SDK and Pest, and it installs like anything else:

composer require vizra/evals --dev
php artisan migrate

An eval is a Pest test:

use App\Agents\SupportBot;

it('answers support questions from documented policy', function () {
    expect(SupportBot::class)->toPassEval(fn ($eval) => $eval
        ->dataset(base_path('evals/support.jsonl'))
        ->samples(3)
        ->assert(fn ($a, $row) => $a
            ->notEmpty()->gate()
            ->contains($row->expected())
            ->costBelow(0.02))
        ->judge('Answers using only documented store policy.', min: 7)
        ->gate(minScore: 0.8, maxRegressions: 0)
    );
});

Three details in there matter more than the rest.

samples(3) — every row runs three times, because agents are nondeterministic and one sample proves nothing. You get a distribution with spread, not a single lucky roll.

->gate() — if notEmpty() fails, the judge is skipped for that sample. A broken response should never spend judge tokens.

judge(...) — a structured-output judge returning {score, reasoning}. No regex parsing of a model's prose, and the reasoning is persisted, because in six weeks the reasoning is the only thing that will tell you what went wrong. Point the judge at a different model family than the agent under test; models grade their own family leniently. There is an evals:calibrate command that measures judge agreement against human-labelled data, because an uncalibrated judge is an opinion with a number attached.

Plain pest skips evals entirely — zero tokens, zero cost. pest --evals runs them against the real model. They never fire by accident in an unrelated suite.

What happens on the second run

This is the part the whole package exists for. Your first passing run becomes the suite's baseline. Rows are joined across runs by content hash, so when a specific row gets worse, the build fails and names it:

Eval [pest: answers support questions from documented policy] — score 61.7%,
pass rate 33.3% across 6 samples (run 01kyw…).
Gate failed: 2 rows regressed against the reference run (allowed: 0).
  ↓ regressed: "What is your refund policy?" 96.7% → 51.7%
  ↓ regressed: "Can I return it?" 93.3% → 55.0%

Not "a test failed". Which inputs got worse, and by how much.

Alongside the judge you can assert on the real AgentResponse rather than on text: which tools were called, with which arguments, in what order, how many steps, what it cost, which model answered. Getting the right answer by calling the wrong tool is still a bug.

There is a second package, vizra/evals-ui, that mounts a dashboard route inside your own app — score trends per suite, per-sample drill-downs with the judge's reasoning, two runs side by side. It reads the same database your evals already write to. Nothing to sign up for, nothing leaves your network.

Then it all lives on one laptop

That is the honest limit of the free setup, and it is worth naming before you hit it.

Everything above runs against your own database. Which means your baseline is whatever the last person to run it happened to have, CI throws its history away with the container, and nobody else on the team can see the trend. That is completely fine for one developer. It stops being fine the moment the answer matters to anyone else.

Vizra Cloud is the paid layer for that: shared history, shared baselines, CI runs that survive the container, and a Run button for people who never open a terminal. One project is free; it is $29/month after that.

One thing to be precise about, because it is a security claim and not a marketing one: Cloud does not run your evals. They execute inside your application, on your keys, against your data. Cloud receives finished runs over HTTP and records them. It holds no model credentials and cannot reach into your infrastructure. If you would rather not send model inputs and outputs off-site at all, VIZRA_CLOUD_SAMPLES=false sends the scores without them.

Two things this is not

It is not a replacement for the ADK. The ADK built agents; Evals tests them. There is no migration path because there is nothing to migrate to. If you need to build an agent, use the official Laravel AI SDK — not this. I would rather send a lapsed ADK user somewhere useful than pretend I still have what they came for.

It is not a competitor to Pest's evals plugin. It is built on Pest. The plugin evaluates a run; this evaluates a run and remembers it, which is what turns a score into a regression check. If you are already using the plugin and it does what you need, keep using it.

Where to start

If you want to see what the output actually looks like before installing anything, the demo is a real dataset with six weeks of history behind it. Otherwise the quickstart is about five minutes, and the source is on GitHub.

The ADK taught me something expensive: build the thing the ecosystem is missing, not the thing you already know how to build. Laravel has an AI SDK now. What it does not have is a way to prove the agent you shipped last month is still doing its job.