Jev Gives Your Laravel App a Probability. Here Is What to Do When It Says 0.69

TypeSafe released Jev on 15 September 2026. Two days later the Laravel AI SDK grew a Classification API on its 1.x branch, with TypeSafe as the first provider. Freek already used it to kill a hand-maintained spam list, and the PR's own example routes support tickets. Both stop at the same line: $response['department']->choice.

That is the boring number. The interesting one sits next to it. A customer asks "can we pay by bank transfer instead of card before we sign up for the annual plan". Jev says billing with a confidence of 0.69, and sales is the runner-up at 0.23. What does your code do now?

The obvious code does nothing. It reads ->choice, routes the ticket, and the 0.69 evaporates. Which means you paid for a model that can say "I'm not sure" and then built a system that cannot hear it.

This post builds the missing half. A threshold policy that treats confidence as a per-action decision. A review queue in Filament for the calls the model should not make alone. A write-back so the truth lands next to the guess. And an Artisan command that turns "calibrated" from a marketing word into a table you can read. All of it against 60 real-looking support tickets I wrote and labelled by hand, so the numbers at the end are measured, not imagined.

What the Classification API actually returns

First the ground rules, because they shape everything after. Classification is on the 1.x branch only, so you install a dev version:

composer require laravel/ai:1.x-dev
php artisan vendor:publish --tag=ai-config

Stable is v0.11.2 at the time of writing, and 1.x has an upgrade guide branch open, so a 1.0 tag is close. The docs on laravel.com do not mention Classification yet. The PR author also said the API may be flagged experimental after release. Every code block here ran against commit ca8d9bf of 1.x on Laravel 13.32. If a method is renamed before you read this, the source under vendor/laravel/ai/src/Classification is short enough to diff in a minute.

Config is one key. config/ai.php already carries 'default_for_classification' => 'typesafe' and a typesafe provider reading TYPESAFE_API_KEY. The default model is jev-latest.

There are three question types, and they return three different answer objects:

use Laravel\Ai\Classification;
use Laravel\Ai\Classification\Boolean;
use Laravel\Ai\Classification\Choice;
use Laravel\Ai\Classification\Score;

$response = Classification::of([
    'subject' => $ticket->subject,
    'body' => $ticket->body,
])->questions([
    'department' => new Choice('Which team should handle this ticket?', [
        'billing' => 'Invoices, failed payments, plan changes, VAT',
        'technical' => 'Bugs, errors, integrations, API problems',
        'sales' => 'Pricing questions, quotes, trials, upgrades before buying',
        'refund' => 'The customer is asking for money back',
    ]),
    'urgent' => new Boolean('Does the customer need a response today?'),
    'frustration' => new Score('How frustrated is the customer?', [
        'Calm, stating facts',
        'Annoyed but polite',
        'Angry or threatening to leave',
    ]),
])->classify();

// Real answers for the bank transfer ticket from the intro
$response['department']->choice;         // 'billing'
$response['department']->confidence;     // 0.69
$response['department']->probabilities;  // ['billing' => 0.77, 'sales' => 0.23, 'technical' => 0, 'refund' => 0]
$response['urgent']->probability;        // 0.34
$response['urgent']->isTrue(0.7);        // false
$response['frustration']->level();       // 0
$response['frustration']->label();       // 'Calm, stating facts'
$response->meta->model;                  // 'jev-1.13.0'
$response->usage->inputTokens;           // 478

Two details matter for the rest of the post.

Boolean answers carry a probability and nothing else. There is no confidence property on BooleanAnswer, because for a yes/no question the probability already is the certainty. 0.5 means the model has no idea, 0.99 or 0.01 means it does.

Choice and Score answers carry both. The probabilities array is the real output, and confidence is a single number TypeSafe derives from its shape. Their confidence docs put it plainly: a distribution concentrated on one option is a confident answer, a flat one is not. It is not the same as the top probability. One ticket in my run came back billing at 0.84 with a confidence of 0.78, because the remaining 0.16 sat on a single rival option rather than being spread thin. And the SDK types confidence as ?float, so a future provider that cannot measure it returns null. Your policy has to handle that case from day one.

The other line worth pinning on the wall comes from the System One page. Calibration is measured across groups of predictions, and in their words "it does not guarantee that an individual answer is correct." So a 0.69 does not mean this ticket is 69% billing. It means that across many answers where Jev said 0.69, roughly 69% should turn out right. That is a statement about your whole queue, and you can only check it if you keep records. Hold that thought.

A threshold is a policy, not a number

The naive version is if ($answer->confidence >= 0.8). It fails the first time a refund request auto-routes to billing and sits there for two days. Misrouting a pricing question costs a forwarded email. Misrouting a refund costs a chargeback. The same confidence should not be allowed to do both.

TypeSafe's own docs suggest three ranges (act, proceed with caution, do not act) and note that the boundaries move with the stakes. Here is what that looks like as a Laravel class:

namespace App\Support;

use Laravel\Ai\Responses\Data\ChoiceAnswer;

final class ConfidencePolicy
{
    /**
     * @param  float  $floor  Below this the model is guessing. Never act.
     * @param  array<string, float>  $actAt  Per-option threshold to act without a human.
     */
    public function __construct(
        private float $floor = 0.5,
        private array $actAt = [
            'billing' => 0.75,
            'technical' => 0.75,
            'sales' => 0.75,
            'refund' => 0.9,   // money leaves the building, so the bar is higher
        ],
    ) {}

    public function decide(ChoiceAnswer $answer): Decision
    {
        if ($answer->confidence === null || $answer->confidence < $this->floor) {
            return Decision::Review;
        }

        $threshold = $this->actAt[$answer->choice] ?? 1.0;

        return $answer->confidence >= $threshold ? Decision::Act : Decision::Review;
    }
}

Decision is a two-case enum, Act and Review. Three things are deliberate here. A null confidence goes to review, not to a default threshold. An option missing from $actAt gets a threshold of 1.0, so adding a fifth department to the Choice without adding a threshold fails safe. And the numbers are constructor arguments, so a test can pass a strict policy and a lenient one without touching config.

The 0.69 from the intro hits the floor check, passes it, then fails billing's 0.75. Review. Which is the right answer for a sales question wearing billing vocabulary. The customer has not bought anything yet. (Running the same ticket again a minute later gave 0.70. Jev is not perfectly deterministic, so a policy that flips on the second decimal is a policy with a bad threshold.)

The action that ties it together is short. It classifies, decides, records, and only touches the ticket when the policy says so:

namespace App\Actions;

class RouteTicket
{
    public function __construct(private ConfidencePolicy $policy) {}

    public function handle(Ticket $ticket): ClassificationDecision
    {
        $started = hrtime(true);

        $response = Classification::of([...])->questions([...])->classify();

        $department = $response['department'];
        $decision = $this->policy->decide($department);

        $record = $ticket->decisions()->create([
            'question' => 'department',
            'suggested' => $department->choice,
            'confidence' => $department->confidence,
            'probabilities' => $department->probabilities,
            'decision' => $decision->value,
            'model' => $response->meta->model,
            'input_tokens' => $response->usage->inputTokens,
            'latency_ms' => (int) ((hrtime(true) - $started) / 1_000_000),
        ]);

        if ($decision === Decision::Act) {
            $ticket->update(['department' => $department->choice, 'routed_by' => 'model']);
        }

        return $record;
    }
}

Every call writes a classification_decisions row, whether the model acted or not. That table is the audit log, and it is the input to everything below. If you already run Spatie's activity log, you could log there instead, but a dedicated table with typed columns for confidence and suggested makes the calibration query at the end a plain where, not a JSON path.

The review queue is the other half of automation

A decision of Review has to land somewhere a human will actually look. In my apps that is Filament, so the queue is a resource over the same classification_decisions table, filtered to what the model was unsure about and nobody has resolved:

public static function getEloquentQuery(): Builder
{
    return parent::getEloquentQuery()
        ->where('decision', 'review')
        ->whereNull('actual')
        ->with('ticket')
        ->orderBy('confidence');
}

public static function getNavigationBadge(): ?string
{
    return (string) static::getEloquentQuery()->count();
}

Lowest confidence first, because those are the ones where the runner-up is most likely right. The badge on the sidebar is the whole "someone must work this queue" problem made visible. Two row actions do the work:

->recordActions([
    Action::make('accept')
        ->icon(Heroicon::Check)
        ->color('success')
        ->action(fn (ClassificationDecision $record) => static::resolve($record, $record->suggested)),
    Action::make('override')
        ->icon(Heroicon::ArrowUturnLeft)
        ->color('gray')
        ->schema([
            Radio::make('department')
                ->options(fn (ClassificationDecision $record) => collect($record->probabilities)
                    ->map(fn (float $p, string $option) => sprintf('%s (%.2f)', $option, $p))
                    ->all())
                ->required(),
        ])
        ->action(fn (ClassificationDecision $record, array $data) => static::resolve($record, $data['department'])),
])

The override form shows the model's own probabilities next to each option. The reviewer sees that sales was at 0.23, which is often enough context to decide in two seconds. And resolve() writes the same two columns in both cases:

protected static function resolve(ClassificationDecision $record, string $department): void
{
    $record->update([
        'actual' => $department,
        'actual_source' => 'review',
        'reviewed_at' => now(),
    ]);

    $record->ticket->update(['department' => $department, 'routed_by' => 'human']);
}

This is the same shape as the tool approval flow the SDK ships for agents, applied to a classifier. The agent version pauses a tool call until a human approves it. This version pauses a routing decision. Same principle, and the same rule about who gets to press the button.

Write the truth back, including for the calls you got right

The review queue gives you ground truth for the decisions the model doubted. That is a biased sample. If you only ever learn the outcome of low-confidence calls, you can never find out whether your 0.9s are actually 90% right, and that is the question that decides whether your threshold is too high or too low.

So the auto-routed decisions need an outcome too. In a real helpdesk the natural moment is ticket closure, when an agent has handled it and the department it was closed in is known. That is one listener:

class RecordRoutingOutcome
{
    public function handle(TicketClosed $event): void
    {
        $event->ticket->decisions()
            ->where('question', 'department')
            ->whereNull('actual')
            ->update([
                'actual' => $event->ticket->department,
                'actual_source' => 'closed',
            ]);
    }
}

The whereNull('actual') matters. A reviewed decision already has its truth from a human and must not be overwritten by whatever department the ticket eventually drifted to.

For the demo I do not have a helpdesk, so a small command writes each ticket's labelled department onto its auto-routed decision, standing in for the closure event. Same two columns, same whereNull guard.

If you also want a global record of every classification for cost tracking, the SDK fires Laravel\Ai\Events\Classified with the invocation id, provider, model, prompt and response after every call. It is the right hook for a usage ledger. It is the wrong hook for outcome tracking, because the invocation id never reaches your calling code, so you cannot join it back to the ticket later. Log decisions from the action, log usage from the event.

Now calibration is a number you own

With suggested, confidence and actual on the same row, the report is a single query and two tables:

$decisions = ClassificationDecision::query()
    ->where('question', 'department')
    ->whereNotNull('actual')
    ->get();

The first table buckets by confidence and asks how often the model was right in each bucket. The second replays every threshold from 0.5 to 0.9 and asks what would have happened: how many tickets would have auto-routed, how many would have gone to a human, and how many of the automatic ones would have been wrong.

collect([0.5, 0.6, 0.7, 0.8, 0.9])->map(function (float $threshold) use ($decisions) {
    $auto = $decisions->filter(fn ($d) => $d->confidence >= $threshold);
    $wrong = $auto->reject->wasCorrect()->count();

    return [
        number_format($threshold, 2),
        sprintf('%d (%d%%)', $auto->count(), round($auto->count() / $decisions->count() * 100)),
        $decisions->count() - $auto->count(),
        $auto->isEmpty() ? '-' : sprintf('%d%%', round(($auto->count() - $wrong) / $auto->count() * 100)),
        $wrong,
    ];
});

I ran the 60 tickets through jev-latest, which resolved to jev-1.13.0. 18 billing, 18 technical, 14 sales, 10 refund, with 10 written to sit between two departments (a "charge I don't recognise" that is billing, and its twin with "reverse it please" that is a refund). The policy above auto-routed 58 and sent 2 to review. I resolved those two in Filament, overriding both, then wrote the labels back onto the rest to stand in for ticket closure. Here is the report:

60 decisions with a known outcome, model jev-1.13.0

Accuracy by confidence bucket:

If the act threshold had been:

Read the first table top to bottom. Below 0.7 Jev was wrong both times, which is what "I'm not sure" should look like. From 0.8 up it was right 55 times out of 55. The bucket that matters is 0.7 to 0.8, where one answer in three was right, and my 0.75 bar sits in the middle of it. Both wrong auto-routes landed there, at 0.76 and 0.78, and both were tickets a human could argue either way ("Invoice PDF will not download" went to technical, my label said billing, and honestly the model has a case).

Now read the second table as a menu. At 0.75 I got 2 wrong routes and 2 reviews. Moving the bar to 0.80 costs three more tickets in the queue and removes every wrong route. That is not a judgement call any more. It is a row in a table, and the next run of the command tells you whether it held.

Sixty tickets is a small sample and I wrote them, so treat the percentages as a demonstration of the method, not a benchmark of Jev. The point is that the table exists. You can run it every Monday on last week's decisions, and when it says your 0.75 bucket is only 70% right, you raise the threshold and the review queue grows by a known amount. Threshold tuning stops being a feeling.

Cost, for the record. The 60 tickets used 28,479 input tokens, an average of 475 per ticket with three questions each. TypeSafe lists Jev at $42 per billion input tokens on typesafe.ai, which is $0.042 per million, so the whole run cost about a tenth of a cent. Latency from a VPS in Vienna was 589ms on average and 583ms median, with the slowest call at 704ms. Freek measured 639ms for a similar three-question call, so that is roughly what to budget for.

Testing without spending a cent

The SDK ships a fake for classification, which is where the policy gets its real test. You hand it the exact answer you want and assert what your code did with it:

use Laravel\Ai\Classification;
use Laravel\Ai\Responses\Data\ChoiceAnswer;

it('sends a 0.69 department call to the review queue instead of acting', function () {
    Classification::fake([
        ['department' => new ChoiceAnswer('billing', [
            'billing' => 0.77, 'sales' => 0.23, 'technical' => 0, 'refund' => 0,
        ], confidence: 0.69)],
    ]);

    $ticket = Ticket::create([...]);

    $decision = app(RouteTicket::class)->handle($ticket);

    expect($decision->decision)->toBe('review')
        ->and($ticket->fresh()->department)->toBeNull();

    Classification::assertClassified(fn ($prompt) => $prompt->asks('department'));
});

Any question you do not override gets a random but shape-valid answer, so the urgent and frustration questions in the action do not need stubbing. Classification::fake() also accepts a closure that receives the ClassificationPrompt, so you can return 1.0 when $prompt->contains('ASAP') and 0.0 otherwise, and preventStrayClassifications() makes an unfaked call throw. The second test in my suite feeds a refund at 0.85 and then at 0.93 and asserts that only the second one acts, which is the per-option threshold doing its job.

If you queue the routing (and you should, since a classification is an HTTP call to a third party), the Horizon setup for AI SDK jobs applies unchanged. Short timeout, its own queue, a retry that does not double-write the decision row.

Trade-offs

You are building on a dev branch. The class names are stable enough that I would ship this behind a composer.lock pin, but not on 1.x-dev without one. When 1.0 tags, check the upgrade guide the same way you did at 0.x.

One provider. Today Classification has TypeSafe and nothing else. The abstraction is there (ClassificationProvider, ClassificationGateway), so a second provider is a PR away, but right now a TypeSafe outage is a classification outage. The SDK's failover loop in PendingClassification::classify() already iterates providers, it just has one to iterate.

Boolean has no confidence, by design. If you gate on a Boolean, gate on distance from 0.5. A probability of 0.55 for "urgent" is a shrug, not a yes.

A review queue only works if someone works it. The badge count is not decoration. If it climbs past what your team clears in a day, the fix is not to lower the thresholds, it is to look at what the model keeps doubting. In my 60 tickets, three of the five lowest-confidence answers were tickets from the batch of ten I had deliberately written to sit between two departments. That is what you want a classifier to do.

And the calibration report is only as honest as your write-back. Skip the closure listener and you will tune thresholds on the biased half of the data. That shortcut is where the wrong lessons come from.

FAQ

Does the Laravel AI SDK Classification API work on the stable release?

Not yet. Classification landed on the 1.x branch on 17 September 2026 (laravel/ai PR #1010). The latest stable tag is v0.11.2. Install laravel/ai:1.x-dev to use it now, and expect a 1.0 release to follow, since the branch already has an upgrade guide in progress.

What is the difference between probability and confidence in a Jev answer?

For a Choice or Score, probabilities is the full distribution across your options or levels, and confidence is one number TypeSafe derives from how concentrated that distribution is. For a Boolean, there is only probability, because a yes/no answer's certainty is the probability itself. Confidence can be null if a provider cannot measure it, so always handle that case.

Can I use a different threshold for each option?

Yes, and you should. The policy class in this post keeps a floor below which nothing acts, then a per-option threshold above it. A refund route needs a higher bar than a sales route because the cost of being wrong is higher. Options without an explicit threshold default to 1.0, which sends them to review.

How do I test classification code without calling TypeSafe?

Classification::fake() accepts an array of per-question answers or a closure. Pass a ChoiceAnswer with the exact probabilities and confidence you want to exercise, run your code, and assert on the outcome. Classification::assertClassified() and assertNothingClassified() check that the call happened, and preventStrayClassifications() makes any unfaked call throw.

Is Jev cheaper than using an LLM for the same classification?

TypeSafe publishes $42 per billion input tokens with no charge for output. Whether that beats your current LLM setup depends on your prompt length and volume, and I have not benchmarked the two side by side here. What I can say from this run is the token count and latency per ticket above. A fair comparison against a small LLM with structured output is a separate post.

What to take away

The Classification API is ten lines of code and everyone will write those ten lines. The decisions table, the per-option policy, the review queue and the write-back are maybe 200 more, and they are what let you raise a threshold on a Monday morning because a report told you to, instead of because a customer complained.

Jev's whole pitch is that it tells you when it is unsure. Build the part that listens.