Skip to main content

Insight · August 2, 2026

Evals are the
new unit test.

A unit test asserts one answer. AI does not give you one answer. Evals are how teams check probabilistic software before it ships.

01 · The shift

The hard problem moved from writing prompts to keeping quality.

For a year the skill everyone chased was the prompt. Then teams started putting agents into production, and the ground moved. A vendor ships a new model version. Someone edits a system prompt. A tool changes the shape of what it returns. Behavior that worked yesterday breaks in a way nobody sees until a customer does.

Writing a clever prompt is a one time win. Knowing whether the system still works after every one of those changes is the standing problem, and it is the one that decides who ships quality and who ships a guess. Garry Tan, who runs Y Combinator, put it plainly in early 2025. Evals are emerging as the real moat for AI startups. Not the model. Not the prompt. The ability to measure quality and hold it.

02 · Why the old test does not fit

A unit test asserts one answer. A model does not give you one.

Classic software is deterministic. Same input, same output. So a unit test can assert the result equals a fixed value and fail the build the moment it does not. That assertion is the whole safety net under everything we ship.

Language models broke the assertion. Ask the same model to write a product description twice and you get two different descriptions. Both can be good. Neither equals a stored string. assertEquals has nothing to hold onto. The output is a probability, not a constant, and you cannot pin a probability to a fixed value.

So the test changes shape. Instead of asserting one right answer, an eval scores an output against a bar you set. Is it relevant. Is it grounded in the source it was given. Did it call the right tool. Did the job actually finish. The check moves from equality to judgment, and that judgment is the eval.

The one line to keep

“Prompts are how you ask. Evals are how you know.”

03 · What an eval is made of

Five parts, whatever framework you run it in.

01

A dataset

Real inputs the system will face, each paired with the outcome you expect. A handful gets you started. Trust the aggregate once you have a few hundred.

02

A task

The exact thing the system does with each input. Answer the question, call the tool, finish the job.

03

A grader

The rule that decides pass or fail. Exact match when the answer is checkable. A model or a person when it is a matter of judgment.

04

A bar

The score the suite has to clear before a change ships. Below the bar, the merge stops.

05

A record

Every run stored, so a regression shows up as a number that moved, not a complaint from a user.

04 · The grader problem

The hard part is deciding what counts as right.

When the answer is checkable, grading is easy. The math is correct or it is not. The code runs or it does not. The tool returned the field or it did not. The researcher Jason Wei framed the checkable case well. Verifying an answer is close to building the environment the model trains against.

The trouble is the work where quality is a matter of judgment. Tone. Helpfulness. Whether an answer is actually grounded in the document it cites. For that, teams reach for a model as the grader, an approach usually called LLM as judge. It scales to thousands of cases and it tracks human judgment closely on many tasks.

But a judge is a model too, carrying the same failure modes it is meant to catch. Eugene Yan, who has written widely on evaluation, warns that model graders favor the first answer they see, reward length, and lean toward text that looks like their own. His advice is to grade in plain pass or fail rather than a one to five score, and to check the judge against a person you trust before you believe it.

05 · The same change, with and without a suite

Where the bar catches what a person would miss.

Picture a support agent that reads a customer message and drafts a reply. Someone edits the system prompt to make the tone warmer. Without a suite, the change goes out and you learn it broke refund policy answers a week later, when a customer quotes one back to you.

With a suite, the same edit runs first against a few hundred real conversations, each labeled with the correct action. The warmth score goes up. The policy accuracy score drops two points below the bar. The merge stops. No customer ever sees it. Same change, same model, one of them a quiet incident and the other a number on a screen.

eval refund_reply
  dataset  300 real support threads, correct action labeled
  task     draft a reply and pick one action: answer, escalate, refund
  grade    action matches the label
           and no reply promises a refund the policy forbids
  bar      action accuracy at least 0.95, policy violations at 0
  gate     block the merge when the bar is not met

An eval, run on every change · the merge waits on the bar

06 · Past the merge

A unit test goes quiet. An eval keeps working.

A unit test runs in the pipeline and then goes silent until the next change. An eval does not have that luxury, because the model keeps drifting after the change ships. So the strongest teams wire the same eval into two places. A gate before the merge, and a guardrail at runtime that samples live outputs, scores them, and can stop a bad answer before it reaches a person. A regression becomes a number that moved, not a support ticket.

Agents raise the bar again. A single answer is one thing to grade. An agent takes a path, calls tools, and changes state along the way. Grading only its last message misses most of what it did. Anthropic makes the point in its own guidance. An agent eval has to grade the state the agent left behind, the actual outcome in the environment, not just the words at the end.

07 · When the number lies

A green suite is only as honest as its data.

An eval is a measurement, and measurements can be wrong. Public benchmarks are full of bad labels. An audit of a widely used coding benchmark found that nearly six in ten of the failures it flagged came from broken tests rather than broken code. Reviewers correcting a hard math benchmark had to change more than four in ten of the answers. Across major benchmarks the label error rate averages a few percent, and a few percent is enough to flip which model looks best.

The lesson is not to skip the suite. It is to treat the suite itself as something you inspect. Look at the cases it gets wrong. Fix the dataset. Check the judge against a person. Hamel Husain, who writes about improving AI products, calls looking at your own failures the highest return activity in the whole process. Eugene Yan puts the same warning the other way. Buying or building another evaluation tool will not save the product. The work is the looking.

Closing

You cannot hold quality you refuse to measure.

Pick the one output your product cannot get wrong. Write ten real examples, each with the answer you expect. Score your system against them today, before the next model update decides the question for you.

Donny Smith

Written by · August 2, 2026

· Founder & Executive Creative Director, Bttr.

Over the past 15 years, he has led creative teams and contributed to products used by millions of people worldwide, working with companies including Apple, Opendoor, JP Morgan, GE Aerospace, Pepsi, and Alterra Mountain Company.

LinkedIn

Garry Tan · Y Combinator, February 2025 · Eugene Yan and Hamel Husain on evaluation practice · Jason Wei on verifiable tasks · Anthropic guidance on agent evaluation · benchmark audit figures reviewed August 2026

Share this perspective

More insights

Adjacent perspectives.

What Is LLM as a Judge

7 min read

What Is LLM as a Judge

LLM as a judge is the practice of using one language model to score another model’s output against a set of criteria. You hand the judge a rubric and either a single answer to rate or two answers to compare, and it returns a verdict: a number, a label, or a winner. It exists because the other options fall short. Anthropic’s evaluation guidance frames the tradeoff directly: code based grading is the fastest and most reliable but lacks nuance, human grading is the most flexible and highest quality but slow and expensive, and model based grading runs at the scale of code with something closer to the judgment of a person. The catch is that the grader is itself a model, so it carries a model’s biases. Zheng and colleagues named three in their foundational MT-Bench study: position bias, where a judge favors the answer shown first, verbosity bias, where it prefers the longer answer, and self enhancement bias, where it rates its own kind of output higher. The method still works. That same study found a strong judge agreed with human preferences more than 80 percent of the time, about the rate humans agree with one another, and 85 percent excluding ties against 81 percent among the people. The operator rule follows: validate the judge against human labels before you trust it, swap positions to cancel order effects, and grade with a different model than the one that wrote the answer.

What Is an AI Guardrail

7 min read

What Is an AI Guardrail

An AI guardrail is a check that runs around a model at the moment of a request. It reads what goes into the model and what comes out, screens for prompt injection, sensitive data, and off policy answers, and can block, redact, or retry when a check fails. It is enforcement code that sits outside the model, so the model itself does not have to be trusted to behave. Anthropic documents the core pattern directly: pass each input or tool result to a small, fast model such as Claude Haiku, ask only whether it carries an injection attempt, and let the content reach the main model only if the screen comes back clean. A guardrail is not an eval. An eval grades quality before you ship, offline and against a fixed set. A guardrail acts at run time, on the live request, one call at a time. What a guardrail is, how it differs from an eval, and how to layer a cheap deterministic check under a model based one.

Bttr. Field Brief

The brief Bttr. writes for senior buyers.

Monthly. One signal worth your time on Brand Operating Systems, AI search visibility, and the infrastructure buildout. No filler.