Insight · August 2, 2026
Evals are the
new unit test.
A unit test asserts one answer. AI does not give you one answer. Evals are how teams check probabilistic software before it ships.
01 · The shift
The hard problem moved from writing prompts to keeping quality.
For a year the skill everyone chased was the prompt. Then teams started putting agents into production, and the ground moved. A vendor ships a new model version. Someone edits a system prompt. A tool changes the shape of what it returns. Behavior that worked yesterday breaks in a way nobody sees until a customer does.
Writing a clever prompt is a one time win. Knowing whether the system still works after every one of those changes is the standing problem, and it is the one that decides who ships quality and who ships a guess. Garry Tan, who runs Y Combinator, put it plainly in early 2025. Evals are emerging as the real moat for AI startups. Not the model. Not the prompt. The ability to measure quality and hold it.
02 · Why the old test does not fit
A unit test asserts one answer. A model does not give you one.
Classic software is deterministic. Same input, same output. So a unit test can assert the result equals a fixed value and fail the build the moment it does not. That assertion is the whole safety net under everything we ship.
Language models broke the assertion. Ask the same model to write a product description twice and you get two different descriptions. Both can be good. Neither equals a stored string. assertEquals has nothing to hold onto. The output is a probability, not a constant, and you cannot pin a probability to a fixed value.
So the test changes shape. Instead of asserting one right answer, an eval scores an output against a bar you set. Is it relevant. Is it grounded in the source it was given. Did it call the right tool. Did the job actually finish. The check moves from equality to judgment, and that judgment is the eval.
The one line to keep
“Prompts are how you ask. Evals are how you know.”
03 · What an eval is made of
Five parts, whatever framework you run it in.
01
A dataset
Real inputs the system will face, each paired with the outcome you expect. A handful gets you started. Trust the aggregate once you have a few hundred.
02
A task
The exact thing the system does with each input. Answer the question, call the tool, finish the job.
03
A grader
The rule that decides pass or fail. Exact match when the answer is checkable. A model or a person when it is a matter of judgment.
04
A bar
The score the suite has to clear before a change ships. Below the bar, the merge stops.
05
A record
Every run stored, so a regression shows up as a number that moved, not a complaint from a user.
04 · The grader problem
The hard part is deciding what counts as right.
When the answer is checkable, grading is easy. The math is correct or it is not. The code runs or it does not. The tool returned the field or it did not. The researcher Jason Wei framed the checkable case well. Verifying an answer is close to building the environment the model trains against.
The trouble is the work where quality is a matter of judgment. Tone. Helpfulness. Whether an answer is actually grounded in the document it cites. For that, teams reach for a model as the grader, an approach usually called LLM as judge. It scales to thousands of cases and it tracks human judgment closely on many tasks.
But a judge is a model too, carrying the same failure modes it is meant to catch. Eugene Yan, who has written widely on evaluation, warns that model graders favor the first answer they see, reward length, and lean toward text that looks like their own. His advice is to grade in plain pass or fail rather than a one to five score, and to check the judge against a person you trust before you believe it.
05 · The same change, with and without a suite
Where the bar catches what a person would miss.
Picture a support agent that reads a customer message and drafts a reply. Someone edits the system prompt to make the tone warmer. Without a suite, the change goes out and you learn it broke refund policy answers a week later, when a customer quotes one back to you.
With a suite, the same edit runs first against a few hundred real conversations, each labeled with the correct action. The warmth score goes up. The policy accuracy score drops two points below the bar. The merge stops. No customer ever sees it. Same change, same model, one of them a quiet incident and the other a number on a screen.
eval refund_reply
dataset 300 real support threads, correct action labeled
task draft a reply and pick one action: answer, escalate, refund
grade action matches the label
and no reply promises a refund the policy forbids
bar action accuracy at least 0.95, policy violations at 0
gate block the merge when the bar is not metAn eval, run on every change · the merge waits on the bar
06 · Past the merge
A unit test goes quiet. An eval keeps working.
A unit test runs in the pipeline and then goes silent until the next change. An eval does not have that luxury, because the model keeps drifting after the change ships. So the strongest teams wire the same eval into two places. A gate before the merge, and a guardrail at runtime that samples live outputs, scores them, and can stop a bad answer before it reaches a person. A regression becomes a number that moved, not a support ticket.
Agents raise the bar again. A single answer is one thing to grade. An agent takes a path, calls tools, and changes state along the way. Grading only its last message misses most of what it did. Anthropic makes the point in its own guidance. An agent eval has to grade the state the agent left behind, the actual outcome in the environment, not just the words at the end.
07 · When the number lies
A green suite is only as honest as its data.
An eval is a measurement, and measurements can be wrong. Public benchmarks are full of bad labels. An audit of a widely used coding benchmark found that nearly six in ten of the failures it flagged came from broken tests rather than broken code. Reviewers correcting a hard math benchmark had to change more than four in ten of the answers. Across major benchmarks the label error rate averages a few percent, and a few percent is enough to flip which model looks best.
The lesson is not to skip the suite. It is to treat the suite itself as something you inspect. Look at the cases it gets wrong. Fix the dataset. Check the judge against a person. Hamel Husain, who writes about improving AI products, calls looking at your own failures the highest return activity in the whole process. Eugene Yan puts the same warning the other way. Buying or building another evaluation tool will not save the product. The work is the looking.
Closing
You cannot hold quality you refuse to measure.
Pick the one output your product cannot get wrong. Write ten real examples, each with the answer you expect. Score your system against them today, before the next model update decides the question for you.
Garry Tan · Y Combinator, February 2025 · Eugene Yan and Hamel Husain on evaluation practice · Jason Wei on verifiable tasks · Anthropic guidance on agent evaluation · benchmark audit figures reviewed August 2026
Share this perspective
More insights
Adjacent perspectives.
Bttr. Field Brief
The brief Bttr. writes for senior buyers.
Monthly. One signal worth your time on Brand Operating Systems, AI search visibility, and the infrastructure buildout. No filler.