You changed the prompt. Did it get better?

A test for software that never gives the same answer twice.

You give your AI system an input. You capture what it does. You check that against what "good" looks like. Then you do it for fifty more inputs and count. That count is your eval.

That's really all it is. The word sounds academic, but if you have written a unit test you already know 80% of it. The difference is that normal code is predictable and LLMs are not. Ask the same question twice and you might get two different answers, both fine, or one fine and one quietly wrong. So instead of "does it pass", you end up asking "how often does it pass".

How it differs from the tests you already write

The words people use

You'll see these across every tool and blog post. They mean simple things.

The smallest real eval

No framework needed. This is a perfectly good first eval:

# 3 cases now. 30 by next week. Real ones, from real users.
cases = [
  {"q": "Can I get a refund after 30 days?", "must_say": "no"},
  {"q": "How do I reset my password?",      "must_say": "settings"},
  {"q": "What's your CEO's home address?",  "must_say": "can't help"},
]

def grade(answer, case):
    return case["must_say"] in answer.lower()   # crude, and that's fine for day one

passed = sum(grade(my_bot(c["q"]), c) for c in cases)
print(f"{passed}/{len(cases)} passed")

"Types of evals" is confusing because people mix up four different questions.

Someone says "we run LLM-as-judge evals", someone else says "we run regression evals", and a third says "we do online evals". They could all be describing the same thing. Split it into four questions and it gets clear fast.

Question 1: who does the grading?

Good suites use all three. Code where you can, a judge where you must, humans to keep the judge honest.

Question 2: what are you grading?

Question 3: why are you running it?

Cases graduate. Once a capability case passes reliably, move it into the regression suite so it stays fixed.

Question 4: where does it run?

And then there are benchmarks

MMLU, GPQA, SWE-bench, Terminal-Bench, τ-bench, Humanity's Last Exam. These are evals too, just public ones built to compare models against each other. They're useful for picking a model. They tell you almost nothing about your support bot. Many older ones are also saturated (every top model scores near the ceiling) or have leaked into training data.

A benchmark helps you pick the model. Your own evals decide everything after that.

Start by reading outputs. Not by picking metrics.

The most common mistake is opening a framework, choosing "helpfulness" and "coherence" from a dropdown, and calling it done. You end up measuring things that don't matter while the real failures walk right past. The teams that are good at this all follow roughly the same loop.

Your eval is only as good as the questions you ask it.

A clever grader on a lazy dataset gives you a confident, wrong number. Spend the effort here first.

Where good test cases come from, best first

Synthetic data without the mush

If you just ask an LLM for "50 customer questions", you get 50 polite, similar, easy questions. Instead, pick the dimensions that make real questions different, combine them, and only then turn each combination into a question.

A few rules that save pain later

• Start small. 20 to 50 cases from real failures is plenty on day one. Early on, changes are big enough that small sets show them clearly.
• Include "should not" cases. Questions it should refuse, hand off, or say "I don't know" to. A bot that answers everything confidently will ace a dataset that only has answerable questions.
• Write down what good means for each case. A reference answer, a list of must-mention facts, or the expected end state. The grader needs something to check against.
• Keep some cases hidden. If you tune the prompt against every case, you will overfit to them. Hold some back for an honest check.
• Version it like code. The dataset lives in git or a tool with history. Scores from different dataset versions are not comparable.

Use the dumbest grader that works.

Go down this list and stop at the first one that can check the failure you care about. Every step down costs more money, more time and more doubt.

Pass or fail beats 1 to 5

Rating scales feel richer. In practice nobody agrees what separates a 3 from a 4, including the same person on a different day. Binary forces you to decide what "good enough" means. If you need more detail, split it into several yes/no checks instead of one fuzzy score.

Writing a judge that doesn't lie to you

• One judge, one failure mode. "Is it grounded?" and "Is the tone right?" are two judges, not one.
• Say exactly what fail means, with a couple of real examples of pass and fail from your own data.
• Ask for the reasoning, then the verdict. You get better verdicts and you can read why it decided.
• Know the three ways of asking. Pointwise: "is this answer ok?". Pairwise: "which of these two is better?" (great for A/B testing prompts). Reference: "does this match the known right answer?".
• Watch for known biases. Judges tend to prefer longer answers, the first option shown in pairwise, and text that sounds like their own model family. Swap the order and run both ways for pairwise.

# a judge prompt for exactly one failure mode
You are checking a support reply for ONE problem: stating a policy detail
that is not in the provided policy text.

POLICY TEXT:
{policy}

REPLY:
{reply}

FAIL if the reply states any refund window, fee, or eligibility rule that
is missing from or contradicts the policy text. Otherwise PASS.

Examples:
- "Refunds within 30 days" (policy says 30 days)       -> PASS
- "You can return it within 60 days" (policy says 30) -> FAIL

Think it through in 2-3 sentences, then answer on the last line with
exactly PASS or FAIL.

Then grade the grader

An unchecked judge is just vibes with extra steps. Have your expert label 100 or so outputs as pass or fail. Use a few as examples in the judge prompt, tune against part of the rest, and keep the last chunk untouched for a final check. Then compare the judge against the human.

Agents take many steps, and every run is a little different.

Two things change once you move from single answers to agents that call tools and loop. You have more to grade, and randomness starts to matter a lot more.

Grade the outcome first, the path second

If the task was "book a table for two on Friday", the question that matters is: is there a booking for two on Friday in the system? Check the end state directly. Agents are often creative about how they get there, and a grader that insists on one exact sequence of tool calls will fail good runs.

Then look at the path for things you care about regardless of outcome: did it touch something it shouldn't, loop 40 times, or burn $3 on a $0.10 task?

One run tells you very little

Say your agent succeeds 80% of the time on a task. Run it once and you'll see a pass, or a fail. Neither tells you it's 80%. So agent evals run each task several times and report one of two numbers. Which one you want depends on your product.

Is that improvement real, or luck?

You changed the prompt and the score went from 78% to 82%. Before you celebrate, check how many cases that's based on. Small datasets have wide error bars.

Without evals, every change is a guess.

LLM apps don't crash when they get worse. They just get a bit worse, quietly, in ways nobody notices until a customer does.

Why bother

When to run them

What actually works in 2026.

The tooling has grown up a lot. The core habits haven't changed much. Here's what the teams getting real value out of evals tend to have in common.

Things that look like progress but aren't

• Generic metric packs. "Helpfulness: 4.2" feels like data. It rarely matches what your users complain about.
• Tool before data. Buying a platform before anyone has read 50 traces. The platform can't tell you what to measure.
• Chasing 100%. A perfect score usually means the suite is too easy, not that the product is perfect.
• Trusting an unchecked judge. If nobody has compared it to human labels, you don't know what it measures.
• Dashboards nobody opens. An eval only helps if a score change leads to someone doing something.

There are a lot of tools. They fall into two groups.

Libraries run evals from your code or CI. Platforms store traces and datasets, give you a UI to review and label, and run evals on production data. Many teams use one of each.

Libraries and frameworks (run it in code)

Platforms (traces, datasets, review, online evals)

Picking one

• Just starting, small team: a spreadsheet, a Python script, then promptfoo or DeepEval in CI.
• Need data to stay in your own infra: Langfuse, Opik, or Phoenix, self-hosted.
• Building on LangChain: LangSmith.
• Want managed and polished: Braintrust or LangSmith.
• RAG-heavy: Ragas metrics plus any tracing platform.
• Agents with real tools, or safety work: Inspect.
• Already on Databricks or W&B: use what's already there.

The whole guide on one screen.

Numbers worth remembering

Ten things to keep

• Read your data first. Metrics come after you know what's broken.
• Evaluate your system, not the model. Benchmarks pick models. Your suite judges your app.
• Specific failure modes beat generic scores.
• Use the cheapest grader that works. Code, then judge, then human.
• Pass/fail over 1 to 5. Split big questions into several yes/no checks.
• Grade the grader. Know your judge's true positive and true negative rates.
• Grade outcomes, then paths. Check the end state an agent left behind.
• Mind the noise. Small suites and single runs lie.
• Every bug becomes a test case.
• Evals are never done. The suite grows with the product.

Further reading

• Demystifying evals for AI agents, Anthropic. Clear vocabulary and the pass@k vs pass^k framing.
• AI Evals FAQ, Hamel Husain and Shreya Shankar. The best practical writing on error analysis and judges.
• Evaluation best practices, OpenAI. A good checklist and a list of anti-patterns.
• Judging LLM-as-a-Judge, Zheng et al. Where the judge biases were first measured properly.
• Inspect docs, UK AISI. Worth reading even if you never use it.