Learning

Evals: The Missing Piece in How We Build AI Agents

Notes from studying and implementing evals across my agents. What graders are, how to build test cases, and why measuring your agents is the most important thing most people skip.

May 25, 20268 min98 views
#evals#ai-agents#graders

These are my notes from studying and implementing evals across my own agents. I've heard the term thrown around in ML circles for a while, but it wasn't until I started building agents that handle real, autonomous work that I understood why they matter. This isn't a finished framework - it's what I'm learning works in practice as I build.


Your AI agent is probably broken. You just don't know it yet.

We build agents that write emails, handle customer questions, manage inventory, generate reports. They produce outputs that look fine. So we assume they work. We run a few examples, check if the outputs "feel right," and ship. It's vibe-based development - and it works until it doesn't.

Then you start measuring. And you find the bugs you never would have caught by reading outputs.

This is about evals, graders, and why measuring your agents is the most important thing most people skip.


Why this matters now

Over the past few weeks I've been fine-tuning my memory systems and agents. I have multiple agents running, and the goal is for them to live autonomously - handling work without me checking every output.

But autonomy without measurement is just hope.

Every time I update an agent - to use context better, to handle more cases, to be more efficient with resources - I need to know: did this change actually make it better? Or did I break something that was working?

That's the role of evals. And the more autonomous your agents become, the more critical evals get.


What are evals?

What are evals - examining test results

Evals are tests for AI agent behavior. Not "does the code run" - but "does the agent actually do what it should?"

A developer writes unit tests for code. An agent builder writes evals for behavior.

The difference: code is deterministic - same input, same output, every time. Agents aren't. The same prompt can produce different responses, different tool calls, different quality. That's why traditional testing doesn't work here, and why evals exist.

Think of it this way: if you hired a new employee to answer customer emails, you wouldn't just check if they replied. You'd check if they replied correctly, in the right tone, to the right person, within a reasonable time. Evals do that for your agents.

Every eval has three parts: what you're testing (your test cases and expected results), how you run it (manually or automated), and how you grade it (the rules that decide pass or fail).

Two categories that matter:

Regression evalsthings that must always work. These protect what's already working. If a regression eval breaks, something fundamental changed. Target: near 100% pass rate. Always.
Capability evalswhat can this agent do well? These start with low pass rates and climb over time as you improve. They measure your ceiling, not your floor.

Building your test case library

A test case is one example of what the agent will encounter in the real world. You don't need hundreds. You need the right ones.

A good library covers four types:

Happy paththe typical, well-formed inputs your agent sees most often. A clear customer question. A straightforward data request. These should always pass.
Edge casesunusual inputs. Very short prompts, very long documents, ambiguous questions, inputs in unexpected formats or languages. These reveal assumptions your agent makes.
Adversarial casesinputs designed to trip the agent up. Contradictory information, trick questions, requests slightly out of scope. These show how your agent fails.
Regression casesspecific bugs you already found and fixed. Every time you fix a bug, add it as a test case. This prevents the same bug from coming back.

Start with 20-50 test cases sourced from real failures and real usage. Not hypothetical scenarios - actual inputs that caused problems. Your bug tracker and conversation logs are the best source material.


What are graders?

So you have your test cases. You run them. But how does the eval actually know if the agent got it right?

That's the grader. It's the answer key - the rule that decides pass or fail.

There are three types:

Code-based

Strengths

Fast, cheap, objective, same result every time

Weaknesses

Brittle to valid variations, can't judge nuance

Model-based (LLM judge)

Strengths

Flexible, captures nuance, handles subjective quality

Weaknesses

Non-deterministic, more expensive, needs calibration

Human

Strengths

Gold standard for quality, expert judgment

Weaknesses

Slow, expensive, needs domain expertise

Most teams start with code-based and model-based. Human graders calibrate the other two.

Here are the grader patterns I've been working with:

String Check (exact_match)

What it checks

Is the output exactly what you expected?

Example

"Route this to sales" - did it say "sales"? Pass.

Set Match

What it checks

Are ALL expected items present?

Example

Extract 5 entities from a contract. Got all 5?

Action Taken

What it checks

Did the agent DO it, not just SAY it?

Example

It said it sent the email. Did the API actually fire?

LLM Judge

What it checks

A second AI scores the output using a rubric

Example

Does this customer reply sound empathetic, not robotic?

Text Similarity

What it checks

How close is the output to a reference?

Example

Compare a generated summary to a human-written one.

Composite

What it checks

Multiple graders combined into one check

Example

Email was sent (action) AND tone was right (judge).

Efficiency

What it checks

Was it fast enough? Cheap enough?

Example

Did it answer in under 2,000 tokens, not 15,000?

One distinction that changed how I think about grading:

Output vs Outcome.

Output is what the agent says it did. Outcome is what actually happened.

"I sent the email" is an output. The email arriving in the inbox is the outcome. Your grader should check the outcome when possible - not trust the agent's self-report.


Creating your own graders

A grader is just a question and a way to check the answer. If you can define what "right" looks like, you can build a grader for it.

You notice your agent sometimes responds in English when the user wrote in French. No standard grader catches this. So you create one:

language_matchdid the agent respond in the user's language? Detect input language, detect output language, compare. Match = pass.

Your eval suite just got smarter because you defined what matters to your specific use case.

More examples:

brand_voice

What it checks

Does the output sound like your brand? Rubric with your specific tone rules.

no_hallucination

What it checks

Did the agent only reference real data? Cross-check claims against the source.

tool_order

What it checks

Did the agent search before creating? Check the trace: was lookup called before write?

cost_guard

What it checks

Did this task stay under budget? Total tokens x price per token < threshold.

Standard graders cover 80% of what you need. Custom graders cover the 20% that makes your agent yours.


Deterministic vs Non-deterministic

If these terms are new to you, here's the simplest way to think about it:

Deterministic means there's one right answer and you can check it automatically - like checking if 2+2 equals 4. Did the agent save the file? Yes or no. Did it pick the right category? Exact match. Same question, same answer, every time. No opinion involved.

Non-deterministic means the answer can vary and you need judgment to evaluate it - like grading an essay. Does the response sound natural? Is the recommendation useful? Did it handle the edge case well? These need a rubric and a judge, because "good" isn't a single fixed answer.

You need both. Deterministic evals catch broken behavior. Non-deterministic evals catch degraded quality.

When your evals are non-deterministic, run them multiple times:

pass@kdid it pass at least once in k attempts? The optimistic view.
pass^kdid it pass every single time? The consistency view.

An agent that passes 4 out of 5 times is very different from one that passes 5 out of 5. Both metrics matter. Know which one you're measuring.


Hill climbing: the improvement loop

Hill climbing - each flag marks a measured step upward

Hill climbing is a term from optimization. The idea is simple: you're standing somewhere on a hill, but there's fog everywhere. You can't see the top. All you can do is feel the ground under your feet and take a step in whichever direction goes up.

In agent development, the "ground" is your eval score. The "step" is a change you make. The "fog" is the fact that you can't predict how a change will affect everything.

This is where evals become powerful.

Run your eval suite. Get a score. See what failed. Fix one thing. Run again.

Each cycle, the score goes up - or it doesn't, and you revert the change. That's one step on the hill. You don't need to see the top. You just need to know if this step went up or down.

The key is "fix ONE thing." Not five. Not a rewrite. One change, one measurement. If the score went up, keep it. If it went down, undo it and try something else. This is how you avoid breaking things that work while you improve things that don't.

Anthropic shared their own numbers from a production agent: 62% to 83% after restructuring. 83% to 92% after fine-tuning. Not one big leap - many small steps, each one measured.

Without evals, you make changes and hope they help. With evals, you know.


Where to start

You don't need a big eval suite on day one. Start with three:

  1. One exact_match eval for your most common task - does it route, label, or categorize correctly?
  2. One action_taken eval for the most important side effect - does it actually save, send, or create what it should?
  3. One llm_judge eval for output quality - does it match the tone and format you want?

Those three cover the basics: is the answer right, did the action happen, and is the quality acceptable.

Run those three after every change. When you find gaps, add more tests. Every bug you fix becomes a new regression test. That's the natural growth of an eval suite - driven by real failures, not theoretical coverage.

Here's what the growth looks like in practice:

  • Week 1: 3 evals. You catch the obvious stuff.
  • Week 3: 10 evals. You've added regression tests for bugs you found.
  • Month 2: 20 evals. You start adding capability tests for new features.
  • Month 3: You notice your agents are more reliable than they've ever been - and you can prove it with numbers.

As your agents take on more work and run more autonomously, your eval suite grows with them. It's the only way to trust what you can't watch.


References and further reading

These are the resources I've been studying while building this into my own workflow:


*These are notes from my own learning and implementation process. I'm actively building this into my workflow and will share what works and what doesn't as I go. If you're building agents and want to compare notes, I'd love to hear what you're measuring.*

Enjoyed this?

Carol Ships: building, shipping, figuring it out.

Have another workaround to share?

Start the thread below!

Comments

No comments yet. Be the first to share your thoughts!