Building agents is easy. Knowing if they work is the hard part.

I'm running evals on my agents now. Actual tests with graders, regression cases, and a scoring system that tells me if a change made things better or worse.

The shift wasn't technical. It was realizing that building something and trusting it works are two completely different things. I was doing the first. I wasn't doing the second.

The first time I ran evals on one of my agents, they caught a bug I never would have found by reading outputs. A classification that silently failed on edge cases. The agent looked fine. The eval proved it wasn't.

Now every change gets measured. Every bug becomes a regression test. The score either goes up or it doesn't, and I know exactly which change caused it.

Building agents is the exciting part. Measuring them is the part that makes them reliable.