Evals (AI evaluations)

What it is

An eval is a set of test inputs, the qualities a good output should have, and a way to grade the results. OpenAI and Anthropic both document how to define success criteria and build evaluations.

How it works

1. Define success: what a good answer must contain or avoid.
2. Collect representative test cases, including hard and edge cases. Anthropic lists examples such as irrelevant or missing input, overly long input, and ambiguous cases where even humans would disagree.
3. Grade the outputs with code checks, a model grader or human review, and track scores as you change prompts or models.

How many examples? IonBin suggests starting with 20 to 50 real examples. That’s our guidance, not a vendor figure. Anthropic advises favoring volume: more questions with automated grading beat fewer questions graded by hand.

Benefits

• An objective way to compare models and prompts.
• Catches regressions when vendors update models.
• Builds the trust needed to automate a step.
• The set compounds (IonBin’s view). Every failure you catch becomes a test case. Over a few months you build a private benchmark of your own work that shows quickly whether a new model is better for you. Incrementality tests play the same role for marketing.

Limits and risks

• Bad test sets give false confidence.
• Model graders have their own biases, so spot-check them against human judgment.
• Evals need upkeep as the task changes.
• Christopher Penn warns against judging models on generic benchmarks. Test the tasks you actually do.