Evaluating output quality

"It looked good the three times I tried it" isn't evidence. Outputs vary, prompts change and models get updated. An evaluation (eval) tells you whether a change made things better or worse, with numbers.

Start with an eval set: real inputs, each with what a good output is or how to judge one. Include the common cases, the awkward ones (empty input, two questions at once, another language) and every failure you've seen in real use. A few dozen good examples beat a thousand made-up ones; grow it over time.

Use the cheapest check that works. Code checks come first: is it valid JSON, is the label one of the allowed ones, does the number match within a tolerance, does it contain the required phrase? They're fast, free and never change their mind.

For open-ended answers, a second model can grade the first against a written rubric ("Does it answer the question? Does it use only the policy? Is it under 80 words?"). Model graders have biases: they can favour longer answers, the first option shown, or text like their own. Check the grader against a sample of human judgements before trusting it, and prefer pass or fail per criterion over a 1-to-10 score.

expected = ["refund", "delivery", "other", "refund", "delivery"]
got      = ["refund", "delivery", "refund", "refund", "other"]

right = sum(e == g for e, g in zip(expected, got))
print(right, "of", len(expected))
print(f"{right / len(expected):.0%}")

Pick the metric that matches the cost of a mistake. When one class is rare, accuracy hides failure: if 2% of tickets are urgent, a model that never says "urgent" is 98% accurate and useless. Recall (of the truly urgent ones, how many did it catch?) and precision (of the ones it flagged, how many were urgent?) tell you what you need.

Run the evals on every prompt or model change, like tests, and keep the scores so you can see the trend. In production, log inputs and outputs (with care for private data), review a sample regularly, and add new failures to the eval set.