Every AI feature we ship starts the same way: before the feature exists, the thing that measures it does.
That sounds backwards. It is much more satisfying to wire a model into a product, watch it answer well, and call it done. The problem shows up later, and always in the same shape: someone changes a prompt, upgrades a model, adds a retrieval step — and nobody can say whether the product got better or worse. The demo still looks fine. That is the trap.
What an eval actually is
An eval is not a benchmark score. It is a set of cases drawn from your real problem, each with a judgement about what a good answer looks like. Fifty well-chosen cases beat five thousand generic ones, because the fifty encode what your users actually ask and what your business considers wrong.
The useful property is boring: it turns “the AI seems worse today” into a number you can put in a pull request.
Why it goes first
Writing the harness first forces a conversation that is easy to skip — what counts as correct here? Teams often discover at this point that they disagree, and it is far cheaper to discover that before the feature is built than after it is in front of customers.
It also changes what shipping means. A model upgrade stops being a leap of faith and becomes a diff: these twelve cases improved, these two regressed, here is the trade. That is an engineering decision rather than a vibe.
The cost
It is real. Building a decent harness for a new feature is typically a few days, and it is unglamorous work that produces nothing a customer can see. We do it anyway, because the alternative — discovering a regression through a support ticket — costs more, and it costs it at the worst possible moment.
The version of this we believe: if a model is doing work that matters, it should be measured like any other dependency. Version it, test it, and know when it changes.