Offline dataset misses failures that production finds: a judge on live traffic

On live traffic, the value of an LLM judge is not in the scoring but in what you do with flagged answers. The five-step loop: calibrate on human-labeled data, pick low scores, confirm the failure, add it to evals, then fix and re-run.