https://anaivebidder.com/tags/testing/Recent content in Testing on A naive bidderen-usMon, 26 Oct 2026 12:05:13 +0100https://anaivebidder.com/posts/build-your-own-llm-benchmark/Mon, 26 Oct 2026 12:05:13 +0100Ruben Afonsohttps://anaivebidder.com/posts/build-your-own-llm-benchmark/<p>Every LLM launch comes with the same slide: a wall of benchmark scores, each a point or two above last month&rsquo;s. Those numbers matter, but they measure someone else&rsquo;s problem, and they make you pay for capabilities you probably don&rsquo;t need.</p> <p>You write your code, you own the unit tests. You leverage an LLM, you should own the benchmark too.</p> <p>Writing the first cases is easy. The hard part is structuring them so the benchmark keeps evolving as cases pile up, new models ship every month and someone asks why a model failed a case. This post covers building your own benchmark and the harness that runs it.</p>