https://anaivebidder.com/tags/benchmarks/Recent content in Benchmarks on A naive bidderen-usTue, 10 Nov 2026 06:00:00 +0100https://anaivebidder.com/posts/llm-reconciliation-benchmark/Tue, 10 Nov 2026 06:00:00 +0100Ruben Afonsohttps://anaivebidder.com/posts/llm-reconciliation-benchmark/<p>Can an AI agent open an Outlook email, follow it into an Excel workbook, query a database it has never seen and come back with the right number?</p> <p>In my last post, <a href="https://anaivebidder.com/posts/build-your-own-llm-benchmark/">How to Build your Own LLM Benchmark</a>, I shared tips for building your own. I kept two things back: the model names and the benchmark itself. Today you get both.</p> <p>Meet <a href="https://github.com/rubenafo/finrec-bm"><strong>finrec-bm</strong></a>, my benchmark for financial reconciliation, now on GitHub. It hands AI agents a number to verify and a pile of evidence: emails, spreadsheets, screenshots and a database. The answer has to be right to the cent, with the paper trail to prove it.</p>https://anaivebidder.com/posts/build-your-own-llm-benchmark/Mon, 26 Oct 2026 12:05:13 +0100Ruben Afonsohttps://anaivebidder.com/posts/build-your-own-llm-benchmark/<p>Every LLM launch comes with the same slide: a wall of benchmark scores, each a point or two above last month&rsquo;s. Those numbers matter, but they measure someone else&rsquo;s problem, and they make you pay for capabilities you probably don&rsquo;t need.</p> <p>You write your code, you own the unit tests. You leverage an LLM, you should own the benchmark too.</p> <p>Writing the first cases is easy. The hard part is structuring them so the benchmark keeps evolving as cases pile up, new models ship every month and someone asks why a model failed a case. This post covers building your own benchmark and the harness that runs it.</p>