https://blog.defectiv.es/en/tags/benchmark/Recent content in benchmark on Gaia Lab · BlogHugoen-GBWed, 23 Sep 2026 09:00:00 +0200https://blog.defectiv.es/en/posts/medir-llm-cuantizados-gguf-bfcl-bigcodebench-ruler/Tue, 22 Sep 2026 12:45:00 +0200https://blog.defectiv.es/en/posts/medir-llm-cuantizados-gguf-bfcl-bigcodebench-ruler/<p>Third instalment of <strong>Cluster X-ray</strong>. It is the best example of what the <a href="https://blog.defectiv.es/en/posts/que-corre-de-verdad-en-nuestro-cluster/">overview post</a> said: the cluster today is above all a <strong>measurement laboratory</strong>. Not a single parameter is trained here.</p> <h2 id="the-question">The question <a class="hanchor" href="#the-question" aria-label="Enlace a esta sección">#</a></h2> <p>Open models are distributed quantized: the same weights compressed to 8, 6, 5, 4, 3 or 2 bits so they fit on a small card or a laptop. The community publishes hundreds of variants, but rarely with a comparable measure of what is lost. The line asks <strong>how well each quantization level of each family really performs on tasks that matter for using the model as a tool</strong>, and at what speed and energy cost it does so on each card.</p>