Published full-model reference, then measured local recipes.
| Recipe | GPQA | Terminal | IFBench | Auto | Verified |
|---|---|---|---|---|---|
| 88.4 | 48.3 | — | 41.7 | 94.0 | |
| — | — | 66.0 | — | — |
The published full-model reference is the top row — or the dashed gold tick on Graphic — not rank #1. A blank cell, missing tick, or missing bead has no recorded cite. Scores are never invented.
Weight pin not recorded on this receipt. No download URL.
Recipe JSON ↓ON THIS SETUP · MEASURED BY LOCALSONLY
Local GB10 · 128GB · llama.cpp
500 human-verified repository issues. Public-comparable headline for repo software engineering. Agent and harness belong on the cite.
swe-bench-verified · 100 tasks
Repository tests verify issue resolution. Task subsets should only be compared when their selection and evaluation protocol match.
Multi-step tasks in a terminal environment.
Version 2.1 · 89 tasks
The task must pass its verification in a terminal environment. Tools, harness, context budget and execution failures all contribute to the result.
Sales, marketing, operations, support, finance and HR workflows.
Local evaluation · 300 tasks
A workflow is complete only when every assertion passes. Mean reward gives partial credit for progress; it is not the share of jobs finished. Task sets and evaluation budgets can differ between receipts.
Mean reward 72.2% · partial credit, not completed jobs.
Graduate-level science questions.
Diamond · version 1.0 · 198 tasks
Credit comes from answering the science question correctly. Multiple-choice accuracy does not establish research or experimental competence.
BEYOND THE HEADLINE
Each block represents one recorded task, in receipt order from left to right, then top to bottom. Fill shows the outcome; work-category colors are not used here.
Tested by LocalsOnly · UD-IQ3_XXS · GB10 · 128GB. Partial progress still requires further work before a task is complete.
Inspect individual task outcomes →Completed the recorded checks.
4 / 4 assertions · 10 steps
Follow this exact task ↗sales.update_contact_phone · ID 1
sales · 4 / 4 assertions · 10 steps / 15 tools · passed
Did not complete the recorded checks. The record does not establish a cause.
4 / 8 assertions · 16 steps
Follow this exact task ↗sales.advance_opportunity_stage · ID 2
sales · 4 / 8 assertions · 16 steps / 34 tools · not passed
Context, sampling, harness settings and dataset versions can vary by benchmark. Each evidence link above contains its run-specific configuration.