Published full-model reference, then measured local recipes.
| Recipe | GPQA | Terminal | LCB | IFBench | Auto | Verified |
|---|---|---|---|---|---|---|
| DeepSeek reference / publishedPublished reference | — | 82.7 | — | — | 25.1 | 79.0 |
| 75.8 | — | 90.3 | 65.3 | 19.3 | 64.0 |
The published full-model reference is the top row — or the dashed gold tick on Graphic — not rank #1. A blank cell, missing tick, or missing bead has no recorded cite. Scores are never invented.
Weight pin not recorded on this receipt. No download URL.
Recipe JSON ↓ON THIS SETUP · MEASURED BY LOCALSONLY
Local GB10 · 128GB · llama.cpp
Solve programming problems with executable tests.
Version 6.0 · 93 tasks
Generated solutions are checked by executable tests. The benchmark version and problem window matter; this is different from maintaining an existing repository.
500 human-verified repository issues. Public-comparable headline for repo software engineering. Agent and harness belong on the cite.
swe-bench-verified · 100 tasks
Repository tests verify issue resolution. Task subsets should only be compared when their selection and evaluation protocol match.
Follow verifiable instruction constraints.
Loose prompt grading · 300 tasks
This receipt uses loose prompt-level grading. It is not the strict score, and the local and published protocols are not matched.
Sales, marketing, operations, support, finance and HR workflows.
Local evaluation · 300 tasks
A workflow is complete only when every assertion passes. Mean reward gives partial credit for progress; it is not the share of jobs finished. Task sets and evaluation budgets can differ between receipts.
Mean reward 42.0% · partial credit, not completed jobs.
Graduate-level science questions.
Diamond · version 1.0 · 198 tasks
Credit comes from answering the science question correctly. Multiple-choice accuracy does not establish research or experimental competence.
BEYOND THE HEADLINE
Each block represents one recorded task, in receipt order from left to right, then top to bottom. Fill shows the outcome; work-category colors are not used here.
Tested by LocalsOnly · UD-IQ3_XXS · GB10 · 128GB. Partial progress still requires further work before a task is complete.
Inspect individual task outcomes →Completed the recorded checks.
1 / 1 assertions · 23 steps
Follow this exact task ↗sales.priority_selection · ID 4
sales · 1 / 1 assertions · 23 steps / 63 tools · passed
The attempt stopped before completion. The record does not establish why.
0 / 3 assertions · 28 steps
Follow this exact task ↗sales.multi_hop_lookup · ID 1
sales · 0 / 3 assertions · 28 steps / 63 tools · aborted_task
Context, sampling, harness settings and dataset versions can vary by benchmark. Each evidence link above contains its run-specific configuration.