Six tests. Real local runs. Find out what each model can do.
Can it finish a task in a real terminal?
2× GB10 (DGX Spark head + ASUS GX10 worker) · 256GB total
The paper separates execution, coherence and verification failures. A task can fail because the environment or tools break, because the agent loses track of the goal, or because it fails to check its work.
Terminal-Bench research ↗Can it fix an issue in an existing codebase?
2× GB10 (DGX Spark head + ASUS GX10 worker) · 256GB total
SWE-bench Verified is a 500-issue subset that human engineers marked as solvable. A resolved score is a patch that passes the repository tests. Agent, harness, and attempt policy belong with the cite. This is not SWE-bench Pro.
SWE-bench paper ↗Can it turn a problem into working code?
GB10 · 128GB
Time-indexed programming problems support generation, repair, execution and prediction tests. Version, time window and scenario matter when interpreting a result.
LiveCodeBench paper ↗Can it reason through difficult science?
2× GB10 (DGX Spark head + ASUS GX10 worker) · 256GB total
Expert-written biology, chemistry and physics questions measure difficult scientific question answering. They do not establish practical experimental or research competence.
GPQA paper ↗Can it actually finish the office workflow?
2× GB10 (DGX Spark head + ASUS GX10 worker) · 256GB total
Will it follow the precise instructions?
2× GB10 (DGX Spark head + ASUS GX10 worker) · 256GB total
The benchmark tests generalization to 58 unfamiliar precise constraints. Strict and loose criteria answer different questions and must be labeled separately.
IFBench paper ↗