LocalsOnlyevaluationsHow to read this
← Model evidence

LocalsOnly · measured2026-09-03

SWE-bench Pro

Pro evaluation · 50 tasks. Compare only matched task sets and protocols.

Qwen 3.8 27B · UD-Q4_K_XL · GB10 · 128GB · llama.cpp

● Tested by LocalsOnly
WALL-CLOCK TIME32.5 h
INPUT TOKENS34,597,811

Recorded effort for this run. Wall time and token counts are not a dollar cost; hardware power and utilization also matter.

Reported score20.0%10 of 50 tasks passed
Throughput10.8 t/s

Tokens per second · see measurement basis

Concurrency1

As recorded on this receipt

32 infrastructure outcomes are recorded. Failure counts may overlap with task outcomes.
Run configuration & provenance +
Context
32,768
Temperature
0
Wall time
1952.1 minutes
Revision
4ca720788d1e01f1bff70c033e0d0028fd02e502
Reasoning
off
Agent version
2.0.0
Output token limit
4,096
Dataset revision
sha256:88411d32ff27e53a4c1a7e29f0c2aeba180c8e5d60f221cab5ed56325f33549d
Harness host
Mac mini · 64GB
Model host
ASUS GX10 · 128GB
Input tokens
34,597,811
Output tokens
398,236
Cached input tokens
33,294,322
Agent turn / step limit
50
Concurrency
1
Weight file size
17.56 GB

Throughput measurement: Measurement method not separately reported. This should not be treated as standardized decode-only speed.

terminus-2

Run ID: measured-spark-1x-unsloth-ud-q4_k_xl-swe-bench-pro-n50-2026-09-03
Tasks

Task explorer

Passed tasks first, then other outcomes grouped by recorded issue. Use the filters to inspect a particular outcome.

50 task records
10 Passed40 Not passed

All 50 recorded tasks, before filters or pagination. 25 of 50 matching tasks shown below. Recorded issues can also occur on scored tasks.

PassedScale ai/instance ansible ansible a7d2a4e03209cff1e97e59fd54bb2b05fdbdbec6 v0f01c69f1e2528b935359cfe578530722bca2c59+

Task ID: scale-ai/instance_ansible__ansible-a7d2a4e03209cff1e97e59fd54bb2b05fdbdbec6-v0f01c69f1e2528b935359cfe578530722bca2c59
Recorded name: scale-ai/instance_ansible__ansible-a7d2a4e03209cff1e97e59fd54bb2b05fdbdbec6-v0f01c69f1e2528b935359cfe578530722bca2c59

Reward: 1

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
PassedScale ai/instance internetarchive openlibrary 3f7db6bbbcc7c418b3db72d157c6aed1d45b2ccf v430f20c722405e462d9ef44dee7d34c41e76fe7a+

Task ID: scale-ai/instance_internetarchive__openlibrary-3f7db6bbbcc7c418b3db72d157c6aed1d45b2ccf-v430f20c722405e462d9ef44dee7d34c41e76fe7a
Recorded name: scale-ai/instance_internetarchive__openlibrary-3f7db6bbbcc7c418b3db72d157c6aed1d45b2ccf-v430f20c722405e462d9ef44dee7d34c41e76fe7a

Reward: 1

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
PassedScale ai/instance internetarchive openlibrary 3f580a5f244c299d936d73d9e327ba873b6401d9 v0f5aece3601a5b4419f7ccec1dbda2071be28ee4+

Task ID: scale-ai/instance_internetarchive__openlibrary-3f580a5f244c299d936d73d9e327ba873b6401d9-v0f5aece3601a5b4419f7ccec1dbda2071be28ee4
Recorded name: scale-ai/instance_internetarchive__openlibrary-3f580a5f244c299d936d73d9e327ba873b6401d9-v0f5aece3601a5b4419f7ccec1dbda2071be28ee4

Reward: 1

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
PassedScale ai/instance internetarchive openlibrary 7f7e53aa4cf74a4f8549a5bcd4810c527e2f6d7e v13642507b4fc1f8d234172bf8129942da2c2ca26+

Task ID: scale-ai/instance_internetarchive__openlibrary-7f7e53aa4cf74a4f8549a5bcd4810c527e2f6d7e-v13642507b4fc1f8d234172bf8129942da2c2ca26
Recorded name: scale-ai/instance_internetarchive__openlibrary-7f7e53aa4cf74a4f8549a5bcd4810c527e2f6d7e-v13642507b4fc1f8d234172bf8129942da2c2ca26

Reward: 1

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
PassedScale ai/instance internetarchive openlibrary 25858f9f0c165df25742acf8309ce909773f0cdd v13642507b4fc1f8d234172bf8129942da2c2ca26+

Task ID: scale-ai/instance_internetarchive__openlibrary-25858f9f0c165df25742acf8309ce909773f0cdd-v13642507b4fc1f8d234172bf8129942da2c2ca26
Recorded name: scale-ai/instance_internetarchive__openlibrary-25858f9f0c165df25742acf8309ce909773f0cdd-v13642507b4fc1f8d234172bf8129942da2c2ca26

Reward: 1

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
PassedScale ai/instance internetarchive openlibrary c05ccf2cd8baa81609434e0e35c4a63bc0da5a25 v0f5aece3601a5b4419f7ccec1dbda2071be28ee4+

Task ID: scale-ai/instance_internetarchive__openlibrary-c05ccf2cd8baa81609434e0e35c4a63bc0da5a25-v0f5aece3601a5b4419f7ccec1dbda2071be28ee4
Recorded name: scale-ai/instance_internetarchive__openlibrary-c05ccf2cd8baa81609434e0e35c4a63bc0da5a25-v0f5aece3601a5b4419f7ccec1dbda2071be28ee4

Reward: 1

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
PassedScale ai/instance navidrome navidrome 29b7b740ce469201af0a0510f3024adc93ef4c8e+

Task ID: scale-ai/instance_navidrome__navidrome-29b7b740ce469201af0a0510f3024adc93ef4c8e
Recorded name: scale-ai/instance_navidrome__navidrome-29b7b740ce469201af0a0510f3024adc93ef4c8e

Reward: 1

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
PassedScale ai/instance protonmail webclients 32ff10999a06455cb2147f6873d627456924ae13+

Task ID: scale-ai/instance_protonmail__webclients-32ff10999a06455cb2147f6873d627456924ae13
Recorded name: scale-ai/instance_protonmail__webclients-32ff10999a06455cb2147f6873d627456924ae13

Reward: 1

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
PassedScale ai/instance protonmail webclients a6e6f617026794e7b505d649d2a7a9cdf17658c8+

Task ID: scale-ai/instance_protonmail__webclients-a6e6f617026794e7b505d649d2a7a9cdf17658c8
Recorded name: scale-ai/instance_protonmail__webclients-a6e6f617026794e7b505d649d2a7a9cdf17658c8

Reward: 1

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
PassedScale ai/instance qutebrowser qutebrowser 1a9e74bfaf9a9db2a510dc14572d33ded6040a57 v2ef375ac784985212b1805e1d0431dc8f1b3c171+

Task ID: scale-ai/instance_qutebrowser__qutebrowser-1a9e74bfaf9a9db2a510dc14572d33ded6040a57-v2ef375ac784985212b1805e1d0431dc8f1b3c171
Recorded name: scale-ai/instance_qutebrowser__qutebrowser-1a9e74bfaf9a9db2a510dc14572d33ded6040a57-v2ef375ac784985212b1805e1d0431dc8f1b3c171

Reward: 1

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance ansible ansible 4c5ce5a1a9e79a845aff4978cfeb72a0d4ecf7d6 v1055803c3a812189a1133297f7f5468579283f86+

Task ID: scale-ai/instance_ansible__ansible-4c5ce5a1a9e79a845aff4978cfeb72a0d4ecf7d6-v1055803c3a812189a1133297f7f5468579283f86
Recorded name: scale-ai/instance_ansible__ansible-4c5ce5a1a9e79a845aff4978cfeb72a0d4ecf7d6-v1055803c3a812189a1133297f7f5468579283f86

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance ansible ansible 3889ddeb4b780ab4bac9ca2e75f8c1991bcabe83 v0f01c69f1e2528b935359cfe578530722bca2c59+

Task ID: scale-ai/instance_ansible__ansible-3889ddeb4b780ab4bac9ca2e75f8c1991bcabe83-v0f01c69f1e2528b935359cfe578530722bca2c59
Recorded name: scale-ai/instance_ansible__ansible-3889ddeb4b780ab4bac9ca2e75f8c1991bcabe83-v0f01c69f1e2528b935359cfe578530722bca2c59

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance ansible ansible d6d2251929c84c3aa883bad7db0f19cc9ff0339e v30a923fb5c164d6cd18280c02422f75e611e8fb2+

Task ID: scale-ai/instance_ansible__ansible-d6d2251929c84c3aa883bad7db0f19cc9ff0339e-v30a923fb5c164d6cd18280c02422f75e611e8fb2
Recorded name: scale-ai/instance_ansible__ansible-d6d2251929c84c3aa883bad7db0f19cc9ff0339e-v30a923fb5c164d6cd18280c02422f75e611e8fb2

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance flipt io flipt 2ca5dfb3513e4e786d2b037075617cccc286d5c3+

Task ID: scale-ai/instance_flipt-io__flipt-2ca5dfb3513e4e786d2b037075617cccc286d5c3
Recorded name: scale-ai/instance_flipt-io__flipt-2ca5dfb3513e4e786d2b037075617cccc286d5c3

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance flipt io flipt 2eac0df47b5ecc8bb05002d80383ceb08ab3620a+

Task ID: scale-ai/instance_flipt-io__flipt-2eac0df47b5ecc8bb05002d80383ceb08ab3620a
Recorded name: scale-ai/instance_flipt-io__flipt-2eac0df47b5ecc8bb05002d80383ceb08ab3620a

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance flipt io flipt 5aef5a14890aa145c22d864a834694bae3a6f112+

Task ID: scale-ai/instance_flipt-io__flipt-5aef5a14890aa145c22d864a834694bae3a6f112
Recorded name: scale-ai/instance_flipt-io__flipt-5aef5a14890aa145c22d864a834694bae3a6f112

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance flipt io flipt 9f8127f225a86245fa35dca4885c2daef824ee55+

Task ID: scale-ai/instance_flipt-io__flipt-9f8127f225a86245fa35dca4885c2daef824ee55
Recorded name: scale-ai/instance_flipt-io__flipt-9f8127f225a86245fa35dca4885c2daef824ee55

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance flipt io flipt 518ec324b66a07fdd95464a5e9ca5fe7681ad8f9+

Task ID: scale-ai/instance_flipt-io__flipt-518ec324b66a07fdd95464a5e9ca5fe7681ad8f9
Recorded name: scale-ai/instance_flipt-io__flipt-518ec324b66a07fdd95464a5e9ca5fe7681ad8f9

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance flipt io flipt a0cbc0cb65ae601270bdbe3f5313e2dfd49c80e4+

Task ID: scale-ai/instance_flipt-io__flipt-a0cbc0cb65ae601270bdbe3f5313e2dfd49c80e4
Recorded name: scale-ai/instance_flipt-io__flipt-a0cbc0cb65ae601270bdbe3f5313e2dfd49c80e4

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance flipt io flipt b4bb5e13006a729bc0eed8fe6ea18cff54acdacb+

Task ID: scale-ai/instance_flipt-io__flipt-b4bb5e13006a729bc0eed8fe6ea18cff54acdacb
Recorded name: scale-ai/instance_flipt-io__flipt-b4bb5e13006a729bc0eed8fe6ea18cff54acdacb

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance flipt io flipt b433bd05ce405837804693bebd5f4b88d87133c8+

Task ID: scale-ai/instance_flipt-io__flipt-b433bd05ce405837804693bebd5f4b88d87133c8
Recorded name: scale-ai/instance_flipt-io__flipt-b433bd05ce405837804693bebd5f4b88d87133c8

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance flipt io flipt e2bd19dafa7166c96b082fb2a59eb54b4be0d778+

Task ID: scale-ai/instance_flipt-io__flipt-e2bd19dafa7166c96b082fb2a59eb54b4be0d778
Recorded name: scale-ai/instance_flipt-io__flipt-e2bd19dafa7166c96b082fb2a59eb54b4be0d778

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance flipt io flipt e594593dae52badf80ffd27878d2275c7f0b20e9+

Task ID: scale-ai/instance_flipt-io__flipt-e594593dae52badf80ffd27878d2275c7f0b20e9
Recorded name: scale-ai/instance_flipt-io__flipt-e594593dae52badf80ffd27878d2275c7f0b20e9

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance future architect vuls 457a3a9627fb9a0800d0aecf1d4713fb634a9011+

Task ID: scale-ai/instance_future-architect__vuls-457a3a9627fb9a0800d0aecf1d4713fb634a9011
Recorded name: scale-ai/instance_future-architect__vuls-457a3a9627fb9a0800d0aecf1d4713fb634a9011

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Not passedScale ai/instance future architect vuls 1832b4ee3a20177ad313d806983127cb6e53f5cf+

Task ID: scale-ai/instance_future-architect__vuls-1832b4ee3a20177ad313d806983127cb6e53f5cf
Recorded name: scale-ai/instance_future-architect__vuls-1832b4ee3a20177ad313d806983127cb6e53f5cf

Reward: 0

Recorded issue: No failure explanation in this task record.

The receipt contains outcome metadata. Full prompts and execution transcripts are not included in this view.
Showing 25 of 50 tasks