Every run, instance by instance.
Choose a run and step through it one instance at a time. Each instance shows the learner at work up to the verdict, then what it did once it knew the outcome and what it had kept by then.
Pick a run
OpenClaw, as shipped
Every learning mechanism the harness ships switched on.
- Stream, budget used
- 26.8 solved 89%
- Epilogue, budget used
- 28.0 solved 88%
- Model spend
- $7.61 2,413 calls
All 257 runs
Budget used is the share of the cap of 16 that an instance used, averaged over the instances, where a wrong answer and a demonstration each cost 1, so lower is better. Solved is the share solved within the cap. On the grid tracks the stream is the order’s 200 instances and the epilogue the 25 held-out tasks after it, which a fresh run serves alone. ARC-AGI-3 charges a level the actions it took against the human baseline, with a cap of five times it.
| Stream | Epilogue or fresh | |||||||
|---|---|---|---|---|---|---|---|---|
| Preset | Order | Track | ||||||
| Stateless floor | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 49.0 | 56% | 47.5 | 52% | $1.53 |
| Stateless floor | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 47.2 | 59% | 51.3 | 40% | $1.40 |
| Stateless floor | GPT-6 Luna, low | Random | ARC-AGI-1 | 45.8 | 63% | 47.5 | 48% | $1.41 |
| Stateless floor | GPT-6 Luna, low | Fresh | ARC-AGI-1 | – | – | 53.3 | 40% | $0.18 |
| Stateless floor | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 33.3 | 83% | 31.0 | 84% | $2.06 |
| Stateless floor | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 32.8 | 84% | 26.8 | 84% | $1.85 |
| Stateless floor | GPT-6 Luna, medium | Random | ARC-AGI-1 | 32.6 | 81% | 27.0 | 84% | $1.95 |
| Stateless floor | GPT-6 Luna, medium | Fresh | ARC-AGI-1 | – | – | 27.5 | 84% | $0.24 |
| Stateless floor | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 26.9 | 88% | 27.5 | 88% | $2.49 |
| Stateless floor | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 26.8 | 90% | 25.5 | 88% | $2.19 |
| Stateless floor | GPT-6 Luna, high | Random | ARC-AGI-1 | 25.4 | 90% | 26.0 | 84% | $2.18 |
| Stateless floor | GPT-6 Luna, high | Fresh | ARC-AGI-1 | – | – | 22.3 | 92% | $0.30 |
| Demo-cache floor | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 38.1 | 63% | 47.8 | 48% | $1.60 |
| Demo-cache floor | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 38.8 | 62% | 49.0 | 48% | $1.52 |
| Demo-cache floor | GPT-6 Luna, low | Random | ARC-AGI-1 | 37.7 | 62% | 46.3 | 52% | $1.52 |
| Demo-cache floor | GPT-6 Luna, low | Fresh | ARC-AGI-1 | – | – | 50.0 | 48% | $0.17 |
| Demo-cache floor | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 25.3 | 81% | 27.5 | 80% | $2.17 |
| Demo-cache floor | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 23.6 | 84% | 30.8 | 80% | $2.01 |
| Demo-cache floor | GPT-6 Luna, medium | Random | ARC-AGI-1 | 27.4 | 79% | 31.3 | 76% | $2.29 |
| Demo-cache floor | GPT-6 Luna, medium | Fresh | ARC-AGI-1 | – | – | 32.3 | 80% | $0.28 |
| Demo-cache floor | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 17.4 | 92% | 27.5 | 88% | $2.41 |
| Demo-cache floor | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 16.3 | 91% | 22.0 | 92% | $2.35 |
| Demo-cache floor | GPT-6 Luna, high | Random | ARC-AGI-1 | 14.7 | 92% | 25.0 | 88% | $2.14 |
| Demo-cache floor | GPT-6 Luna, high | Fresh | ARC-AGI-1 | – | – | 23.5 | 88% | $0.33 |
| Program library | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 15.7 | 82% | 33.5 | 76% | $0.77 |
| Program library | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 16.5 | 84% | 34.3 | 68% | $0.80 |
| Program library | GPT-6 Luna, low | Random | ARC-AGI-1 | 17.2 | 79% | 36.8 | 68% | $0.80 |
| Program library | GPT-6 Luna, low | Fresh | ARC-AGI-1 | – | – | 34.3 | 68% | $0.14 |
| Program library | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 11.7 | 94% | 22.0 | 88% | $1.03 |
| Program library | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 10.1 | 96% | 22.3 | 88% | $0.90 |
| Program library | GPT-6 Luna, medium | Random | ARC-AGI-1 | 8.9 | 96% | 19.3 | 92% | $0.86 |
| Program library | GPT-6 Luna, medium | Fresh | ARC-AGI-1 | – | – | 20.5 | 88% | $0.18 |
| Program library | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 7.8 | 98% | 16.8 | 96% | $1.06 |
| Program library | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 7.9 | 97% | 20.3 | 100% | $1.30 |
| Program library | GPT-6 Luna, high | Random | ARC-AGI-1 | 6.8 | 98% | 16.0 | 96% | $1.13 |
| Program library | GPT-6 Luna, high | Fresh | ARC-AGI-1 | – | – | 19.3 | 96% | $0.31 |
| Program library, nothing kept | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 31.0 | 76% | 36.8 | 60% | $1.15 |
| Program library, nothing kept | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 29.1 | 76% | 34.5 | 64% | $1.04 |
| Program library, nothing kept | GPT-6 Luna, low | Random | ARC-AGI-1 | 29.1 | 77% | 36.3 | 64% | $1.10 |
| Program library, nothing kept | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 23.7 | 91% | 22.0 | 88% | $1.60 |
| Program library, nothing kept | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 24.8 | 92% | 21.3 | 88% | $1.65 |
| Program library, nothing kept | GPT-6 Luna, medium | Random | ARC-AGI-1 | 23.2 | 92% | 23.3 | 88% | $1.55 |
| Program library, nothing kept | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 21.1 | 95% | 20.3 | 100% | $2.05 |
| Program library, nothing kept | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 21.8 | 93% | 20.8 | 92% | $1.98 |
| Program library, nothing kept | GPT-6 Luna, high | Random | ARC-AGI-1 | 19.8 | 98% | 17.5 | 100% | $1.84 |
| Prime Agent, as shipped | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 42.8 | 71% | 46.3 | 56% | $1.77 |
| Prime Agent, as shipped | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 43.0 | 71% | 44.8 | 60% | $1.61 |
| Prime Agent, as shipped | GPT-6 Luna, low | Random | ARC-AGI-1 | 40.9 | 73% | 43.8 | 60% | $1.63 |
| Prime Agent, as shipped | GPT-6 Luna, low | Fresh | ARC-AGI-1 | – | – | 42.0 | 60% | $0.18 |
| Prime Agent, as shipped | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 23.6 | 95% | 24.5 | 80% | $2.05 |
| Prime Agent, as shipped | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 24.0 | 94% | 25.0 | 92% | $1.91 |
| Prime Agent, as shipped | GPT-6 Luna, medium | Random | ARC-AGI-1 | 24.0 | 94% | 24.0 | 88% | $2.04 |
| Prime Agent, as shipped | GPT-6 Luna, medium | Fresh | ARC-AGI-1 | – | – | 22.3 | 88% | $0.24 |
| Prime Agent, as shipped | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 20.3 | 97% | 16.5 | 96% | $1.97 |
| Prime Agent, as shipped | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 18.7 | 97% | 27.0 | 84% | $2.04 |
| Prime Agent, as shipped | GPT-6 Luna, high | Random | ARC-AGI-1 | 21.6 | 97% | 20.0 | 100% | $2.04 |
| Prime Agent, as shipped | GPT-6 Luna, high | Fresh | ARC-AGI-1 | – | – | 22.0 | 88% | $0.35 |
| Prime Agent, memory wiped | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 42.1 | 71% | 43.3 | 60% | $1.70 |
| Prime Agent, memory wiped | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 44.0 | 71% | 41.3 | 60% | $1.69 |
| Prime Agent, memory wiped | GPT-6 Luna, low | Random | ARC-AGI-1 | 41.8 | 72% | 40.5 | 68% | $1.75 |
| Prime Agent, memory wiped | GPT-6 Luna, low | Fresh | ARC-AGI-1 | – | – | 39.8 | 64% | $0.17 |
| Prime Agent, memory wiped | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 25.1 | 92% | 25.0 | 84% | $2.12 |
| Prime Agent, memory wiped | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 23.7 | 93% | 19.5 | 96% | $1.84 |
| Prime Agent, memory wiped | GPT-6 Luna, medium | Random | ARC-AGI-1 | 23.0 | 95% | 22.3 | 92% | $1.97 |
| Prime Agent, memory wiped | GPT-6 Luna, medium | Fresh | ARC-AGI-1 | – | – | 17.8 | 100% | $0.24 |
| Prime Agent, memory wiped | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 20.5 | 98% | 19.8 | 96% | $2.13 |
| Prime Agent, memory wiped | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 20.1 | 97% | 22.0 | 92% | $1.96 |
| Prime Agent, memory wiped | GPT-6 Luna, high | Random | ARC-AGI-1 | 19.5 | 97% | 21.3 | 96% | $2.16 |
| Prime Agent, memory wiped | GPT-6 Luna, high | Fresh | ARC-AGI-1 | – | – | 23.3 | 88% | $0.46 |
| Prime Agent, asked to store | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 39.6 | 75% | 38.3 | 68% | $2.33 |
| Prime Agent, asked to store | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 37.6 | 75% | 39.0 | 64% | $2.09 |
| Prime Agent, asked to store | GPT-6 Luna, low | Random | ARC-AGI-1 | 40.0 | 74% | 40.0 | 60% | $2.27 |
| Prime Agent, asked to store | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 18.6 | 96% | 22.8 | 92% | $2.64 |
| Prime Agent, asked to store | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 19.4 | 95% | 25.8 | 84% | $2.52 |
| Prime Agent, asked to store | GPT-6 Luna, medium | Random | ARC-AGI-1 | 21.3 | 96% | 29.5 | 84% | $2.80 |
| Prime Agent, asked to store | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 15.3 | 98% | 21.5 | 92% | $2.55 |
| Prime Agent, asked to store | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 10.3 | 98% | 20.3 | 92% | $2.38 |
| Prime Agent, asked to store | GPT-6 Luna, high | Random | ARC-AGI-1 | 14.5 | 99% | 24.0 | 88% | $2.58 |
| Prime Agent, told to keep a program library | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 29.6 | 80% | 39.5 | 60% | $1.92 |
| Prime Agent, told to keep a program library | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 25.5 | 82% | 40.8 | 60% | $1.67 |
| Prime Agent, told to keep a program library | GPT-6 Luna, low | Random | ARC-AGI-1 | 27.8 | 82% | 36.0 | 72% | $1.88 |
| Prime Agent, told to keep a program library | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 12.3 | 97% | 25.5 | 92% | $2.27 |
| Prime Agent, told to keep a program library | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 8.2 | 99% | 19.3 | 96% | $1.62 |
| Prime Agent, told to keep a program library | GPT-6 Luna, medium | Random | ARC-AGI-1 | 9.9 | 98% | 19.5 | 100% | $1.61 |
| Prime Agent, told to keep a program library | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 9.8 | 99% | 20.3 | 96% | $2.37 |
| Prime Agent, told to keep a program library | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 6.3 | 99% | 17.0 | 96% | $1.44 |
| Prime Agent, told to keep a program library | GPT-6 Luna, high | Random | ARC-AGI-1 | 7.9 | 100% | 17.0 | 96% | $1.81 |
| Prime Agent, told to keep a program library, memory wiped | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 41.1 | 76% | 40.0 | 64% | $2.41 |
| Prime Agent, told to keep a program library, memory wiped | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 39.8 | 78% | 35.5 | 72% | $2.25 |
| Prime Agent, told to keep a program library, memory wiped | GPT-6 Luna, low | Random | ARC-AGI-1 | 39.8 | 75% | 41.0 | 64% | $2.30 |
| Prime Agent, told to keep a program library, memory wiped | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 23.4 | 95% | 23.5 | 88% | $3.14 |
| Prime Agent, told to keep a program library, memory wiped | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 22.3 | 96% | 19.8 | 96% | $2.70 |
| Prime Agent, told to keep a program library, memory wiped | GPT-6 Luna, medium | Random | ARC-AGI-1 | 20.2 | 97% | 19.5 | 96% | $2.64 |
| Prime Agent, told to keep a program library, memory wiped | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 20.6 | 96% | 15.3 | 100% | $3.57 |
| Prime Agent, told to keep a program library, memory wiped | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 19.5 | 98% | 19.0 | 92% | $3.26 |
| Prime Agent, told to keep a program library, memory wiped | GPT-6 Luna, high | Random | ARC-AGI-1 | 19.0 | 99% | 20.0 | 92% | $3.19 |
| Hermes Agent, as shipped | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 44.2 | 72% | 54.0 | 44% | $8.79 |
| Hermes Agent, as shipped | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 48.7 | 64% | 53.8 | 44% | $6.86 |
| Hermes Agent, as shipped | GPT-6 Luna, low | Random | ARC-AGI-1 | 45.3 | 71% | 53.3 | 36% | $10.02 |
| Hermes Agent, as shipped | GPT-6 Luna, low | Fresh | ARC-AGI-1 | – | – | 43.8 | 68% | $1.52 |
| Hermes Agent, as shipped | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 29.7 | 89% | 24.8 | 92% | $6.38 |
| Hermes Agent, as shipped | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 25.6 | 90% | 25.3 | 88% | $8.74 |
| Hermes Agent, as shipped | GPT-6 Luna, medium | Random | ARC-AGI-1 | 26.8 | 91% | 23.8 | 84% | $6.62 |
| Hermes Agent, as shipped | GPT-6 Luna, medium | Fresh | ARC-AGI-1 | – | – | 26.3 | 92% | $1.76 |
| Hermes Agent, as shipped | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 21.6 | 92% | 26.0 | 88% | $7.29 |
| Hermes Agent, as shipped | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 22.2 | 94% | 19.5 | 92% | $7.87 |
| Hermes Agent, as shipped | GPT-6 Luna, high | Random | ARC-AGI-1 | 21.4 | 95% | 24.0 | 92% | $6.31 |
| Hermes Agent, as shipped | GPT-6 Luna, high | Fresh | ARC-AGI-1 | – | – | 20.5 | 100% | $1.64 |
| Hermes Agent, memory wiped | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 39.0 | 75% | 38.3 | 64% | $15.34 |
| Hermes Agent, memory wiped | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 41.1 | 73% | 49.5 | 48% | $15.34 |
| Hermes Agent, memory wiped | GPT-6 Luna, low | Random | ARC-AGI-1 | 39.8 | 76% | 51.0 | 44% | $18.42 |
| Hermes Agent, memory wiped | GPT-6 Luna, low | Fresh | ARC-AGI-1 | – | – | 42.0 | 60% | $1.58 |
| Hermes Agent, memory wiped | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 28.0 | 88% | 21.8 | 84% | $15.09 |
| Hermes Agent, memory wiped | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 25.7 | 90% | 25.5 | 88% | $13.67 |
| Hermes Agent, memory wiped | GPT-6 Luna, medium | Random | ARC-AGI-1 | 26.0 | 92% | 25.8 | 84% | $11.02 |
| Hermes Agent, memory wiped | GPT-6 Luna, medium | Fresh | ARC-AGI-1 | – | – | 29.3 | 84% | $2.13 |
| Hermes Agent, memory wiped | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 23.6 | 91% | 22.3 | 92% | $9.95 |
| Hermes Agent, memory wiped | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 22.0 | 94% | 20.3 | 96% | $9.99 |
| Hermes Agent, memory wiped | GPT-6 Luna, high | Random | ARC-AGI-1 | 22.5 | 93% | 21.8 | 92% | $8.39 |
| Hermes Agent, memory wiped | GPT-6 Luna, high | Fresh | ARC-AGI-1 | – | – | 18.3 | 96% | $1.08 |
| Hermes Agent, asked to store | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 37.4 | 76% | 58.0 | 44% | $9.01 |
| Hermes Agent, asked to store | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 41.2 | 71% | 54.5 | 44% | $8.23 |
| Hermes Agent, asked to store | GPT-6 Luna, low | Random | ARC-AGI-1 | 46.1 | 69% | 60.0 | 28% | $13.79 |
| Hermes Agent, asked to store | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 20.7 | 90% | 33.8 | 72% | $6.99 |
| Hermes Agent, asked to store | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 22.7 | 90% | 33.0 | 80% | $5.14 |
| Hermes Agent, asked to store | GPT-6 Luna, medium | Random | ARC-AGI-1 | 17.1 | 93% | 32.3 | 84% | $9.20 |
| Hermes Agent, asked to store | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 16.9 | 95% | 22.0 | 92% | $5.07 |
| Hermes Agent, asked to store | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 19.8 | 95% | 22.8 | 92% | $6.46 |
| Hermes Agent, asked to store | GPT-6 Luna, high | Random | ARC-AGI-1 | 15.9 | 93% | 20.0 | 92% | $7.03 |
| Hermes Agent, told to keep a program library | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 34.4 | 76% | 40.8 | 64% | $12.96 |
| Hermes Agent, told to keep a program library | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 33.8 | 76% | 54.0 | 40% | $12.14 |
| Hermes Agent, told to keep a program library | GPT-6 Luna, low | Random | ARC-AGI-1 | 38.8 | 73% | 48.0 | 56% | $18.78 |
| Hermes Agent, told to keep a program library | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 15.0 | 93% | 24.8 | 88% | $15.90 |
| Hermes Agent, told to keep a program library | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 13.8 | 94% | 25.0 | 84% | $13.37 |
| Hermes Agent, told to keep a program library | GPT-6 Luna, medium | Random | ARC-AGI-1 | 14.8 | 94% | 25.8 | 84% | $12.44 |
| Hermes Agent, told to keep a program library | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 12.9 | 96% | 18.3 | 92% | $10.20 |
| Hermes Agent, told to keep a program library | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 11.8 | 96% | 24.8 | 96% | $10.16 |
| Hermes Agent, told to keep a program library | GPT-6 Luna, high | Random | ARC-AGI-1 | 12.5 | 96% | 18.5 | 96% | $14.07 |
| Hermes Agent, told to keep a program library, memory wiped | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 50.3 | 62% | 45.5 | 56% | $13.78 |
| Hermes Agent, told to keep a program library, memory wiped | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 47.0 | 64% | 43.3 | 60% | $10.53 |
| Hermes Agent, told to keep a program library, memory wiped | GPT-6 Luna, low | Random | ARC-AGI-1 | 48.1 | 65% | 43.0 | 56% | $12.76 |
| Hermes Agent, told to keep a program library, memory wiped | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 26.6 | 89% | 21.5 | 96% | $24.55 |
| Hermes Agent, told to keep a program library, memory wiped | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 26.9 | 89% | 33.5 | 84% | $27.81 |
| Hermes Agent, told to keep a program library, memory wiped | GPT-6 Luna, medium | Random | ARC-AGI-1 | 27.1 | 89% | 25.5 | 84% | $24.36 |
| Hermes Agent, told to keep a program library, memory wiped | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 24.1 | 93% | 22.5 | 88% | $24.88 |
| Hermes Agent, told to keep a program library, memory wiped | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 22.5 | 93% | 20.0 | 92% | $22.02 |
| Hermes Agent, told to keep a program library, memory wiped | GPT-6 Luna, high | Random | ARC-AGI-1 | 21.8 | 95% | 21.0 | 88% | $20.42 |
| OpenClaw, as shipped | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 42.3 | 72% | 40.8 | 68% | $17.22 |
| OpenClaw, as shipped | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 36.6 | 71% | 42.0 | 60% | $16.16 |
| OpenClaw, as shipped | GPT-6 Luna, low | Random | ARC-AGI-1 | 36.8 | 71% | 45.3 | 64% | $10.57 |
| OpenClaw, as shipped | GPT-6 Luna, low | Fresh | ARC-AGI-1 | – | – | 41.0 | 68% | $1.24 |
| OpenClaw, as shipped | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 26.8 | 89% | 28.0 | 88% | $7.61 |
| OpenClaw, as shipped | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 24.4 | 92% | 26.3 | 88% | $6.05 |
| OpenClaw, as shipped | GPT-6 Luna, medium | Random | ARC-AGI-1 | 26.7 | 88% | 28.3 | 92% | $6.15 |
| OpenClaw, as shipped | GPT-6 Luna, medium | Fresh | ARC-AGI-1 | – | – | 22.0 | 92% | $1.17 |
| OpenClaw, as shipped | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 23.2 | 94% | 22.0 | 100% | $6.44 |
| OpenClaw, as shipped | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 23.7 | 91% | 20.0 | 92% | $6.67 |
| OpenClaw, as shipped | GPT-6 Luna, high | Random | ARC-AGI-1 | 22.2 | 96% | 22.5 | 92% | $5.12 |
| OpenClaw, as shipped | GPT-6 Luna, high | Fresh | ARC-AGI-1 | – | – | 20.0 | 100% | $0.93 |
| OpenClaw, memory wiped | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 39.3 | 70% | 44.8 | 52% | $11.73 |
| OpenClaw, memory wiped | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 34.9 | 70% | 50.8 | 36% | $11.02 |
| OpenClaw, memory wiped | GPT-6 Luna, low | Random | ARC-AGI-1 | 35.3 | 70% | 45.3 | 60% | $9.77 |
| OpenClaw, memory wiped | GPT-6 Luna, low | Fresh | ARC-AGI-1 | – | – | 39.3 | 72% | $1.15 |
| OpenClaw, memory wiped | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 25.0 | 91% | 27.8 | 92% | $5.86 |
| OpenClaw, memory wiped | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 22.3 | 93% | 23.0 | 88% | $4.42 |
| OpenClaw, memory wiped | GPT-6 Luna, medium | Random | ARC-AGI-1 | 24.4 | 89% | 27.0 | 92% | $5.83 |
| OpenClaw, memory wiped | GPT-6 Luna, medium | Fresh | ARC-AGI-1 | – | – | 22.8 | 92% | $0.78 |
| OpenClaw, memory wiped | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 23.2 | 93% | 17.5 | 100% | $5.86 |
| OpenClaw, memory wiped | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 21.8 | 93% | 19.0 | 96% | $5.11 |
| OpenClaw, memory wiped | GPT-6 Luna, high | Random | ARC-AGI-1 | 23.4 | 92% | 18.3 | 96% | $6.02 |
| OpenClaw, memory wiped | GPT-6 Luna, high | Fresh | ARC-AGI-1 | – | – | 21.8 | 96% | $0.78 |
| OpenClaw, asked to store | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 38.9 | 68% | 46.0 | 60% | $18.51 |
| OpenClaw, asked to store | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 38.1 | 74% | 43.3 | 56% | $15.59 |
| OpenClaw, asked to store | GPT-6 Luna, low | Random | ARC-AGI-1 | 39.4 | 72% | 46.5 | 60% | $16.60 |
| OpenClaw, asked to store | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 28.0 | 88% | 27.0 | 84% | $6.99 |
| OpenClaw, asked to store | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 26.0 | 88% | 28.8 | 92% | $6.80 |
| OpenClaw, asked to store | GPT-6 Luna, medium | Random | ARC-AGI-1 | 24.5 | 91% | 27.3 | 92% | $8.43 |
| OpenClaw, asked to store | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 23.5 | 94% | 26.0 | 84% | $8.17 |
| OpenClaw, asked to store | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 21.2 | 95% | 19.3 | 92% | $7.26 |
| OpenClaw, asked to store | GPT-6 Luna, high | Random | ARC-AGI-1 | 22.7 | 95% | 21.0 | 96% | $6.95 |
| OpenClaw, told to keep a program library | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 29.7 | 81% | 34.3 | 76% | $63.07 |
| OpenClaw, told to keep a program library | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 26.3 | 84% | 36.3 | 84% | $59.42 |
| OpenClaw, told to keep a program library | GPT-6 Luna, low | Random | ARC-AGI-1 | 28.5 | 83% | 36.3 | 72% | $56.08 |
| OpenClaw, told to keep a program library | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 14.2 | 94% | 26.5 | 84% | $13.53 |
| OpenClaw, told to keep a program library | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 15.5 | 93% | 17.8 | 96% | $18.93 |
| OpenClaw, told to keep a program library | GPT-6 Luna, medium | Random | ARC-AGI-1 | 12.3 | 98% | 23.5 | 88% | $12.35 |
| OpenClaw, told to keep a program library | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 11.4 | 96% | 19.3 | 92% | $10.08 |
| OpenClaw, told to keep a program library | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 9.0 | 97% | 23.5 | 88% | $10.51 |
| OpenClaw, told to keep a program library | GPT-6 Luna, high | Random | ARC-AGI-1 | 10.5 | 98% | 19.3 | 96% | $9.75 |
| OpenClaw, told to keep a program library, memory wiped | GPT-6 Luna, low | Workstreams | ARC-AGI-1 | 36.4 | 81% | 31.5 | 80% | $39.85 |
| OpenClaw, told to keep a program library, memory wiped | GPT-6 Luna, low | Sequential | ARC-AGI-1 | 34.6 | 84% | 34.0 | 72% | $56.08 |
| OpenClaw, told to keep a program library, memory wiped | GPT-6 Luna, low | Random | ARC-AGI-1 | 34.6 | 82% | 34.8 | 76% | $59.09 |
| OpenClaw, told to keep a program library, memory wiped | GPT-6 Luna, medium | Workstreams | ARC-AGI-1 | 23.1 | 95% | 22.5 | 96% | $20.07 |
| OpenClaw, told to keep a program library, memory wiped | GPT-6 Luna, medium | Sequential | ARC-AGI-1 | 22.8 | 94% | 21.3 | 96% | $21.17 |
| OpenClaw, told to keep a program library, memory wiped | GPT-6 Luna, medium | Random | ARC-AGI-1 | 23.0 | 95% | 21.5 | 96% | $19.52 |
| OpenClaw, told to keep a program library, memory wiped | GPT-6 Luna, high | Workstreams | ARC-AGI-1 | 18.6 | 98% | 23.8 | 88% | $15.02 |
| OpenClaw, told to keep a program library, memory wiped | GPT-6 Luna, high | Sequential | ARC-AGI-1 | 20.0 | 98% | 19.5 | 92% | $15.58 |
| OpenClaw, told to keep a program library, memory wiped | GPT-6 Luna, high | Random | ARC-AGI-1 | 19.0 | 97% | 17.5 | 100% | $14.96 |
| Stateless floor | GPT-6 Luna, low | Random | ARC-AGI-2 | 63.7 | 12% | 66.8 | 4% | $2.23 |
| Stateless floor | GPT-6 Luna, low | Fresh | ARC-AGI-2 | – | – | 64.5 | 8% | $0.27 |
| Stateless floor | GPT-6 Luna, medium | Workstreams | ARC-AGI-2 | 54.0 | 35% | 56.5 | 28% | $4.35 |
| Stateless floor | GPT-6 Luna, medium | Random | ARC-AGI-2 | 57.0 | 31% | 57.5 | 24% | $4.58 |
| Stateless floor | GPT-6 Luna, medium | Fresh | ARC-AGI-2 | – | – | 57.8 | 20% | $0.58 |
| Stateless floor | GPT-6 Luna, high | Workstreams | ARC-AGI-2 | 46.2 | 50% | 47.8 | 52% | $6.86 |
| Stateless floor | GPT-6 Luna, high | Random | ARC-AGI-2 | 48.1 | 48% | 47.3 | 44% | $7.19 |
| Stateless floor | GPT-6 Luna, high | Fresh | ARC-AGI-2 | – | – | 42.5 | 64% | $0.77 |
| Stateless floor | GPT-6 Luna, xhigh | Workstreams | ARC-AGI-2 | 43.5 | 59% | 45.0 | 64% | $9.95 |
| Stateless floor | GPT-6 Luna, xhigh | Random | ARC-AGI-2 | 43.0 | 58% | 41.0 | 64% | $9.70 |
| Stateless floor | GPT-6 Luna, xhigh | Fresh | ARC-AGI-2 | – | – | 40.8 | 64% | $1.27 |
| Stateless floor | GPT-6 Luna, max | Workstreams | ARC-AGI-2 | 34.0 | 76% | 34.3 | 84% | $16.61 |
| Stateless floor | GPT-6 Luna, max | Random | ARC-AGI-2 | 32.2 | 78% | 34.5 | 72% | $16.33 |
| Stateless floor | GPT-6 Luna, max | Fresh | ARC-AGI-2 | – | – | 27.3 | 88% | $1.66 |
| Demo-cache floor | GPT-6 Luna, low | Random | ARC-AGI-2 | 53.6 | 24% | 66.5 | 4% | $2.41 |
| Demo-cache floor | GPT-6 Luna, low | Fresh | ARC-AGI-2 | – | – | 66.8 | 4% | $0.29 |
| Demo-cache floor | GPT-6 Luna, medium | Workstreams | ARC-AGI-2 | 41.4 | 53% | 56.5 | 24% | $4.15 |
| Demo-cache floor | GPT-6 Luna, medium | Random | ARC-AGI-2 | 41.8 | 47% | 58.5 | 20% | $4.19 |
| Demo-cache floor | GPT-6 Luna, medium | Fresh | ARC-AGI-2 | – | – | 58.0 | 24% | $0.57 |
| Demo-cache floor | GPT-6 Luna, high | Workstreams | ARC-AGI-2 | 34.0 | 64% | 47.5 | 48% | $6.22 |
| Demo-cache floor | GPT-6 Luna, high | Random | ARC-AGI-2 | 32.6 | 68% | 51.3 | 44% | $6.22 |
| Demo-cache floor | GPT-6 Luna, high | Fresh | ARC-AGI-2 | – | – | 50.0 | 40% | $0.96 |
| Demo-cache floor | GPT-6 Luna, xhigh | Workstreams | ARC-AGI-2 | 27.9 | 76% | 41.3 | 64% | $7.68 |
| Demo-cache floor | GPT-6 Luna, xhigh | Random | ARC-AGI-2 | 26.8 | 76% | 41.5 | 64% | $7.89 |
| Demo-cache floor | GPT-6 Luna, xhigh | Fresh | ARC-AGI-2 | – | – | 42.8 | 60% | $1.18 |
| Demo-cache floor | GPT-6 Luna, max | Workstreams | ARC-AGI-2 | 17.0 | 90% | 32.3 | 72% | $12.20 |
| Demo-cache floor | GPT-6 Luna, max | Random | ARC-AGI-2 | 16.5 | 93% | 33.0 | 84% | $12.59 |
| Demo-cache floor | GPT-6 Luna, max | Fresh | ARC-AGI-2 | – | – | 35.3 | 76% | $2.37 |
| Program library | GPT-6 Luna, medium | Workstreams | ARC-AGI-2 | 29.4 | 64% | 39.8 | 56% | $3.18 |
| Program library | GPT-6 Luna, medium | Fresh | ARC-AGI-2 | – | – | 45.0 | 52% | $0.50 |
| Program library | GPT-6 Luna, high | Workstreams | ARC-AGI-2 | 21.3 | 75% | 37.8 | 68% | $4.48 |
| Program library | GPT-6 Luna, high | Fresh | ARC-AGI-2 | – | – | 38.0 | 68% | $0.82 |
| Program library | GPT-6 Luna, xhigh | Workstreams | ARC-AGI-2 | 15.6 | 85% | 32.0 | 80% | $4.67 |
| Program library | GPT-6 Luna, xhigh | Fresh | ARC-AGI-2 | – | – | 32.3 | 76% | $1.02 |
| Program library | GPT-6 Luna, max | Workstreams | ARC-AGI-2 | 8.0 | 96% | 26.5 | 80% | $6.16 |
| Program library | GPT-6 Luna, max | Fresh | ARC-AGI-2 | – | – | 25.3 | 84% | $1.48 |
| Prime Agent, as shipped | GPT-6 Luna, medium | Workstreams | ARC-AGI-2 | 45.1 | 57% | 46.3 | 56% | $5.86 |
| Prime Agent, as shipped | GPT-6 Luna, medium | Sequential | ARC-AGI-2 | 45.2 | 57% | 47.3 | 44% | $5.79 |
| Prime Agent, as shipped | GPT-6 Luna, medium | Fresh | ARC-AGI-2 | – | – | 47.0 | 52% | $0.87 |
| Prime Agent, as shipped | GPT-6 Luna, high | Workstreams | ARC-AGI-2 | 38.0 | 70% | 36.3 | 76% | $7.06 |
| Prime Agent, as shipped | GPT-6 Luna, high | Sequential | ARC-AGI-2 | 37.4 | 73% | 39.8 | 60% | $6.98 |
| Prime Agent, as shipped | GPT-6 Luna, high | Fresh | ARC-AGI-2 | – | – | 42.8 | 56% | $1.07 |
| Prime Agent, as shipped | GPT-6 Luna, xhigh | Workstreams | ARC-AGI-2 | 34.6 | 75% | 35.5 | 72% | $7.76 |
| Prime Agent, as shipped | GPT-6 Luna, xhigh | Sequential | ARC-AGI-2 | 35.8 | 72% | 36.3 | 76% | $7.45 |
| Prime Agent, as shipped | GPT-6 Luna, xhigh | Fresh | ARC-AGI-2 | – | – | 27.3 | 88% | $0.74 |
| Prime Agent, as shipped | GPT-6 Luna, max | Workstreams | ARC-AGI-2 | 27.7 | 84% | 25.3 | 88% | $9.99 |
| Prime Agent, as shipped | GPT-6 Luna, max | Sequential | ARC-AGI-2 | 28.7 | 83% | 32.3 | 72% | $9.07 |
| Prime Agent, as shipped | GPT-6 Luna, max | Fresh | ARC-AGI-2 | – | – | 33.3 | 76% | $1.58 |
| Prime Agent, memory wiped | GPT-6 Luna, medium | Workstreams | ARC-AGI-2 | 45.0 | 56% | 43.3 | 60% | $5.72 |
| Prime Agent, memory wiped | GPT-6 Luna, medium | Sequential | ARC-AGI-2 | 44.8 | 59% | 42.5 | 60% | $5.59 |
| Prime Agent, memory wiped | GPT-6 Luna, high | Workstreams | ARC-AGI-2 | 37.4 | 71% | 42.8 | 56% | $7.07 |
| Prime Agent, memory wiped | GPT-6 Luna, high | Sequential | ARC-AGI-2 | 39.8 | 68% | 38.8 | 72% | $6.63 |
| Prime Agent, memory wiped | GPT-6 Luna, xhigh | Workstreams | ARC-AGI-2 | 35.4 | 73% | 40.8 | 60% | $8.12 |
| Prime Agent, memory wiped | GPT-6 Luna, xhigh | Sequential | ARC-AGI-2 | 34.7 | 74% | 36.3 | 68% | $7.52 |
| Prime Agent, memory wiped | GPT-6 Luna, max | Workstreams | ARC-AGI-2 | 30.8 | 80% | 28.3 | 88% | $10.45 |
| Prime Agent, memory wiped | GPT-6 Luna, max | Sequential | ARC-AGI-2 | 28.6 | 83% | 26.0 | 88% | $8.64 |
| Prime Agent, as shipped | Claude Opus 5.5, low | Workstreams | ARC-AGI-3 | 35.1 | 79% | 12.5 | 100% | $129 |
| Prime Agent, as shipped | Claude Opus 5.5, low | Fresh | ARC-AGI-3 | – | – | 12.0 | 100% | $1.57 |
| Prime Agent, memory wiped | Claude Opus 5.5, low | Workstreams | ARC-AGI-3 | 42.9 | 75% | 13.4 | 100% | $110 |