Technology

Independent researcher reports 44 percent ARC-AGI-1 benchmark score at low compute cost

An updated small transformer model achieved higher benchmark accuracy for under one dollar of training and inference compute.

Updated Sep 1, 2026 · 10:20 UTC1 reports

The short version

An independent researcher updated an open-source small transformer model to achieve a 44% score on the ARC-AGI-1 benchmark.
The model was trained from scratch on a single Nvidia RTX 5090 GPU in 1.5 hours, costing an estimated total of 67 cents in lifetime compute.
Key changes include adopting modern transformer components like SwiGlu and RMSnorm, expanding to 8 layers, using 3D RoPE embeddings, and incorporating non-overlapping tasks from ARC-2.
The researcher suggested future modifications could push benchmark scores to 65% while keeping costs low.

The updated open-source model achieved a 44% score on the ARC-AGI-1 benchmark at a total reported compute cost of 67 cents.[Hacker News]
Training took 1.5 hours from scratch on an Nvidia 5090 GPU.[Hacker News]
Architectural updates to the model include SwiGlu activation functions, RMSnorm, expansion from 4 to 8 layers, 3D RoPE embeddings, and per-task additive embeddings.[Hacker News]
The model achieved approximately 7% on the ARC-2 benchmark.[Hacker News]
The loss function was modified to exclude input tokens, shifting the training approach to a supervised method.[Hacker News]
Ablation tests revealed that replacing 3D RoPE with 1D positional embeddings or removing per-task embeddings reduced the score to approximately 24%.[Hacker News]

The researcher acknowledged not understanding why removing input token training improved the benchmark score despite worsening the test loss.[Hacker News]