ARC Prize Audits GPT-6 Astra: Scores 62.7% on ARC-AGI-3, Refuting 99.9% Claim
ARC Prize re-evaluates OpenAIs GPT-6 Astra under its neutral harness, recording 62.7% against OpenAIs published 99.9% and challenging AGI claims.
In an independent audit published October 25, 2026, the ARC Prize re-evaluated OpenAIs flagship GPT-6 Astra on ARC-AGI-3 using its standardized neutral harness, recording a 62.7% score. The findings officially refute OpenAIs published 99.9% claim—which was derived from an unstandardized, tools-enabled setup costing $19,000 per run—recalibrating industry timelines around artificial general intelligence.
Independent Audit Refutes OpenAIs 99.9% AGI Claim
On October 25, 2026, the ARC Prize foundation released the results of its rigorous independent evaluation of OpenAIs GPT-6 Astra on the ARC-AGI-3 intelligence benchmark. Using its standard neutral testing harness, ARC Prize recorded a verified score of 62.7%, directly challenging OpenAIs headline 99.9% result published during Astras September debut. While 62.7% remains the highest score achieved by any frontier model to date, ARC Prize formally deemed OpenAIs 99.9% figure invalid under standardized competition criteria.
The $19,000 Tools-Enabled Harness Discrepancy
The audit revealed that OpenAI achieved its near-perfect score through a custom, tools-augmented evaluation pipeline that consumed approximately $19,000 in compute per run, an execution environment unavailable to any other competing lab on the leaderboard. Under standardized compute budgets without specialized scaffolding, Astras performance scaled down to 59.3% and 54.8%. The revelation refutes claims made by industry figures—including NVIDIA CEO Jensen Huang, who had cited the 99.9% score to assert that AGI had arrived—reigniting intense debate over AI evaluation integrity.
Implications for Frontier AI Benchmarking and AGI Timelines
Following the audits publication, OpenAI quietly updated several performance disclosures across its documentation. The dispute underscores a growing industry-wide shift away from vendor-reported benchmark self-evaluations toward third-party adversarial verification. Researchers emphasized that while Astra represents genuine state-of-the-art capability in abstract reasoning, the gap between 62.7% and 99.9% proves that robust, out-of-distribution reasoning remains an unsolved frontier rather than a settled milestone.