For most of AI’s history, the dominant lever for improving model capability was simple: train longer, train on more data, use more parameters. The scaling laws published by Kaplan et al. in 2020 formalized this intuition into a predictable curve — double the compute, get a consistent improvement in loss. OpenAI’s GPT series, Anthropic’s Claude models, and Google’s Gemini all rose to prominence by aggressively climbing this curve.
In 2025 and 2026, a different scaling axis emerged: not training compute, but inference compute. The question shifted from “how much did it cost to train this model?” to “how long do we let it think before it answers?” The results have been surprising enough to force a genuine re-evaluation of what AI capability means and how it is achieved.
The Reasoning Model Breakthrough
OpenAI’s o3 model demonstrated the potential most dramatically. On ARC-AGI-1, a benchmark specifically designed to resist memorization and test novel reasoning, o3 achieved 87.5% — a score that earlier scaling law projections suggested would require models orders of magnitude larger than anything currently feasible. It achieved this not by having more parameters, but by being allowed to generate extended chains of thought before producing a final answer.
DeepSeek R1, released as an open-weight model, reproduced much of this capability at approximately one-twentieth of o3’s reported inference cost. The cost differential matters because it reveals that test-time compute scaling is not primarily about brute force — it is about training models to allocate their reasoning budget efficiently.
The underlying mechanism is a form of iterative self-correction. Where a standard LLM generates an answer in a single forward pass, a reasoning model generates a sequence of intermediate steps, evaluates them (implicitly, through the model’s learned behavior rather than an external critic), revises its approach, and converges on an answer. The “chain of thought” framing popularized by Wei et al. in 2022 captured part of this intuition; the o3/R1 generation took it substantially further.
Why This Changes the Capability Ceiling
Pretraining scaling laws are running into practical limits. The compute budget required to meaningfully advance a frontier dense model is growing faster than the infrastructure to support it. The largest training runs in 2025 consumed tens of thousands of H100s for months; the next generation would require infrastructure that does not yet exist at the required scale.
Test-time compute scaling faces a different constraint: it trades latency for capability, and that tradeoff is adjustable at deployment time. A reasoning model can answer quickly for simple queries and extend its thinking for complex ones. The same model weights support a range of capability levels depending on how long the user (or application) is willing to wait. This is categorically different from a fixed capability level baked into model weights during training.
For tasks where correctness matters more than speed — code generation, mathematical proof verification, multi-step planning, scientific hypothesis generation — the latency cost is acceptable. A model that takes 30 seconds to correctly solve a problem that a faster model would have gotten wrong is unambiguously more valuable for that use case.
The Search Framing
Researchers have started framing test-time compute scaling as a search problem. The model is not just generating text; it is searching a space of possible reasoning paths for one that leads to a correct answer. Different approaches to this search have different efficiency properties.
Chain-of-thought reasoning is a depth-first search through reasoning steps. Process reward models, which score intermediate reasoning steps rather than just final answers, provide a signal to guide that search toward more promising branches. Monte Carlo Tree Search (MCTS) applied to language model reasoning is a more structured version of the same idea — explore multiple branches, evaluate them, prune the bad ones, follow the promising ones.
The performance improvements from reasoning models are therefore not primarily about the model “knowing more” — the base weights have the same knowledge as a standard model of the same size. They are about the model being better at finding the right knowledge and applying it correctly through structured search.
Implications for Model Evaluation
Test-time compute scaling creates a measurement problem. Benchmarks that were designed to measure model knowledge — perplexity on held-out text, accuracy on multiple-choice questions — do not distinguish between a model that knows the answer and a model that can reason its way to it. A benchmark score for a reasoning model conflates the quality of the base model with the quality of its search process and the compute budget allocated for inference.
ARC-AGI-2, released following o3’s near-saturation of ARC-AGI-1, is an attempt to create tasks where reasoning alone is insufficient — where genuine novel concept formation is required. Gemini Deep Think’s 45.1% on this benchmark is impressive; it is also a reminder that 54.9% of the benchmark remains unsolved, and that the gap between sophisticated test-time search and genuine general intelligence is still substantial.
What Changes for Developers
The practical implication for developers building on AI APIs is that the capability-cost tradeoff is now a parameter you can tune. Standard models are cheap and fast; reasoning models are more expensive and slower but dramatically more accurate on complex tasks. The right choice depends on your application.
For simple classification, extraction, and generation tasks, standard models remain the right choice. For tasks where error rates carry significant costs — code that will run in production, financial analysis, medical information synthesis — the additional cost of a reasoning model may be justified by the reduction in error rate. The economics require case-by-case evaluation.
The longer-term implication is structural: the dominance of training compute as the primary axis of AI capability is ending. A smaller, cheaper model that reasons well may outperform a larger, more expensive model that does not — and the reasoning capability can be improved through better search strategies, better reward models, and better training for self-correction without requiring more pretraining compute. This is a fundamental change in the competitive dynamics of AI development, and its effects are only beginning to be felt.