Test-time compute
The computing effort a model uses while answering a question, as opposed to while it is being trained.
Test-time compute is the amount of calculation a model uses when it is actually answering a question, rather than when it was trained. It is also called inference-time compute. Traditionally an answer took a fixed and small amount of effort. Now some systems can be set to use much more.
The everyday comparison is an exam candidate given ten minutes for a question versus ten seconds. With more time to think, a model can work through steps, try several routes and check its own answer. That is how many reasoning models get better results on hard problems.
This matters for AI progress because it offers a second way to improve results. Before, most gains came from spending more on training. Now, spending more on each answer can also help, and some studies suggest the two can be traded off against each other. A 2024 study, for example, found that on some problems a smaller model given more thinking time could do better than a model 14 times its size.
It also has costs. Each answer is slower and more expensive, and there are limits to how far extra thinking helps. Researchers disagree about how much more can be gained this way, and whether it works outside tasks with checkable answers.