Test-Time Compute
Spending more computation at inference time—longer thinking, multiple samples or search—to improve answer quality.
In plain language
Classic scaling laws focus on training compute; test-time compute asks whether more thinking time lets a smaller model reach better results. Methods include longer chains of thought, parallel sampling with voting, tree search and verifier scoring. It helps math, code and planning, but returns diminish with task difficulty and latency/cost rise sharply. Product design must decide which requests deserve extra compute and which need fast responses.
Example
A code-fix task can sample eight candidate patches and pick the one that passes tests; simple QA should not.
Recommended tools
Related terms
Chain of Thought
Prompting a model to write intermediate reasoning steps before concluding.
Inference
The computation a trained model performs to produce output from an input.
Reasoning Model
A model optimized for complex problem solving by generating a longer internal reasoning process before answering.
Agent Memory
A mechanism for an agent to store and retrieve user preferences, past tasks and key facts across sessions.