Reasoning models
A Reasoning modela model trained to produce a long private working before its answer, and to check and correct itself while doing so, rather than one prompted into showing its steps.Full glossary entryIntroduced in Reasoning models is designed to generate an internal scratchpad of intermediate thoughts before returning a final answer. If you ask it to solve a complex accounting problem, rather than immediately predicting the final total, the model spends hundreds or thousands of tokens working through calculations, verifying intermediate values, catching its own mistakes, and only then producing the visible response.
The core concept behind this approach is known as a Chain of thoughtwritten-out intermediate steps between a question and an answer. It improves accuracy on multi-step problems, and it is not a faithful record of how the model reached the answer.Full glossary entryIntroduced in Reasoning models. Early prompting strategies showed that instructing standard models to "think step by step" noticeably improved arithmetic and logical consistency. Modern reasoning models take this further: they are explicitly trained with reinforcement learning on verifiable domains—such as mathematics, logic puzzles, and software engineering—to develop systematic problem-solving habits rather than relying purely on prompt phrasing.
The primary advantage lies in self-correction. When an unspecialized model is prompted to think step by step, it produces sequential text, but it rarely notices when step four contradicts step two. A dedicated reasoning model actively backtracks when it detects an inconsistency, writes out corrections to its scratchpad, and pivots to an alternative approach before delivering an answer.
Thinking tokens consume context and budget
Internal reasoning steps are not a hidden computational trick—they are real tokens generated sequentially by the model. These Thinking tokensthe tokens of a reasoning model's private working. They are billed at the output rate and occupy the context window whether or not the provider shows them to you.Full glossary entryIntroduced in Reasoning models are classified as output tokens and billed at the higher output rate. They also consume room within the model's context window. Because token decoding runs sequentially, a model that generates 4,000 thinking tokens before responding introduces several seconds of latency before delivering its first visible word.
As a result, a single interaction with a reasoning model can cost significantly more than the same query sent to a standard model, even when the final visible response is identical in length. Teams that budget infrastructure costs solely on expected user-facing reply lengths will quickly face unexpectedly high cloud bills.
Controlling reasoning effort
Most providers that offer reasoning models provide configuration knobs to dial this behavior up or down. This control typically appears as a Reasoning effortthe dial that sets how much working a reasoning model does, expressed either as a coarse level such as low or high, or as a target token budget.Full glossary entryIntroduced in Reasoning models parameter (with settings like low, medium, or high) or as an explicit thinking token budget.
An effort parameter sets a qualitative guideline, allowing the model to adapt its internal thought length based on question difficulty within that tier. A token budget provides a numerical target the model tries not to exceed. In both cases, these settings are targets rather than rigid execution caps: the model may finish simple queries far below the allocated budget.
Dialing down reasoning effort is often the most effective way to address slow response times or runaway costs—often yielding better results than dropping down to an entirely smaller model family. The relationship between reasoning depth and accuracy follows diminishing returns: on many practical workflows, a medium effort setting captures the majority of accuracy improvements, while the maximum setting burns significant tokens for marginal accuracy gains.
This paradigm is widely described as Test-time computespending more computation at the moment of answering rather than at training time, as a way of buying accuracy. A smaller model thinking hard can beat a larger one answering immediately.Full glossary entryIntroduced in Reasoning models: investing additional computational power during inference rather than pouring all resources into initial model training. Test-time compute acts as an effective counterweight to model parameter size. On well-defined analytical tasks, a compact model allowed to deliberate thoroughly can frequently outperform a much larger model answering immediately, while costing less to host.
Visibility into internal thoughts
How providers handle internal reasoning logs varies considerably across vendors, which directly impacts debugging and observability.
Some APIs expose the complete raw thought transcript directly in the response payload. Others summarize the scratchpad using a separate smaller model, returning a sanitized paraphrase. In some environments, reasoning logs are hidden entirely, exposing only the billed token count. Certain providers encrypt internal reasoning blocks, requiring the client harness to round-trip the opaque payload on subsequent conversational turns to preserve internal state.
These differences complicate debugging. When an API only provides a synthesized summary of internal thoughts, a flawed answer may be accompanied by a plausible-sounding summary that does not accurately reflect how the model erred.
Furthermore, research indicates that generated reasoning text is not always an exact record of internal network activations. Models can stumble onto correct solutions despite flawed scratchpad explanations, or rationalize answers reached via alternative internal heuristics. Treat reasoning transcripts as helpful diagnostic hints rather than definitive audit trails.
When reasoning models are worth the cost
Reasoning models shine on tasks where logic can be verified against objective ground truth:
- Multi-step mathematics and quantitative analysis: Financial reconciliations, compound interest projections, and coordinate transformations where each step depends on prior outputs.
- Complex dependency planning: Ordering multi-stage database migrations, organizing infrastructure rollout phases, or analyzing task prerequisites.
- Constrained code generation: Tasks requiring strict adherence to multiple constraints, such as refactoring an internal module without modifying public signatures or introducing external libraries.
- Verifiable logic problems: Puzzles, formal specifications, and algorithm implementations covered by automated test suites.
Conversely, enabling heavy reasoning is counterproductive in several common scenarios:
- Direct retrieval queries: When the necessary information is already present in the prompt, extensive deliberation does not uncover new facts. Extracting a purchase order date from an invoice is a straightforward lookup.
- Prose editing and restructuring: Formatting tables, summarizing bullet points, or adjusting email tone requires stylistic alignment rather than logical proofs.
- High-throughput classification: Simple sentiment tagging or routing calls where low latency and cost efficiency outweigh fractional accuracy gains.
- Interactive, latency-sensitive surfaces: Inline code completion or search-as-you-type interfaces where delays of several seconds ruin the user experience.
Start at the lowest setting that solves the task. Increase it only against evidence.
- Run the task at the lowest effort setting the model offers.
- Measure accuracy on your own examples. Do not rely on a published benchmark.
- Raise the effort one step. Measure again.
- Stop at the step where accuracy stops improving.
- Record the thinking token count per call and the time to first token at that setting.
- Turn reasoning off for extraction, formatting and classification tasks.
- Turn reasoning off on any path where a person is waiting for the response.
Reasoning inside agentic tool loops
Pairing reasoning models with tool calls introduces subtle considerations for both cost and workflow stability.
From a cost standpoint, an autonomous agent completing a twenty-step workflow may deliberate before every single tool invocation. Twenty iterations generating 3,000 thinking tokens each adds 60,000 billable output tokens to the run—often dwarfing the cost of the actual user interactions and tool responses.
For workflow integrity, many provider APIs mandate that thinking blocks from preceding turns be returned unmodified alongside incoming tool execution results. This context allows the model to resume its problem-solving trajectory smoothly. If a client Harnessthe program between you and the model API. It assembles the context, executes the tools the model asks for, enforces permissions, and keeps calling the model until the job is done.Full glossary entryIntroduced in The agentic loop strips, modifies, or reorders these internal blocks when formatting message histories, the API may reject the payload or the model may lose conversational continuity. When an agent succeeds on its initial tool call but becomes erratic on subsequent turns, verify how your harness handles historical reasoning blocks.
In well-designed agent architectures, reasoning and tool execution interleave naturally: the model plans its approach, executes an action, evaluates the tool result against its expectations, adjusts its plan, and proceeds. While this interleaved feedback loop offers remarkable autonomous problem-solving capabilities, it represents one of the most token-intensive ways to run a model. The mechanics of orchestrating these cycles are covered in the agentic loop.
Terms introduced
- Reasoning model: a model trained to produce a long private working before its answer, and to check and correct itself while doing so.
- Thinking tokens: the tokens of that working, billed at the output rate and counted against the context window whether or not you see them.
- Test-time compute: spending more computation when answering rather than when training, as a way of buying accuracy.
- Reasoning effort: the dial that sets how much working a model does, as a coarse level or as a token budget.
- Chain of thought: written-out intermediate steps between a question and an answer, useful for accuracy but not a faithful record of how the answer was reached.
How providers do it
Every vendor here has converged on a named effort level instead of a token budget, and none of them will show you the actual working.
| Question | Anthropic | OpenAI | |
|---|---|---|---|
| What the dial is called | output_config.effort | reasoning.effort | thinking_level |
| Levels | low, medium, high, xhigh, max | none, minimal, low, medium, high, xhigh, max, varying per model | minimal, low, medium, high |
| Older token budget | budget_tokens, minimum 1,024, now rejected on 4.7 and later | Not offered | thinkingBudget, still accepted, -1 for dynamic |
| Can it be switched off? | Not on the newest models | none on GPT-5.6, rejected on GPT-6 Astra | 0 budget on some models; 2.5 Pro cannot |
| Raw reasoning returned? | Never | Never | Never |
| What you can get | A summary from a different model, or nothing | A summary, auto/concise/detailed | A summary plus an encrypted signature |
| Billing | Output tokens, full working not the summary | Output tokens | Output tokens plus thought tokens |
| Usage field | output_tokens_details.thinking_tokens | output_tokens_details.reasoning_tokens | total_thought_tokens |
| Must blocks be replayed with tool results? | Yes, unmodified, or 400 | Yes, or use previous_response_id | Yes in stateless mode |
Every row is confirmed except Google's default thinking level, where two vendor pages disagree, and the error each raises when a block is dropped.
The last row is the one that breaks agents. A harness that rebuilds the message history from its own record, rather than replaying what the API returned, will fail on the second tool call of a reasoning run.
- Anthropic
- OpenAI
What this maps to: the thinking field on the Messages API, with the depth dial now living in output_config.effort rather than in a token budget.
| Question | Answer | Status |
|---|---|---|
| Does a reasoning mode exist? | Yes, called extended thinking. Current form: thinking: {"type": "adaptive", "display": "summarized"} | confirmed |
| Is it on by default? | On Opus 5, Sonnet 5, Fable 5.1, Mythos 5.1, Fable 5, Mythos 5 and Mythos Preview, "thinking is already on and needs no configuration" | confirmed |
| How is the budget expressed? | As an effort level: output_config: {effort: "low" | "medium" | "high" | "xhigh" | "max"}. Default is high | confirmed |
| The older token budget | thinking: {"type": "enabled", "budget_tokens": N}, minimum 1,024 tokens, and less than max_tokens. "The budget is a target rather than a strict cap" | confirmed |
| Is the token budget still accepted? | Deprecated on the 4.6 models. Claude 4.7 and later return a 400 telling you to use adaptive and output_config.effort | confirmed |
| Can thinking be turned off? | thinking: {"type": "disabled"} returns a 400 on Fable 5.1, Mythos 5.1, Fable 5, Mythos 5 and Mythos Preview | confirmed |
| Is the reasoning text returned? | Never raw. "No display setting returns the raw chain of thought." display: "summarized" returns a summary; display: "omitted" returns thinking blocks with an empty thinking field and is the default on Opus 5, Sonnet 5 and the Fable and Mythos models | confirmed |
| Who writes the summary? | A different model. "Summarization is processed by a different model from the one you target in your requests. The thinking model does not see the summarized output" | confirmed |
| Redaction | redacted_thinking blocks are returned "when portions of Claude's reasoning are safety-redacted", carrying an encrypted data field and no readable text | confirmed |
| How are thinking tokens billed? | As output tokens, "even when the thinking text isn't returned to you", and they count toward max_tokens. "You're charged for the full thinking tokens generated by the original request, not the summary tokens" | confirmed |
| Billed again on later turns? | On models that keep prior-turn thinking, the kept blocks are billed as input tokens on the next request | confirmed |
| Where is the count reported? | usage.output_tokens_details.thinking_tokens, on the final message_delta when streaming | confirmed |
| Interaction with tool calls | Thinking blocks "must be passed back to the API, complete and unmodified", including redacted ones and ones with an empty thinking field. Modifying them returns a 400 | confirmed |
| Forced tool use | Allowed with adaptive thinking except on Fable 5.1 and Mythos 5.1, which return a 400 on tool_choice any or tool. Manual mode allows only auto and none | confirmed |
| Interleaved thinking | Automatic on adaptive models with no beta header. Claude 4.5 and earlier Claude 4 need interleaved-thinking-2025-05-14. Haiku 4.5 does not support it | confirmed |
| Effect on prompt caching | Any thinking-config or effort change invalidates cache breakpoints | confirmed |
Their vocabulary
| Standard term | Their term |
|---|---|
| Reasoning model | Extended thinking, as a mode rather than a separate model line |
| Reasoning effort | output_config.effort, levels low through max |
| Thinking tokens | Thinking tokens, billed as output tokens |
| Chain of thought | Thinking blocks, returned summarized or omitted, never raw |
Where to look
usage.output_tokens_details.thinking_tokens is the figure to chart. It is the part of the bill nobody sees in the reply text.
Last verified: 2026-09-09 against the Anthropic platform docs: thinking, extended thinking, errors, and context windows (platform.claude.com/docs/en/build-with-claude/thinking, /build-with-claude/extended-thinking, /api/errors, /build-with-claude/context-windows).
What this maps to: reasoning.effort on the Responses API, or reasoning_effort on Chat Completions. The reasoning text itself is never returned.
| Question | Answer | Status |
|---|---|---|
| Does a reasoning mode exist? | Yes. GPT-6 Astra is the primary recommendation, with the GPT-5.6, GPT-5.5 and GPT-5.4 families alongside it | confirmed |
| How is the budget expressed? | As an effort level drawn from none, minimal, low, medium, high, xhigh and max. Which values a model accepts differs per model | confirmed |
| Per-model levels | gpt-5.6-sol, gpt-5.6-terra and gpt-5.6-luna: none, low, medium (default), high, xhigh, max. gpt-5.5: the same without max. gpt-6-astra: low, medium, high, xhigh, max, and none returns a 400 | confirmed |
| Parameter name | reasoning.effort on Responses, reasoning_effort on Chat Completions | confirmed |
| Changing effort mid-conversation | configuration_update input items change it without invalidating the prefix cache, provided request-level reasoning.effort stays constant | confirmed |
| Is the reasoning text returned? | No. "Reasoning tokens are not visible via the API." Only summaries are available | confirmed |
| Summary options | The summary parameter takes auto, concise or detailed, and availability differs per model. auto gives the most detailed summarizer that model has | confirmed |
| How are reasoning tokens billed? | They "still occupy space in the model's context window and are billed as output tokens" | confirmed |
| Where is the count reported? | usage.output_tokens_details.reasoning_tokens | confirmed |
| Headroom advice | The guide recommends "reserving at least 25,000 tokens for reasoning and outputs when you start experimenting with these models" | confirmed |
| What if the budget runs out mid-reasoning? | The response returns status: "incomplete" with incomplete_details.reason of max_output_tokens, and this can happen before any visible output token exists | confirmed |
| Carrying reasoning across turns | previous_response_id is "the shortest stateful integration". Stateless callers get an encrypted_content property on reasoning items to pass forward | confirmed |
The include value that requests encrypted reasoning | Not confirmed; the Responses create parameter reference would not load | unconfirmed |
| Multi-turn default | GPT-5.6 models default reasoning context to all_turns, rendering compatible reasoning items from earlier turns into the next sample | confirmed |
Their vocabulary
| Standard term | Their term |
|---|---|
| Reasoning effort | reasoning.effort or reasoning_effort |
| Thinking tokens | Reasoning tokens |
| Chain of thought | Reasoning items, returned as summaries or as encrypted content |
Where to look
usage.output_tokens_details.reasoning_tokens against output_tokens tells you what share of the bill went on working nobody read.
Last verified: 2026-09-09 against the OpenAI developer docs: the reasoning guide, latest-model guidance, and the per-model pages for gpt-6-astra and gpt-5.5 (developers.openai.com/api/docs/guides/reasoning, /api/docs/guides/latest-model, /api/docs/models/gpt-6-astra, /api/docs/models/gpt-5.5).
What this maps to: thinkingConfig on the Gemini API. Google moved from a numeric token budget to a named level, and the two cannot be sent together.
| Question | Answer | Status |
|---|---|---|
| Does a thinking mode exist? | Yes, across gemini-3.8-flash, gemini-3.7-flash, gemini-3.5-flash, gemini-3.5-flash-lite, gemini-3.1-pro-preview, gemini-2.5-pro, gemini-2.5-flash and gemini-2.5-flash-lite | confirmed |
| How is the budget expressed? | thinking_level (thinkingLevel), with values minimal, low, medium and high | confirmed |
| Per-model levels | gemini-3.8-flash: low, medium (default), high, and minimal returns an error. gemini-3.7-flash: low, medium (default), high. gemini-3.5-flash: minimal, low, medium (default), high. gemini-2.5-flash-lite: off by default | confirmed |
| The default level | The per-model table says medium for the 3.x Flash models. The Gemini 3 guide says "If thinking_level is not specified, Gemini 3 will default to high" | unconfirmed; the two vendor pages disagree |
| The older numeric budget | thinkingBudget, still accepted for backward compatibility. gemini-2.5-pro takes 128 to 32,768 and "cannot disable thinking". gemini-2.5-flash takes 0 to 24,576, where 0 disables it. gemini-2.5-flash-lite takes 512 to 24,576 | confirmed |
| Special budget values | -1 means dynamic thinking, where "the model will adjust the budget based on the complexity of the request". 0 disables thinking where the model allows it | confirmed |
| Can both be sent? | No. "You cannot use both thinking_level and the legacy thinking_budget parameter in the same request" | confirmed |
| Default behaviour | "Gemini models engage in dynamic thinking by default, automatically adjusting the amount of reasoning effort based on the complexity of the request" | confirmed |
| Is the reasoning text returned? | Summaries only. thinking_summaries takes auto (the default) or none. The legacy tree calls it includeThoughts, returning "summarized versions of the model's raw thoughts". A thought block may hold only a signature and no summary | confirmed |
| Thought signatures | "An encrypted representation of the model's internal reasoning state. Always present, even when the model performs minimal reasoning." They appear only on thought steps and built-in tool steps | confirmed |
| Must they be sent back? | In stateless mode, yes: "You MUST always resend all thought blocks exactly as they were received from the model". In stateful mode the server manages them | confirmed |
| The error raised when a signature is dropped | Not documented on a page read today | unconfirmed |
| How are thinking tokens billed? | "Response pricing is the sum of output tokens and thinking tokens" | confirmed |
| Where is the count reported? | interaction.usage.total_thought_tokens, or usageMetadata.thoughtsTokenCount in the legacy tree | confirmed |
| Interaction with function calling | "Gemini 3 series models use an internal 'thinking' process that improves function calling", and the SDKs handle thought signatures for you | confirmed |
Their vocabulary
| Standard term | Their term |
|---|---|
| Reasoning effort | thinking_level, or the legacy thinkingBudget in tokens |
| Thinking tokens | Thought tokens, total_thought_tokens |
| Chain of thought | Thought blocks, returned as summaries plus an encrypted signature |
Where to look
total_thought_tokens in the usage block. Compare it against total_output_tokens to see the real cost of a level change.
Last verified: 2026-09-09 against the Gemini API docs: thinking, the legacy thinking guide, the Gemini 3 guide, latest-model migration, and function calling (ai.google.dev/gemini-api/docs/thinking, /gemini-api/docs/generate-content/thinking, /gemini-api/docs/gemini-3, /gemini-api/docs/latest-model, /gemini-api/docs/function-calling).