A guided reading of Sebastian Raschka's "Controlling Reasoning Effort in LLMs" — zoomed out to where it lands in the story, through every section of the article, and zoomed in to the research around it.
Primary source: Controlling Reasoning Effort in LLMs, Sebastian Raschka, Ahead of AI, Jul 18 2026. Every other claim on this page carries its own source link where it is made.
−
Zoom out
For three years the industry's question was "can models reason?" That question is closed — reasoning models are the default. The question that replaced it, and the one this article answers, is an engineering question: how much reasoning do you actually want to pay for, and who decides? That shift — from capability to cost control — is the story.
The road to the effort knob
JAN 2022
Chain-of-thought prompting
"Let's think step by step" shows that intermediate tokens buy accuracy — reasoning as a prompt trick (Wei et al.).
SEP 2024
OpenAI o1 productizes it
Inference-time scaling becomes a product: hidden reasoning traces, tiered pricing. Reasoning as a SKU.
JAN 2025
DeepSeek-R1 opens the recipe
RLVR on final-answer correctness alone produces self-correction. Nine days later, Stanford's s1 paper shows a 1,000-example fine-tune plus the word "Wait" can force longer thinking. Reasoning as a commodity.
MAR 2025
The overthinking backlash
"Stop Overthinking" surveys a new field: models burn thousands of tokens on problems that need ten. Reasoning as a cost problem.
AUG 2025
GPT-5's Auto router faceplants
OpenAI ships automatic effort selection, users revolt, and the model picker comes back within days. Automation loses; knobs win.
2026
Everyone ships an effort knob
gpt-oss, GPT-5.6, DeepSeek V4, Nemotron 3, Kimi K2.5/K3, GLM-5, Qwen3, Inkling — one dial, many different training recipes behind it. This article is the first side-by-side of how they all work.
Why this matters to engineers now
Effort is the second axis of the pricing sheet. Model choice sets your floor; effort setting decides your bill. And because providers can tune effort server-side, it's also the mechanism behind every "did the model get dumber?" thread — the weights didn't change; the token budget did.
1×
The article
§1 · What a reasoning model is
"Reasoning" is an output format, not a mind
Same prompt, two models. Press play on the right one.
Conventional model
PROMPT: Is 391 prime?
No — 391 = 17 × 23.
Answer arrives immediately. Right or wrong, you see only the conclusion.
Reasoning model
PROMPT: Is 391 prime?
<think> …
No — 391 = 17 × 23.
Anatomy of the difference — same answer, extra tokens in front
TakeawayPay tokens, get accuracy. Everything else on this page is about pricing that pattern.
§2 · Training vs. inference scaling
Two dials were always there. RLVR taught the model to turn its own.
Two ways to buy reasoning: training scaling — reinforcement learning with verifiable rewards ("RLVR": checkers and unit tests replace human graders) and inference scaling (more tokens, more samples, more refinement).
R1's landmark finding: reward only the final answer, and backtracking, self-correction, "aha moments" emerge on their own. Nobody graded the reasoning; the reasoning appeared anyway.
DeepSeek-R1's recipe — hover any step. The "aha moments" were never rewarded directly; they emerged because they help the final answer be right.
TakeawayReasoning behavior is cheap to elicit — a reward on outcomes, not process. Which is exactly why every lab now has it, and why the differentiator moved to controlling it.
The photo finish nobody mentions
DeepSeek-R1 and Kimi K1.5 hit arXiv the same day — January 22, 2025. "RLVR" itself was coined two months earlier, in AI2's Tülu 3.
R1 won the narrative because R1-Zero proved pure RL, no supervised fine-tuning, was enough. The quiet consequence: most of today's LLMs are effectively reasoning models now.
Raschka's §2.3: RLVR training already buys implicit inference scaling (more tokens per answer); self-consistency and self-refinement stack on top, on demand.
It's packaging — the same tokens survive any wrapper
TakeawayThe tags exist so interfaces can fold the working away. It's a serialization format — JSON, not neurons.
§4 · On/off switches
The thinking toggle is trained, not prompted
First-generation reasoning models thought at full verbosity about everything — including "hello."
Qwen3's fix — Thinking Mode Fusion — is two training examples and a string trick.
/think: <think>{reasoning}</think>{answer} /no_think: <think></think>{answer} // the empty pair, thousands of times
The hard switch — flip it and watch what the model actually receives
The tokenizer never sends /no_think. It pre-writes the start of the model's own reply — and the model, seeing tokens it thinks it already said, just continues. The off-switch is prompt injection into the model's own mouth. Pure template code; the weights never change.
The /think flags are the trained "soft" switch; enable_thinking=False is the "hard" one. And this toggle is the simplified ancestor of every effort knob in the next section — off/on first, then a dial.
TakeawayToggles require mixed-mode training data. If a model has a working off-switch, someone built a dataset for it.
§5 · How effort settings work
Effort is a trained token budget with a price curve
In gpt-oss, the whole interface is three words in the system prompt: Reasoning effort: low/medium/high.
more effort → more tokens → higher accuracy → diminishing returns at the top
Build it with RL
effort-conditioned penalties
The effort label rides in the prompt during the reinforcement-learning stage. Low effort → harsh per-token penalty; high effort → gentle.
Build it with SFT
after the reinforcement-learning stage
Fine-tune on (prompt, effort label, target-length response) triples. The model learns label → length by example.
Real systems likely do both.
The cleanest case study is Inkling: a continuous effort value, 0.2 to 0.99, in the system message. The per-token cost in the RL reward scales with it:
R(e) = R_task − λ(e) · N_tokens // low effort → big λ, tokens hurt; high effort → small λ, think away
30M+ rollouts later: one smooth dial. No discrete modes at all.
Takeaway — the strategic oneModel size and reasoning effort are independent scaling axes. On cost-performance charts (Raschka cites Artificial Analysis), a smaller model at high effort overlaps a bigger model at low effort. "Which model?" is now inseparable from "at what effort?"
Try it · two ways to buy thinking
The knob vs. the hack — play with both
1 · The trained knob — drag the effort dial (Inkling-style, 0.2 → 0.99)
0.20 · terseeffort = 0.450.99 · exhaustive
1,900
thinking tokens (illustrative)
73%
benchmark accuracy (illustrative curve)
1.9×
cost vs. effort 0.2
+9.1%
accuracy gained per next 0.1 effort
Shapes, not measurements — the point is the diminishing-returns curve: past ~0.7, tokens keep climbing and accuracy barely moves. This is Raschka's Figure 20 pattern, drawn as a toy.
2 · The decode-time hack — press WAIT (the s1 trick, no training required)
PROMPT: How many primes are there between 100 and 130?
<think> 101, 103, 107, 109, 113 … that's 5 so far, then 127. I count 6. </think>
ANSWER: 6 primes.
Stanford's s1 paper: when the model tries to stop, append the word "Wait" and it re-opens its own reasoning — often catching real mistakes. AIME24 scaled from 50% → 57% with nothing but this. The trained effort knobs above are this hack, industrialized.
§6 · Six flagships, six recipes
Same dial, six different machines behind it
The specialist factory — how one checkpoint learns every mode
Why this is the punchline of §6: the "effort setting" you pick at inference is a memory of a specialist that no longer exists. DeepSeek V4 and Kimi K3 both train separate models per effort level, then compress the whole family into one checkpoint — and your system prompt summons the right ghost.
Kimi K3's budget enforcement — a reward cliff, not a nudge
From the K3 technical report: exceeding the budget doesn't shave the reward, it inverts it. Train a max-effort specialist on a generous budget, then tighten it to mint high- and low-effort specialists — three efforts × three domains (general, agents, coding) = the nine teachers distilled into one K3.
Kimi's toggle method — train tight, then loosen, repeat
TakeawayAt inference, Toggle is invisible — it lives entirely in training, and the shipped model has no budgeted / unconstrained switch. Thinking is simply the default; the separate instant mode is the chat template pre-filling empty think tags — the same off-switch hack as Qwen3.
The shared skeleton — all six recipes reduce to this
Raschka's comparison table (his Figure 32) varies the details — discrete vs. continuous effort, SFT vs. RL emphasis, template changes — but every model walks these three steps. His verdict: no clear winner.
§7 · Conclusion
The holy grail is automatic effort — and it already failed once
The obvious endgame is the model choosing its own effort per query. GPT-5 shipped exactly that as "Auto" — and it got removed from the UI. Raschka's near-term prediction: effort stays an explicit input, but routers and agent wrappers will increasingly infer the right mode from task state, with the user keeping override.
The unresolved tensionUsers demand the knob and resent turning it. Providers want the router because effort is their serving cost. Whoever controls the default controls both the bill and the vibes.
Three dials now exist. Tap each one.
Model size
training scaling
Effort
sequential inference scaling
Parallel subagents
lateral inference scaling
Tap a dial. Raschka's call: effort stays an explicit input while routers increasingly pick it — with a user override for latency, cost, and max performance.
The crossover question
Bigger model on low, or smaller model on high?
The question everyone actually has, answered with the three best datasets we could find — drawn, not described.
Quality vs. effort — 26 real coding tasks
GPT-5.5 Codex on real merged pull requests (GraphQL-go-tools) · stet.sh
Tests saturate early; craft keeps climbing. "Does it pass?" plateaus at high — but "would a human have shipped this?" keeps rising all the way to xhigh. Which metric you pay for is the whole decision.
…and what each step costs
mean $ per task and wall-clock, same 26 tasks
The last step doubles the bill. med→high buys the biggest quality jump for 1.4× cost; high→xhigh costs 2.2× and even regressed test-pass by 4 points. The study's verdict: "high looks like the practical sweet spot."
The middle child problem — GPT-5.6's own family, per Artificial Analysis
Source: Artificial Analysis, "GPT-5.6 has landed". The same article: every 5.6 model pushes past 5.5's Pareto frontier, and Sol-max is even slightly more token-efficient than 5.5 (15k vs 16k tokens per task). Generation beats effort: 5.6-low territory overlaps what 5.5 needed high effort to reach.
14×
how much larger a model can be matched by a small model spending its compute at test time instead (compute-matched) — Snell et al., 2024
4×
efficiency gain of compute-optimal test-time scaling over naive best-of-N sampling, same paper
⚠
the catch: it only works on problems where the small model already has a non-trivial success rate — test-time compute amplifies competence, it doesn't create it
log scale. Once someone pays to create the behavior, copying it is nearly free — source
≠
"no single technique consistently outperforms others across all tasks" — the survey's bottom line, and this page's
So: bigger-low or smaller-high? Three rules1 · Upgrade generation before raising effort. A new family's low tier overlaps the old family's high tier.
2 · Audit the middle tier. GPT-5.6's Terra is entirely covered by Luna and Sol operating points.
3 · Smaller-at-high only works inside its competence. Snell's 14× holds where the small model already sometimes succeeds — past its ceiling, no amount of thinking saves it.
Field guide
How to reason about reasoning — which effort, when, and what to expect
The practical section. Raschka's §5.1 establishes the mechanics — effort ↑ ⇒ tokens ↑ ⇒ accuracy ↑, with diminishing returns at the top — and OpenAI's reasoning best-practices guide supplies the task-level advice. Combined into one matrix:
Task
Effort
What to expect
Extraction, formatting, classificationwell-defined, an answer either right or wrong at a glance
OFF / low
Same accuracy as high effort, at a fraction of the cost. This is where overthinking lives — reasoning models burn thousands of tokens on ten-token questions (Stop Overthinking). OpenAI's own guidance: standard models "when speed and cost are more important than perfect accuracy."
Interactive chat, customer-facing UXa human is waiting on the tokens
low → med
Effort is a latency knob here, not a quality knob. Every point of effort adds seconds of thinking before the first visible word. Cap it, and enforce a hard budget — models are trained to survive the cutoff (Nemotron's truncation training, Qwen3's emergent stop).
The default that's usually right. The gpt-oss curves (Raschka Fig. 18) show medium capturing most of high's accuracy at far fewer tokens. Escalate only the failures — re-running the 10% that broke at high effort beats running 100% there.
Hard debugging, ambiguous specs, legal/financial document analysisambiguity, cross-references, needle-in-haystack
high
This is what the effort knob is for. OpenAI's list of reasoning-model sweet spots: ambiguity, relationships across large documents, code review. Expect 2–5× the tokens and visible gains — this is the steep part of the curve, before saturation.
Verifiable problems — math, checkable codeyou can test the answer mechanically
med + voting
Don't think harder, sample more. Majority voting across parallel runs stacks on any effort level and parallelizes (wall-clock flat) — the DeepSeekMath-V2 recipe, and the logic of Ultra's four subagents. Verify with your checker, keep the winner.
Split the effort by role. OpenAI's planner-executor pattern: a reasoning model plans the strategy, cheap models execute the steps. Per-turn toggles (GLM-5) and per-step budgets exist exactly for this — reasoning before every tool call is where agent bills explode.
Offline batch, no deadlineovernight pipelines, evals, research
max — or a smaller model at max
Latency is free, so buy accuracy — but check the overlap zone first. A smaller model at max effort can match a bigger model at low effort on the cost-accuracy curves; for batch work the cheaper point on the same curve wins.
Two prompting notes from OpenAI's guide that surprise people: don't ask reasoning models to "think step by step" — they already do, and the instruction can hurt; and skip few-shot examples first — brief, clear, zero-shot instructions with explicit success criteria work best.
+
Zoom in
Upstream · Jan 2025
s1: budget forcing — effort control without training for it
Stanford's minimalist counterpoint to everything in §6: fine-tune Qwen2.5-32B on just 1,000 curated examples, then control thinking length at decode time — append "Wait" when the model tries to stop and it re-examines its answer, scaling AIME24 from 50% to 57% and beating o1-preview by up to 27% on competition math. The trained effort knobs in Raschka's article are the industrial descendants of this hack.
"Stop Overthinking" — the survey that named the cost problem
The TMLR survey that catalogued the overthinking phenomenon — reasoning models spending thousands of tokens on trivial questions — and taxonomized the fixes (model-based, output-based, prompt-based efficient reasoning). It's the academic backdrop that explains why every 2026 flagship ships length penalties: the waste was measured first.
The GPT-5 router backlash — automation's first public defeat
OpenAI launched GPT-5 with an automatic router deciding how hard to think per query; users revolted over inconsistent quality and the loss of control, and OpenAI restored the model picker with explicit Auto/Fast/Thinking settings within a week. Fortune's framing: the router "ignited a user backlash — but it might be the future of AI." Raschka's conclusion cites this exact failure.
Artificial Analysis — where the effort curves live
The independent benchmark shop whose cost-vs-intelligence charts Raschka reproduces as his Figure 24. Their per-model pages now publish reasoning and non-reasoning variants side by side (e.g. GPT-5.6 Sol at max vs. non-reasoning) — the practical tool for the "smaller model at high effort vs. bigger model at low effort" decision.
The State of LLM Reasoning Model Training — two warnings this page needs
The loss function rewards rambling: in PPO-style training, a longer response dilutes the per-token penalty — so verbosity is partly a math exploit, not a personality. And benchmarks wobble: on small sets like AIME24, changing a random seed shifts scores several points. Wear that caveat over every effort curve on this page.
Understanding Reasoning LLMs — the four ways to build one
Raschka's earlier taxonomy: inference-time scaling, pure reinforcement learning, fine-tuning plus reinforcement learning, and distillation. Its warning list is the effort matrix's ancestor — reasoning models are "more expensive, more verbose, and sometimes more prone to errors due to overthinking," and wrong for summarization, translation, and knowledge QA.
The State of LLM Reasoning Model Inference — where the wild numbers live
The companion survey behind this page's stat tiles: the 1B-beats-405B result, a 7B with inference scaling surpassing R1, and the observation that o1 — heavy inference scaling and all — ran cheaper than the larger GPT-4.5. Its verdict is the honest one: no technique wins everywhere.
Inkling's announcement — the continuous dial, shown not told
Thinking Machines Lab's launch post for Inkling is the cleanest primary source on continuous effort conditioning — the effort-vs-tokens-vs-benchmarks charts Raschka reproduces come from here, and the page itself is worth opening on screen: the interactive graphics tell the 0.2 → 0.99 story better than any summary. Pair it with the technical report's detail that most of post-training was asynchronous RL over 30M+ rollouts with the token-cost coefficient λ(e) doing the steering.
The "did the model get dumber?" discourse — effort knobs explain the vibes
The week this report was built, the timeline was full of "downgraded to opus 5" and "opus 5 is giving sonnet 3.7," plus a 312-reply thread on dodging Claude Code usage limits. Server-side effort tuning, budget truncation, and router-selected modes are the exact mechanisms by which an unchanged model can feel different day to day — the article is the technical decoder ring for the discourse.
My take: effort is also a capacity-planning problem
The article looks at the knob through a training-first lens — how it gets built into the weights.
Run these models yourself and a second view appears: every effort level is a different token-length distribution, so a different memory footprint, batch shape, and cost per request.
The knob on the model is a knob on the cluster — the same mechanism enterprises are buying cost dashboards to control.
Grounding: Raschka §5.4 for the cost-curve framing · fleet arithmetic shown on screen so you can check it