Quick Reference: Domain 5 — Model Selection and Optimisation

Claude is built on an autoregressive model, pretrained "to predict the next word, given the previous context of text in the document". At inference it emits one token at a time. Each token is conditioned on everything before it, including its own output. Three consequences are tested:

• Generation is serial, so response time scales with output length. Input length drives the prefill (the processing of the input) and therefore time to first token; output length drives the rest of the response time. Choose the wrong one and you answer a latency complaint with a capability lever.
• There is no lookahead and no revision. A token, once emitted, cannot be changed. That is why a stated output contract (a statement of the required output format) and a worked example placed before the task work better than an instruction to fix the format afterwards.
• Each token conditions the next, so an early mistake leads to more mistakes instead of correcting itself. The model is continuing what it has written, not sampling afresh at each position.

Thinking is the same mechanism run before the visible answer: reasoning tokens are generated first, billed as output, and counted against max_tokens.

A token is a unit of the model vocabulary, not of human text: tokens "can correspond to words, subwords, characters, or even bytes (in the case of Unicode)". Code, JSON, tables and non-English text tokenise differently from prose, so a ratio calibrated on paragraphs misprices a workload of log lines. Don't rely on a tokens-per-word ratio. The documentation gives two such ratios that disagree with each other. The Glossary gives roughly 3.5 characters per token, and the Pricing FAQ gives roughly 4 characters or 0.75 words. That disagreement is the reason to measure.

There are three measuring tools for three jobs, and an answer that uses one for another's job is always wrong:

The counting endpoint accepts the same structured inputs as a real message (system prompt, tools, images, PDFs) and returns a single input_tokens field. Its answer is explicitly "an estimate" and it does not apply caching logic.

Dated current-state note (August 2026): "Claude 4.7 and later models and Claude Mythos Preview use a newer tokenizer. The same input text produces approximately 30 percent more tokens than on earlier models." The endpoint "returns the count under the tokenizer of the model you pass", so a count taken against last year's model is the wrong number for this year's, both for the bill and for the space left in the context window. Recount whenever you change models.

The window is not an input allowance. Everything competes for the same space: "the system prompt, every message in messages (including tool results, images, and documents), and your tool definitions. The output Claude generates for the turn, including its extended thinking, counts too."

Two consequences are often stated as their opposite. Caching creates no extra space in the context window: "Cached prompt prefixes still occupy the context window: prompt caching changes what you pay for those tokens, not whether they count." And bigger is not automatically better: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot."

Overflow has two behaviours, and the way to tell them apart is whether the input alone is too big:

Dated current state (August 2026): the newer behaviour can be turned on for earlier models — the docs say to "use the model-context-window-exceeded-2025-08-26 beta header" to opt in. The stop reason is also "currently typed only in the SDKs' beta namespace", so typed access goes through client.beta.messages.

Both cases are arithmetic problems against a token budget, and both have fixes that change the tokens or the budget: send less, retrieve fewer and better passages, split the job, or move to a larger window. Neither is a prompt fault and neither is fixed by a more capable tier. Rolling first-in-first-out truncation of old messages is chat-interface behaviour, not API behaviour, and it is a wrong answer that often appears here.

The claim in this domain least likely to change, because it is a property of the service rather than of a release: "Even with temperature set to 0, the results will not be fully deterministic, and identical inputs might produce different outputs across API calls. This applies both to Anthropic's first-party inference service and to inference through third-party cloud providers."

Temperature zero gives consistency, not reproducibility. Never build a component whose correctness requires two identical calls to return identical bytes. Constrain the output format instead: a worked example, an explicit output contract, validation when your code receives the output. Store what has to stay stable rather than regenerating it and comparing.

Dated current-state note (checked 29 October 2026): the sampling parameters do not merely have a weak effect, they are rejected. The API parameter deprecations table lists temperature, top_p and top_k as "Removed in API version 2026-10-21". Under that version, a request that sends any of them returns a 400 error on every model. On earlier API versions, a non-default value returns a 400 on Claude 4.7 and later models and Claude Mythos Preview. On older models, such as Claude Sonnet 4.6, those earlier versions restrict the parameters only while thinking is on. The recommended replacement is "Omit and use prompting to guide model behavior."

The guide names these four together, and the knowledge that stays true is which problem each one solves. They are not interchangeable.

Extended thinking is the older interface: the earlier Claude Haiku 4.5 still uses it, and the newest models reject it.

Effort is the setting used most and the first one to try, unless you are a model or two behind the current line-up, when Anthropic's cost guidance says to upgrade first. The default depends on the model. As of 14 October 2026, "Most Claude models default to high effort", while "Claude Opus 5.5 and Claude Haiku 5.5 default to medium". "Setting effort to the model's default ("medium" on Claude Opus 5.5 and Claude Haiku 5.5, "high" on other models) produces exactly the same behavior as omitting the effort parameter entirely." Two properties make it the first choice. It "applies to every output token", so "it works whether or not thinking is enabled". And it covers tool calls as well as text, so "Lower effort also means fewer and terser tool calls" rather than merely shorter prose. It is "a behavioral signal, not a strict token budget".

Dated current-state note (September 2026): adaptive thinking is already on with no configuration needed on Claude Opus 5.5, Opus 5, Sonnet 5.5, Sonnet 5, Fable 5.1, Fable 5 and the Mythos models, and as of 14 October 2026 on Claude Haiku 5.5 too. As of 14 November 2026 the same is true of Claude Fable 5.2. On those models the thinking display field defaults to "omitted". Thinking blocks are returned with an empty thinking field until you set display: "summarized". No setting returns the raw chain of thought. Read the thinking blocks rather than asking for the reasoning in the answer. The reason is that this request can be refused with stop_details.category: "reasoning_extraction". As of 14 November 2026 that applies to Claude Fable 5.2, Fable 5.1, Opus 5.5, Opus 5, Sonnet 5.5 and Fable 5. This category is billed even when the refusal arrives before any output. As of 13 November 2026, fast mode delivers "up to 2.5x higher output tokens per second compared to standard speed" at premium pricing. It is available on Claude Fable 5.2, Claude Mythos 5.2 and Claude Opus 5.5, in research preview on Claude Opus 5, and deprecated on Claude Opus 4.8, where it is removed on 30 November 2026. The price is $16 in and $80 out per MTok on Claude Fable 5.2 and Mythos 5.2, $8 and $40 on Claude Opus 5.5, and $10 and $50 on Claude Opus 5 and Opus 4.8. Its gain is in OTPS (output tokens per second), "not time to first token (TTFT)", so it makes a visible difference on long streamed answers and almost none on short ones. It is unavailable with the Batch API or a Priority Tier commitment, and switching between fast and standard speed invalidates the prompt cache.

The selection rule, in three sentences. A hard problem answered shallowly needs more reasoning. A routine problem that costs too much needs less effort. An answer that is already correct but arrives too slowly for a person watching needs faster delivery, meaning streaming first and fast mode where it exists, not a different model.

The guide's vocabulary is zero-shot (an instruction, no demonstration), single-shot (one worked input-output pair) and multi-shot (several). The current documentation calls the same family "few-shot or multishot prompting" and makes the durable claim: "Examples are one of the most reliable ways to steer Claude's output format, tone, and structure."

The quality tests are named. Examples should be Relevant ("Mirror your actual use case closely"), Diverse ("Cover edge cases and vary enough that Claude doesn't pick up unintended patterns") and Structured (wrapped in <example> tags so Claude can tell them from instructions), with "Include 3-5 examples for best results."

Diversity is the one candidates underrate: a set of examples sharing an incidental property teaches that property. If every example happens to be a run with no exceptions, the model learns that notes never report an exception.

When to use examples: a failure of form or of boundary, meaning inconsistent structure, unclear classification edges or drifting tone. Not a failure of budget (a request that will not fit) or of fact (something the model cannot know). Examples cost input tokens on every call, which is exactly why a stable example block at the front of a prompt is a caching candidate.

Underneath every Claude application is one HTTP POST and one JSON document. A request to POST /v1/messages has three headers, plus anthropic-workspace-id with a multi-workspace key. The first is the credential as Authorization: Bearer <key>. As of September 2026 the older x-api-key header is the legacy fallback and is still accepted. The second is anthropic-version. Each SDK major version sends one API version, and current majors send 2026-10-21. The third is content-type. The response is ordinary JSON with an id, a role, an array of typed content blocks, a stop_reason and a usage object. Failures arrive in a fixed envelope: one JSON object with a fixed structure. It has a top-level type of error and a nested error object that always carries a type and a message. It also has a request id, which is also sent in the request-id response header.

Anthropic ships client SDKs for seven languages (Python, TypeScript, C#, Go, Java, PHP, Ruby) plus the ant CLI: "Each SDK provides idiomatic interfaces, type safety, and built-in support for streaming, retries, error handling, and OpenTelemetry tracing." In detail, that is credential and version headers, typed requests and responses, a typed exception hierarchy, automatic retry of the transient class, a default timeout, parsed SSE events with an accumulation helper, one interface over sync and async, _request_id on responses, auto-pagination, and dedicated client classes for the cloud platforms. None of it is interesting. All of it becomes your work as soon as you remove the dependency.

What the exam questions test is what it emphatically does not change. No SDK-only models, no SDK discount, no higher or waived rate limits, no uptime guarantee, no immunity from an error class. Models, prices, rate limits and contractual terms are properties of the API and of your agreement, not of a package. The exam-safe formulation: the SDK changes how convenient it is to call the API and nothing about the API itself.

Branch on the class, most specific first, never on message text, which is not a contract:

Two defaults are misread more often than anything else here. The retried set: "Certain errors are automatically retried 2 times by default, with a short exponential backoff. Connection errors (for example, because of a network connectivity problem), 408 Request Timeout, 409 Conflict, 429 Rate Limit, and >=500 Internal errors are all retried by default." The timeout: "By default requests time out after 10 minutes." They combine, because "requests that time out are retried twice by default", so the worst-case total time is roughly the timeout multiplied by the attempt count.

Not every 429 is transient. As of September 2026 a usage tier's monthly spend cap also returns one. It has no retry-after header and an error.details.error_code of enforced_spend_limit_reached. It "keeps failing until access resumes", the SDK's own retries included. Read that field before queueing a RateLimitError for retry.

Adding your own retry loop to the SDK retries does not add resilience. It multiplies attempts, re-sends deterministic failures that will never succeed, and sends more requests to an endpoint that is already over its rate limit. Set max_retries and the timeout deliberately at one layer, and let the other decide policy: degrade, queue, or fail with a visible error.

Construct one client and reuse it. The client owns an HTTP connection pool. A client built inside every request handler discards that pool on every call. Each call then pays for a fresh TLS handshake, and under concurrency the number of open sockets multiplies. This defect presents as latency that grows with traffic and connection errors that look like an upstream problem, so a question describing exactly those symptoms is usually describing this defect.

A question that often appears shows a developer proposing the right transport at the wrong layer.

MDN on SSE: "This is a one-way connection, so you can't send events from a client to a server." MDN on WebSockets: they make it "possible to open a two-way interactive communication session between the user's browser and a server". Anthropic separates the two layers in its own hosting guidance: an application container "exposes an HTTP or WebSocket endpoint" to its clients, while separately "The SDK needs outbound HTTPS to api.anthropic.com."

No documented Anthropic API surface accepts a websocket connection from a client, so "open a websocket to Anthropic so the user sees text appear" is wrong at the protocol level rather than merely suboptimal. The reverse is equally testable: streaming from the API needs no websocket in your front end, and a websocket in your front end gives you no streaming from the API. Independent decisions at different layers.

A stream can fail after it has already returned a 200. A 200 opens the stream, and "The API might occasionally send errors in the event stream. For example, during periods of high usage, you might receive an overloaded_error", delivered as an event: error frame mid-stream. Error handling wrapped around the call but not the iteration produces a silently truncated answer with no exception. Two more mechanics. Streaming is sometimes obligatory. The trigger is expected duration rather than any published max_tokens figure. The Python SDK "will throw a ValueError if a non-streaming request is expected to take longer than approximately 10 minutes". Passing stream=True or overriding timeout disables that check. And "The token counts shown in the usage field of the message_delta event are cumulative", so they are read rather than summed.

The ordering is the part that stays stable, and Anthropic states it itself. As of 13 November 2026 the cost guidance reads: "From lowest to highest cost and capability, the current models are Claude Haiku 5.5, Claude Sonnet 5.5, Claude Opus 5.5, and Claude Fable 5.2 (the frontier model)". In October 2026 the same sentence ended with Claude Fable 5.1, and in September 2026 it began with Claude Haiku 4.5 and Claude Sonnet 5. That change is the reason to learn the family order and not the version numbers. Comparative latency is in the reverse order, so the three axes the guide names, quality, latency and cost, are not independent.

Everything in this table is correct as of its date and will change. It follows the Models overview comparison table:

From the 4.6 generation the dateless format is itself a pinned snapshot, so every id in the table is already pinned. Older models carry a snapshot date in the id instead: the older Claude Haiku 4.5 is claude-haiku-4-5-20251001, and claude-haiku-4-5 is a convenience pointer that resolves to it.

Documented roles, which follow from the ordering: Haiku is for "The lowest latency and price": real-time work, high-volume processing and sub-agent tasks. Sonnet is "The best combination of speed and intelligence" for code generation, data analysis and agentic tool use. Opus is for "Complex agentic coding and enterprise work". Fable 5.2 is "For demanding reasoning and long-horizon agentic work".

Capability was not uniform when the guide was written, which is why it names adaptive thinking support in its own scope. Claude Haiku 4.5, the fastest model then and an older model now, rejects adaptive thinking, using the older extended thinking form instead, and it has no effort parameter either. As of 13 November 2026 all four current models have adaptive thinking, Claude Haiku 5.5 included. A design that depends on the model regulating its own reasoning depth still cannot use any model without it, and Haiku 4.5 is one of them. Such a model is excluded on capability, before cost is discussed.

Start from the criteria the documentation names: Capabilities, Speed, Cost and Effort. The fourth is the one candidates skip, and it is a setting inside a model rather than a choice between models: "Several Claude models support an effort parameter that trades intelligence for latency and cost within a single model. Tuning effort is often a better lever than switching models."

Two documented starting strategies mark the two ends of the range of choices, and neither is a default:

• Efficiency-first: start with the fastest tier, test thoroughly, upgrade "only if necessary for specific capability gaps". Best uses: prototyping, tight latency limits, cost-sensitive and high-volume straightforward work.
• Capability-first: start with "the strongest starting point for your task", optimise the prompt for it, then consider "increasing efficiency by lowering effort or downgrading models over time". That is not the same as the most capable model: as of 13 November 2026 the page starts on Claude Opus 5.5 and moves to the frontier Claude Fable 5.2 only if evals at xhigh or max effort are still not good enough. Best uses: complex reasoning, "tasks requiring nuanced understanding", accuracy outweighing cost.

Then compare with the right measure and on the right sample. Measure: "You pay for completed tasks, though, so compare models on cost per completed task. A more capable model finishes a task with less work: fewer turns, less searching, less re-reading of its own context, and less backtracking." Sample: "Price the tail of your workload, not the median: compare models on the hardest tenth of your tasks, not the typical one." And the sentence that disproves every argument based on a price list: "The ranking flips by workload, and no price list tells you which way."

The eval is the test that decides. "Create benchmark tests specific to your use case - having a good evaluation set is the most important step in the process", run "with your actual prompts and data". A benchmark table or a price cut is not a test that can approve a migration, because published numbers are measured on somebody else's tasks. Vary one thing at a time, run each effort level in a separate session (changing top-level effort mid-session can invalidate all or part of the cache and distort the cost comparison), and remember a prompt tuned for the old model is not a neutral test of the new one.

An effort sweep (a test of each effort level) can end in a policy, not one level. As of September 2026 the cost guidance documents one for checkable work: "run every task at a low setting and re-run only the failures at a higher one". On its coding benchmark that slightly beat running everything at high, for a little over half the cost. It needs a failure signal you trust, and every first-pass failure takes the elapsed time of two runs.

The vocabulary is exact and testable. Legacy is not Deprecated and Deprecated is not Retired:

Anthropic commits to "providing at least 60 days' notice before model retirement for publicly released models", and "Every Claude model ID is a pinned snapshot". Two planning facts follow. First, a retirement date is a deadline on your roadmap, so you cannot stay on a model that works for ever. Second, because ids are pinned, nothing changes unless you change it, and that is what makes a planned migration possible at all.

A new model is a new system, and the differences come in three kinds:

• Behaviour shifts in named ways: default response length, how literally instructions are read, tool-calling tendency, readiness to delegate to subagents.
• Interface changes break code outright: manual extended thinking with budget_tokens, assistant prefill, and non-default temperature, top_p or top_k all return a 400 error on the newest models.
• Cost changes arrive even when nothing breaks, because a prompt written to compensate for an older model is followed literally by a newer one.

Interface changes fail with visible errors, but the expensive ones do not. Swapping the model string in one place and carrying the prompts, effort settings and thinking configuration across unchanged is how the silent failures happen. Run the eval suite, audit the prompts against the model you are moving to, test each effort level again, then run the new model in shadow, alongside the old one, on part of the traffic before the full switch.

A cost model is arithmetic over usage multiplied by real traffic: "price the five priced token counts in each response's usage at their own rates, summed across the task's requests" (wording as of September 2026; cache writes bill at the 5-minute or 1-hour multiplier). The top-level four are input_tokens, cache_creation_input_tokens, cache_read_input_tokens and output_tokens.

Two structural facts drive almost every decision after that:

• Output is priced several times input on every model. A workload that generates a lot relative to what it sends is output-dominated; one that sends a lot relative to what it generates is input-dominated; the fix for each is different.
• input_tokens does not mean all of the input. It counts the tokens after the last cache breakpoint, so on a cached request the real input volume is the sum of all three input fields. Read it alone on a heavily cached workload and you will conclude the prompt shrank when it did not.

Name the larger cost, input or output, before choosing a way to cut cost. A long standing prefix re-sent hundreds of times a day with short answers is an input problem, and shortening the answers targets entirely the wrong cost. A workload sending two hundred tokens and generating four thousand is an output problem, and caching the prompt changes the cost very little. Thinking tokens bill as output and are reported under output_tokens_details.thinking_tokens, so a reasoning-heavy workload is output-dominated even when its visible answers are short. Do the arithmetic on one representative day of real traffic, never on one request.

Tracking is engineering, not accounting. Three layers, three jobs. Per request, log the counts beside the request id and a feature label, on successes as well as failures. Per feature, alert on the ratio you expect: a cache hit rate, an input-to-output ratio, a cost per completed task. A regression should trigger an alert rather than appear on an invoice. Per organisation, the Usage and Cost Admin API is the authoritative aggregate. It is now a legacy API that remains supported, and new integrations should use the Usage and Cost Analytics API. It groups data "by API key, workspace, model, service tier, context window, data residency, or speed". It needs an Admin API key, is not available "to organizations on the claude.ai Team plan or to individual Claude Console accounts", surfaces data within about five minutes, and supports polling once a minute.

Measuring a change against a monthly invoice is the wrong answer. An invoice aggregates every workload and every traffic fluctuation, so nothing can be attributed to anything.

Domain 2 covers how caching is configured. Domain 5 covers whether it saves money, and the economics are asymmetric. Learn the pattern, and check the numbers when you need them. A write costs more than not caching, a read costs a fraction, and the longer lifetime costs more to write than the shorter one.

Caching "saves money after a small number of cache reads". With the 5-minute duration (1.25x write), it saves money after one cache read. With the 1-hour duration (2x write), it saves money after two cache reads.

The rule in words: caching saves money when a large, stable prefix is re-sent often enough within its lifetime to cover the cost of the write. Each of the four conditions matters.

• Large: prompts below the model minimum "are processed without caching" and return no error, so a small prefix silently does nothing. The minimums as of 14 November 2026: 512 tokens on Claude Fable 5.2, Mythos 5.2, Fable 5.1, Mythos 5.1, Opus 5.5, Opus 5, Sonnet 5.5, Fable 5, Mythos 5 and Haiku 5.5; 1,024 on Claude Opus 4.8, Sonnet 5, Sonnet 4.6 and Sonnet 4.5; 2,048 on Mythos Preview and Opus 4.7; 4,096 on Claude Opus 4.6, Opus 4.5 and Haiku 4.5.
• Stable: the prefix must match byte for byte, so a timestamp, a per-request identifier, an unsorted serialised object or a varying tool list ahead of the breakpoint drives the hit rate to zero silently.
• Often enough: a prefix used once loses money outright, paying the write premium and never collecting the read discount.
• Within its lifetime: the clock is "measured from the start of the request that writes or reads the cache entry, not from response end", it "is refreshed at no additional cost each time cached content is used", and generation time counts against it. A response that streams for minutes has used most of the shorter window before the next request begins.

Check-pointing a growing conversation. The request renders in a fixed order, tools then system then messages, and a change at one level invalidates that level and everything after it. The transcript of earlier turns is the stable part and the new turn is the volatile part, so the breakpoint belongs at the boundary and must advance as it moves; automatic caching does that advancing for you. With explicit breakpoints, two limits shape the design: at most 4 breakpoints per request, and a 20-block lookback window per breakpoint. A conversation that grows further past the last write than that lookback will miss, and the fix is a second breakpoint placed closer and from the start. Anything inserted mid-prefix invalidates everything downstream, which is why appending is cheap and editing history is expensive. Configuration counts as prefix: the thinking configuration and the resolved effort level render into the prompt, and switching between fast and standard speed does the same. As of 14 November 2026 a per-message effort change is the exception. It is a beta, on supporting models, and needs the mid-conversation-output-config-2026-07-01 header. An effort-only system message placed after Claude's last reply sets the new level from the next user turn that has new input. The cached prefix still matches.

Verifying is part of the change. A cache_read_input_tokens of zero where you expected a hit is a sign that something silently invalidated the cache, not a pricing surprise.

Anthropic splits them cleanly. Free wins "cut spend without touching quality: prompt caching, token hygiene, a prompt audit against the model you are running, batch processing at 50% off for work that can wait up to 24 hours, and workspace spend limits as the backstop." Trade-offs "exchange cost for intelligence: model choice, effort, output caps and session budgets, an elapsed-time clock, and multi-model architectures." A session budget is a dollar cap on what one Claude Managed Agents session can spend. At the cap, or slightly past it, the platform pauses the session.

Take the free wins first, in this order:

1. Prompt caching was usually the largest lever in Anthropic's measurements.
2. Token hygiene, meaning pruning stale tool output, retrieving fewer and better passages, removing unneeded transcript and shrinking oversized images. In the measured triage run it took that 83 per cent saving to 88 per cent.
3. A prompt audit against the model you actually run, removing instructions written for an older one that a newer one now follows literally.
4. Batch processing at a 50 per cent discount on both input and output, in exchange for asynchronous delivery inside 24 hours. Send through it any work nobody is waiting for, and no work that somebody is waiting for.

Then the trade-offs, where effort is the direct setting and a cheaper tier is legitimate for tasks that pass an eval with a stated pass mark.

And then the setting that does not lower cost per solved task. As of October 2026, max_tokens "caps a single response, invisibly to the model, so lowering it does not make the model economize. The turns that needed the room are discarded and still billed." In Anthropic's internal measurement a tight cap cut off a large share of attempts, and very few of those still passed. Attempts got cheaper and fewer tasks got solved, so cost per solved task barely moved. The correct handling is the opposite of the first idea most people have: set max_tokens high enough that nothing is cut off (the page suggests 64,000 for agentic work, or the 128,000 maximum when one cut-off attempt is costly), "treat stop_reason: max_tokens as a failure", and take the saving from settings the model can respond to.

A task budget is the cost setting the model can see. As of September 2026 it is a beta, set as task_budget inside output_config. It shows the model a running token countdown for the whole agentic loop. So the model trims low-value work and finishes rather than being cut off mid-action. It is advisory, so max_tokens stays the hard per-request limit. A mid-task change invalidates the cached prefix, so set it once on the first request. Size it from your own measured per-task usage. The Task budgets page suggests starting at the 99th percentile of that usage, then testing up or down.

Domain boundary: Messages-API mechanics, streaming event shapes and how cache_control is configured belong to 2.3; pinning and recording a snapshot is 2.6; prompt craft is Domain 6; failure diagnosis is Domain 4. Domain 5 covers the economics and the choice.