LLM Fundamentals

What You Need to Know

Claude is built on an autoregressive language model, which writes text one token at a time. A token is a unit of the model's vocabulary. The context window is all the text the model can reference while it writes, including what it writes. Sampling is how the model picks each next token. Temperature sets how wide that choice is. Model options change how far the model reasons, how hard it works and how fast tokens arrive.

This lesson uses one made-up example throughout: an accounting software company that answers customers in a support chat inside its product. Each request carries the company's support policy pack, an extract of the customer's invoice records, the chat so far and the new question. Customers ask about support hours, invoice status and refunds.

Next-Token Generation, and the Three Things That Follow

Anthropic's glossary says models like this are pretrained "to predict the next word, given the previous context of text in the document". At inference, when it generates a response, the model emits one token at a time. Each token depends on the whole sequence so far, including the tokens it has just produced. Three consequences follow.

Generation is serial, so response time scales with how many tokens the model produces. Input length affects the prefill (the processing of the input before the first output token) and therefore time to first token. Output length affects the time until the last token. A long policy pack delays the first word a customer sees, and a long reply delays the last.

There is no lookahead and no revision. After the model emits a token, that token stays in the output. The model cannot return to an answer that it started badly and restructure it. So an output contract (a statement of the format the output must have) that you state at the start works better than a correction that you add afterwards. A worked example placed before the task works better than an instruction to fix the format if the format changes. If the company wants each refund reply to start with the amount, it shows that format first.

Each token affects the next, so an early error causes more errors and is not corrected. The model is not sampling independently at each position. It is continuing what it has already written.

Thinking is the same mechanism applied before the visible answer. The model generates reasoning tokens first. They are billed as output tokens and count toward max_tokens alongside the response text.

Tokens Are Measured, Never Estimated

Tokens are not units of human text. They "can correspond to words, subwords, characters, or even bytes (in the case of Unicode)". Code, JSON, tables, identifiers and non-English text all tokenise differently from English prose. So a rough rule calibrated on paragraphs gives the wrong price for a workload of log lines. The company's invoice extract is full of invoice numbers and account codes, so it will not cost the same as its policy prose.

The documentation set contains two different rough rules. The glossary says roughly 3.5 characters per token. The pricing FAQ says roughly 4 characters or 0.75 words. Learn that the two rough rules disagree. Do not memorise either figure.

Two methods answer two different questions. Before you send the request, POST /v1/messages/count_tokens accepts the same structured inputs as a real message, including system prompt, tools, images and PDFs. It returns a single field:

import anthropic

client = anthropic.Anthropic()

SUPPORT_CHAT_GUIDE = "You are the support chat assistant for an accounting product."
customer_question = "Invoice 214: sent on 3 March, still shown as unpaid."

count = client.messages.count_tokens(
    model="claude-sonnet-5-5",
    system=SUPPORT_CHAT_GUIDE,
    messages=[{"role": "user", "content": customer_question}],
)
print(count.input_tokens)  # {"input_tokens": 14} on the wire

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

const SUPPORT_CHAT_GUIDE = "You are the support chat assistant for an accounting product.";
const customerQuestion = "Invoice 214: sent on 3 March, still shown as unpaid.";

const count = await client.messages.countTokens({
  model: "claude-sonnet-5-5",
  system: SUPPORT_CHAT_GUIDE,
  messages: [{ role: "user", content: customerQuestion }]
});
console.log(count.input_tokens);

Counting is free. Token counting and message creation "have separate and independent rate limits", so counting costs neither money nor Messages API rate-limit capacity. The endpoint's answer is explicitly "an estimate", and it does not use caching logic. After the request, the usage object on the response is the true figure for billing.

The next point causes problems in many migrations. Token counts are tied to a tokenizer, and tokenizers change between model generations. Dated current state (August 2026): "Claude 4.7 and later models and Claude Mythos Preview use a newer tokenizer. The same input text produces approximately 30 percent more tokens than on earlier models". The count endpoint "returns the count under the tokenizer of the model you pass". So a count that you measured against last year's model is wrong for this year's model. It is wrong for both the cost model and the space left in the context window. Count against the model you will call, and recount whenever you change models.

The Context Window Is One Budget, Shared

The context window is not an input allowance. It "refers to all the text a language model can reference when generating a response, including the response itself". Everything shares the same space: "the system prompt, every message in messages (including tool results, images, and documents), and your tool definitions". The docs also say: "The output Claude generates for the turn, including its extended thinking, counts too". For the company, the policy pack, invoice extract, history and reply share one budget.

Two corollaries are often misunderstood. First, caching creates no extra space. The docs say: "Cached prompt prefixes still occupy the context window: prompt caching changes what you pay for those tokens, not whether they count". Second, thinking is not free space: its tokens count against max_tokens and are billed as output. A bigger context is also not automatically better: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot".

Overflow has two behaviours, which depend on whether the input alone is too big.

Both cases are arithmetic problems against a token budget. Their fixes are budget fixes: send less, retrieve fewer and better passages, split the job, or move to a model with a larger window. Neither is a prompt fault, and a more capable tier (a stronger and more expensive model) fixes neither. The counting endpoint above is how you stay inside the budget. A long support chat that fails before any reply needs a shorter history, not a stronger model.

Sampling, Non-Determinism, and the Dial That Is Now Gone

The model does not always pick the single most likely next token. It samples from a distribution, and temperature widens or narrows that distribution. The glossary says: "Lower temperatures result in more conservative and deterministic outputs that stick to the most probable phrasing and answers". The word deterministic there is comparative. The entry's next paragraph says so. "Even with temperature set to 0, the results will not be fully deterministic, and identical inputs might produce different outputs across API calls". The entry adds: "This applies both to Anthropic's first-party inference service and to inference through third-party cloud providers".

Temperature zero gives consistency, not reproducibility. No setting gives byte-identical output. Two customers asking the same question can get differently worded replies. Never build a component whose correctness requires two identical calls to return identical bytes. Instead, constrain the format with a worked example and an explicit output contract. Validate the response when your code receives it. Store an artefact that you need to be stable, rather than regenerating it and comparing the results. The company stores its refund policy summary once.

Model Options: Which Lever for Which Problem

Four options move a request along the same reasoning, latency and cost axis. They are not interchangeable.

Extended thinking is the original manual form: thinking that you enable with an explicit token budget. It is the older interface. The earlier Claude Haiku 4.5 still uses it, and the newest models reject it.

Adaptive thinking replaces extended thinking, and it delegates the decision to Claude. Setting thinking: {type: "adaptive"} lets Claude "determine when and how deeply to think based on the request". As of 14 November 2026, on Claude Opus 5.5, Opus 5, Sonnet 5.5, Sonnet 5, Haiku 5.5, Fable 5.2, Fable 5.1, Fable 5 and the Mythos models, "thinking is already on and needs no configuration". Use it when Claude answers a hard problem shallowly, such as a refund dispute that gets a one-line reply.

As of September 2026 the thinking configuration also takes a display field, which defaults to "omitted" on those models. Thinking blocks then arrive with an empty thinking field. They are still billed, and you still return them as before. If you set display: "summarized", you get a readable summary. You never get the raw chain of thought. To see the reasoning, read the thinking blocks rather than asking for it in the answer. A request can try to put the internal reasoning in the response text. As of 14 November 2026, on Claude Fable 5.2, Fable 5.1, Opus 5.5, Opus 5, Sonnet 5.5 and Fable 5, such a request can be refused. The refusal carries stop_details.category: "reasoning_extraction", and that category is billed even when the refusal arrives before any output.

Effort is the setting that you change most often, and it is the main option of the four:

chat_summary_prompt = "Summarise this chat for the support record: customer asked twice about a refund."

response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=8192,
    messages=[{"role": "user", "content": chat_summary_prompt}],
    output_config={"effort": "low"},
)

const chatSummaryPrompt = "Summarise this chat for the support record: customer asked twice about a refund.";

const response = await client.messages.create({
  model: "claude-opus-5-5",
  max_tokens: 8192,
  messages: [{ role: "user", content: chatSummaryPrompt }],
  output_config: { effort: "low" }
});

The levels are low, medium, high, xhigh and max, and the default depends on the model. As of 14 October 2026, "Most Claude models default to high effort", while "Claude Opus 5.5 and Claude Haiku 5.5 default to medium". "Setting effort to the model's default ("medium" on Claude Opus 5.5 and Claude Haiku 5.5, "high" on other models) produces exactly the same behavior as omitting the effort parameter entirely". If you omit it on Claude Opus 5.5, the request runs one level lower than it did on Claude Opus 5.

Two properties make effort the first option to try. It "applies to every output token", so "it works whether or not thinking is enabled". It covers tool calls and their arguments too. So "Lower effort also means fewer and terser tool calls", not merely shorter prose. It is "a behavioral signal, not a strict token budget".

Fast mode is the only one of the four that does not reduce quality. It "runs the same model with a faster inference configuration. There is no change to intelligence or capabilities". It delivers a documented multiple of the standard output rate at premium pricing. You enable it with speed: "fast" on the request. Only Claude Opus 5 and Claude Opus 4.8 also need the fast-mode-2026-02-01 beta header (a request header that turns on a pre-release feature). Its gain is in output tokens per second, "not time to first token (TTFT)". So the gain is visible on long streamed answers and almost invisible on short ones. Switching between fast and standard speed invalidates the prompt cache (the stored prompt prefixes that prompt caching reuses). As of 13 November 2026 fast mode is available on Claude Fable 5.2, Claude Mythos 5.2 and Claude Opus 5.5. It is a research preview (an early release) on Claude Opus 5. It is deprecated (marked for removal) on Claude Opus 4.8, and from 30 November 2026 a fast request to that model returns an error. The current throughput multiplier is on the Domain 5 quick reference page.

The selection rule is three sentences long. A hard problem that Claude answers shallowly needs more reasoning. A routine problem that costs too much needs less effort. An answer that is already correct but arrives too slowly for a person watching needs tokens that arrive faster. For that, use streaming first and fast mode where it exists, not a different model. So the company uses thinking for refund disputes, lower effort for support hours, and streaming for slow correct replies.

Zero-Shot, Single-Shot, Multi-Shot

Zero-shot is an instruction with no demonstration. Single-shot adds one worked input-output pair. Multi-shot adds several. The current documentation calls the same family "few-shot or multishot prompting". It states the lasting claim: "Examples are one of the most reliable ways to steer Claude's output format, tone, and structure. A few well-crafted examples... improve accuracy and consistency".

An example demonstrates a format and a decision boundary (the line between inputs that get one decision and inputs that get another) at once, in the same form that the model will generate. A description of a format is an indirect instruction that the model must translate first. Two real outputs shown side by side work better than a sentence describing what an output should look like. So the company shows two past refund replies.

Examples should be "Relevant: Mirror your actual use case closely". They should be "Diverse: Cover edge cases and vary enough that Claude doesn't pick up unintended patterns". And they should be "Structured: Wrap examples in <example> tags... so Claude can distinguish them from instructions". The documentation adds: "Include 3-5 examples for best results". Diversity is the test that candidates underrate. A set of examples that all share an incidental property teaches that property. If every example reply confirms that an invoice is paid, the model learns that replies never report an unpaid invoice.

Use examples when the failure is one of form or of boundary: inconsistent structure, unclear classification edges, or a tone that changes. Do not use them when the failure is one of budget (a request that will not fit) or of fact (something the model cannot know). Examples fix the company's changing reply layout. They cannot supply an invoice total that is in none of its inputs. Examples cost input tokens on every call, which is why they interact with 5.4. A stable example block at the front of a prompt is the kind of prefix that prompt caching is for.

In the exam

Domain 5 is named for model selection, but that is its smallest sub-skill. Two-thirds of the marks are in the fundamentals beneath it: how the model produces text, and what the layer beneath your call is doing. This lesson covers the first half. Most of this first half is a property of the model rather than of a release. So what you learn about the mechanism will not become out of date. Do not rely on a tokens-per-word ratio.

The exam's questions test all three consequences of next-token generation. Under a latency constraint, a candidate who confuses input length with output length chooses the wrong option. The guide names the four model options together. The lasting knowledge is which option addresses which complaint, as in the company's three cases. Writing and iterating the prompts themselves is Domain 6.

Dated Facts

Where a fact is dated, this lesson says so. Every parameter name, level list and model list under Model Options is dated and is on the Domain 5 quick reference page.

Exam Traps

Practice Scenario

Build Exercise

Sources

• Glossary — Claude Docs, tokens, latency, TTFT, temperature and the non-determinism statement
• Context windows — Claude Docs, the shared budget, cached prefixes, thinking tokens and both overflow behaviours
• Token counting — Claude Docs, the endpoint, its estimate status, its rate limits and the tokenizer change
• Thinking — Claude Docs, adaptive thinking, the display setting and the sampling-parameter restriction by model
• Refusals and fallback — Claude Docs, the refusal categories including reasoning_extraction, and which ones are billed
• Effort — Claude Docs, the levels, the per-model default and what effort affects
• Fast mode — Claude Docs, speed, the beta header, OTPS versus TTFT and cache invalidation
• Model deprecations — Claude Docs, the API parameter deprecations table
• Prompting best practices — Claude Docs, examples as a steering technique and the relevance, diversity and structure tests
• Claude Certified Developer Foundations Exam Guide — Domain 5, LLM Fundamentals — Anthropic