Optimising Context Windows and Token Usage

What You Need to Know

The context window is everything that Claude sees in one request. The window is measured in tokens. Every token in the window is billed on every request where it appears. The window holds four things:

• the system prompt
• every message in the history, including tool results, images and documents
• the tool definitions
• the output that Claude generates, including thinking tokens (the tokens of Claude's built-in reasoning)

This lesson uses one made-up example throughout: an insurance company whose claims team uses a Claude assistant. The assistant's system prompt, policy-lookup tool, conversation and policy documents are all in the window. All of them are billed on every request.

Counting the Window

Two documented, free instruments count tokens. The usage field on every response reports what the request consumed. With caching, input tokens are of three kinds: plain, cache-read and cache-write. All three kinds count towards the window. The token counting API estimates a request before you send it. The API's count is inexact, and the API has its own rate limits. Estimate requests before you send them, and make this part of the design. For example, the insurance company counts its largest policy-document request before launch.

Overflow of the window is a hard failure, and it has two distinct cases. Oversized input is rejected outright. On current models, generation that reaches the limit stops early, with its own documented stop reason (the response field that says why generation stopped). You can detect both cases, so set alerts on them. Plan the token budget so that routine requests never approach either case.

Context Rot: Capacity Is Not a Strategy

Each model has a window size from a set of tiers, and output has its own limit. A bigger window gives more space, but it does not decide what that space holds. Anthropic's docs explain why: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot. This makes curating what's in context just as important as how much space is available." Anthropic's engineering guidance calls context an attention budget. The goal of that guidance is the smallest possible set of high-signal tokens (tokens that carry information the task needs) that maximises the likelihood of the desired outcome.

So ask whether every token is worth including, not whether it fits. If the insurance company loads its whole policy library into a million-token window, this fails in three ways:

• context rot degrades answer quality
• the company pays for the full load on every request
• latency grows with input size

The Replay Bill

Each turn resends the whole history. Over 20 turns on one insurance claim, a 50k-token history is 50k input tokens twenty times. At high volume, careless context costs more than model choice and becomes the largest cost line. This makes context strategy a Domain 2 economics decision. Curation, caching and compaction each reduce a different part of this replay bill.

The Management Toolkit

• Curate first. Send the fields, not the file: send the policy fields that a question needs, not the whole policy record. Summarise verbose history and remove formatting noise. Anthropic's long-context guidance says to put longform documents, such as a policy document, at the top of the prompt, above the query. The cheapest optimisation is to leave out tokens that the task does not need.
• Retrieve instead of sending everything. When the source corpus is larger than a sensible context, retrieval brings only the relevant passages for each request. An example is the two policy documents that a question needs. Whether to choose retrieval is a Domain 2 cost-and-context judgement. How to design the RAG pipeline (the system that retrieves the passages) is Domain 3.
• Compact long conversations. Server-side compaction (beta, Claude 4.6-generation and later models) summarises earlier turns into a compaction block. The conversation can then continue past the window limit. A beta feature is not yet final. Compaction is Anthropic's recommended primary strategy for long-running conversations and agentic sessions, such as a full working day on claims. There are two kinds (checked 29 October 2026). With on-demand compaction, your application sends a request that asks for the summary. With threshold compaction, the API writes the summary when input tokens reach a trigger threshold. Anthropic's compaction docs say to use on-demand compaction wherever it is available.
• Edit agentic context. Context editing (beta) clears the oldest tool results (and optionally thinking blocks, the parts of a response that hold Claude's reasoning) server-side once a threshold is reached. It replaces them with placeholders. Use it where the full history adds nothing, such as policy lookups from hours earlier.
• Make smaller savings with structure. Tight output contracts make responses shorter. Minimal tool definitions make the fixed overhead smaller. So do deferred definitions, which load only when a request needs them.

Truncation looks like optimisation. Truncation cuts context to a fixed token count, whatever the task needs, and it loses quality silently. Curation selects by relevance. Truncation cuts by position. If you cut a long policy document to its opening, the later sections disappear unflagged. Optimisation must preserve the information that the task needs.

In the exam

The exam tests whether you treat the window as a budget that you actively manage or as capacity to fill. When cost and quality problems share one question, distinguish them. Some options offer "Enable caching" as the fix for window overflow or for a decline in quality late in the conversation. Such an option is a deliberate wrong answer. Truncation is also a deliberate wrong answer. A wrong option in the guide's own sample question cut a policy to its first 1,000 tokens.

Current window sizes, output limits and stop-reason identifiers are in the Domain 2 quick reference.

Exam Traps

Practice Scenario

Build Exercise

Sources

• Context windows — Anthropic
• Token counting — Anthropic
• Compaction — Anthropic
• Context editing — Anthropic
• Effective context engineering for AI agents — Anthropic
• Claude Certified Architect Professional Exam Guide (v1.0) — Domain 2 — Anthropic