What You Need to Know
An optimisation reduces a Claude system's tokens, latency or cost. A lever is one such change. The dominant term is the biggest part of the bill or the wait. Every lever is useful for some workload. The skill is to match a lever to the workload's dominant term within the 4.1 quality bar.
This lesson uses one made-up example throughout: a car manufacturer whose finance team wants a smaller Claude bill without worse answers. The manufacturer runs a booking chat for service appointments, a supplier invoice pipeline and a nightly sort of customer reviews.
Rule One: Measure First
Traces and usage data identify the dominant term by stage, token type and call pattern. Anthropic's Usage and Cost Admin API programmatically reports usage by model, workspace and API key, and cost by workspace. Documented uses include cache-efficiency analysis and budget alerting. If most spend is input tokens on one stage, tuning output tokens elsewhere is no improvement. It is easy to optimise the most visible part of the bill instead of the largest part. Trimming the manufacturer's chat replies saves nothing while invoice input dominates.
Rule Two: Re-gate With Evals
The quality bar is a constraint. After each change, such as a cheaper invoice model, the evals run again. The change ships only if it is cheaper and still clears the bar. To ship without the rerun is a regression: the saving is measured, but the quality loss is not. Anthropic's effort guidance agrees: reduce effort where evals show that quality holds.
Token and Cost Levers
- Prompt caching suits repeated stable prefixes. A stable prefix is unchanged content at the start of each request: long system prompts, policy documents and tool definitions sent on every request. Cache reads are billed at a small fraction of the base input price. Cache writes are billed at a premium over the base input price (current figures are in the Domain 4 quick reference). The cached content must be a prefix that stays identical byte for byte, so put stable content first. Domain 2 covers the mechanics. The booking chat sends the same warranty policy and tool definitions every time, so it has a prefix to cache. Select caching only when the workload has a repeated prefix.
- Context curation sends only the fields that the task needs and excludes the rest of the file. The cheapest token is one that never enters the request. The invoice stage sends the supplier, line items and totals.
- Tight output contracts cap output tokens, which are the expensive ones. The invoice stage returns a fixed set of fields.
- Effort and reasoning budgets tune spend to the task within a single model. Use low effort for simple classification and high-volume work. Use high effort for complex reasoning. Anthropic states that tuning effort is often a better lever than switching models. Spend on thinking for a trivial task is waste. Sorting reviews into praise and complaints is simple classification, so the nightly review sort runs at low effort.
- Right-sizing per stage gives each pipeline stage the cheapest model that clears its measured bar. An escalation cascade sends most cases to the cheapest model and routes the hardest cases to a stronger model tier. This is Domain 2's pattern, and it is better than both a premium model everywhere and downsizing without evals. Most invoices go to a small model. The hard invoices escalate.
- Batch processing runs work that is not time-sensitive asynchronously, at a discount. Its turnaround and expiry windows are documented (figures in the Domain 4 quick reference). Anthropic describes batch processing as suited to large-scale evaluations, content moderation and bulk operations that do not need immediate responses. The cue is that no user is waiting. The words "overnight", "bulk" and "no one is waiting" point to batch. An interactive surface excludes batch. Nobody waits for the nightly review sort, so it can go to batch. The booking chat cannot.
Latency Levers
Match the lever to the latency that the consumer experiences. At an interactive surface, this latency is time-to-first-token. In a pipeline, it is total completion time (the 4.1 distinction).
- Streaming reduces perceived latency. Users see output while the model generates it, and Anthropic recommends streaming for responsive interactive experiences. Total completion time is unchanged, so streaming is the right lever only where a human is watching. It suits the booking chat and does nothing for the invoice pipeline.
- The smallest model that reaches the measured bar reduces real latency. It is the first recommendation in Anthropic's latency guidance.
- Parallelising independent tool calls removes serial waits. If the booking chat checks free appointment times and service centre opening hours as two separate calls, it can make both at once.
- Caching improves time-to-first-token for long stable prefixes, as well as cost.
Optimisations Trade One Cost for Another and Never Remove the Trade-off
Every lever spends something to save something. Streaming spends nothing but changes only perception. Batch spends turnaround time. Downsizing spends spare capability. Caching spends nothing only when the prefix is genuinely stable. Name the spend and the saving, with eval evidence that the spend is affordable.
In the exam
Professional-level optimisation questions match a lever to the question's dominant term. Two rules apply to every question on this objective, and most wrong options break one of them. In cost-versus-quality questions, right-sizing with a cascade is the balanced answer. An option that promises savings with no named cost and no eval rerun is a tempting wrong answer.
Exam Traps
Offering batch processing's discount for a latency-sensitive, user-facing flow
Batch is asynchronous, and its cue is the absence of a waiting user. An interactive surface cannot wait for a discounted answer, however large the saving. Check who is waiting before offering it.
Switching to the smallest model as a cost fix without re-running the evals
The saving is measured but the quality loss is not. Re-run the evals on the smaller model and ship it only if it still clears the bar.
Offering prompt caching for context-window overflow, wrong answers, or unique one-off documents
Caching reuses a stable repeated prefix to cut cost and time-to-first-token. It does not shrink the context, add capacity, or change what the model says. Document contents that never repeat cannot be cached, though a shared instruction prefix ahead of them can.
Practice Scenario
A nightly compliance sweep summarises 30,000 different documents on a premium model with per-request calls; finance flags the cost and wants it cut by changing the pipeline itself. Evals show a mid-tier model matches summary quality on a representative sample, and no model may replace the premium one unless tests show it meets that bar. What should the architect do?
Build Exercise
Run a Measured Optimisation Pass on One System
What you'll learn
- How to identify the dominant cost and latency terms from traces and usage data
- How to match levers to dominant terms and reject levers that do not move them
- How to state what each optimisation spends to save what
- How to define the eval rerun an optimisation must pass before shipping
- How to project savings at volume against a measured baseline
- Pull a week of traces and usage data for one Claude system and name the dominant cost term and the dominant latency term (stage, token type, call pattern)
Why: Without measurement the effort goes to the most visible line, not the dominant term — and the two are rarely the same.
You should see: Two named terms, for example: input tokens on the retrieval-augmented answering stage dominate cost; serial tool calls dominate latency.
- Match one lever to each dominant term — caching, context curation, output contracts, effort tuning, streaming, parallel tool calls, right-sizing with cascade, or batch — and reject at least one tempting lever that does not move your dominant term, with a sentence on why
Why: Lever selection is the examined skill, and the rejected lever proves the selection was reasoned rather than fashionable.
You should see: One lever per term with a mechanism sentence, plus one explicit rejection, for example: caching rejected — inputs are unique per request, no stable prefix.
- State what each chosen optimisation spends to save what (the cost-performance trade-off)
Why: Optimisations trade cost against performance; naming the spend is what makes the trade-off visible.
You should see: One sentence per lever: spends X (turnaround, spare capability, engineering effort) to save Y (unit cost, tail latency).
- Define the eval rerun: which 4.1 metrics must hold, on which dataset, before the optimisation ships
Why: The eval rerun is the difference between an optimisation and an unmeasured regression.
You should see: A named metric set, the eval dataset version, and the rule: ships only if every guard metric holds.
- Project the saving at volume (per task multiplied by projected volume) and compare it against the measured baseline
Why: Per-task savings that vanish at real volume, or engineering costs that exceed the annual saving, are findings the projection reveals.
You should see: A per-task delta, a volume projection, and an honest comparison against the baseline including the cost of doing the work.
Sources
- Prompt caching — Anthropic
- Batch processing — Anthropic
- Reducing latency — Anthropic
- Effort — Anthropic
- Choosing the right model — Anthropic
- Claude Certified Architect Professional Exam Guide (v1.0) — Domain 4 — Anthropic