What You Need to Know
A service level is a promise about how fast, available or accurate a system is. The envelope is the performance that the real pipeline measurably achieves on the real corpus (the documents and data the system uses). Engineer first, then promise. Never promise first.
This lesson uses one made-up example throughout: a pharmacy chain whose Claude assistant answers customers' order and price questions from its price list and order records. Suppose that the head of customer service names a reply time in a meeting and gives it to engineering. That reply time is a promise that will cause problems later. Measure instead. Use Domain 3's latency decomposition (the time that each pipeline step takes) and Domain 4's percentiles under realistic concurrency (many requests at the same time). Leave headroom, which is spare capacity for load that the measurement never saw.
The Anatomy of a Defensible Commitment
A figure on its own is not a commitment. Six elements make the figure something that people can check and act on:
- The metric: what is measured, and how it is computed.
- The percentile or threshold, because nobody experiences the mean.
- The measurement point: where the clock starts and stops. This point separates a promise about the model from a promise about what the user experiences.
- The window: over what period compliance is assessed, so one bad hour cannot breach a year and one good week cannot hide a bad month.
- The population: who and what the commitment covers. Define it honestly before the measurement, not after it.
- The breach behaviour: what happens when it is missed.
The pharmacy chain's clock should start when the customer sends a message. Anthropic's success-criteria guidance applies the same rigour to quality. It uses a named held-out set of stated size, which is test data that is kept apart during development. It also uses a quality floor (a minimum quality level), a safety rate and a response-time bound.
You Cannot Promise Above Your Dependencies
Before you write an availability or tail-latency clause, check what the underlying platform commits to. Anthropic documents its standard service tier as best-effort availability. A prioritised tier has an explicit uptime target. But it needs a committed capacity arrangement, and that arrangement is no longer generally purchasable. Traffic beyond that commitment uses the standard tier. A promise above the envelope of the platform underneath depends on somebody else's best effort. The pharmacy chain's uptime promise cannot exceed the uptime of its tier.
Negotiating the Impossible Ask
When a stakeholder wants a number that the pipeline cannot achieve, show the latency decomposition. Then the figure is no longer arbitrary. State the envelope that the pipeline does achieve. Then reduce the part of the gap that you honestly can reduce: perceived responsiveness. Anthropic's latency guidance separates baseline latency from time to first token. It recommends streaming because streaming improves how responsive an application feels. Progressive disclosure (showing information in stages), partial results and an asynchronous path with notification use the same method. None of them changes the total completion time.
Accepting the number is giving in to an impossible demand. It is worse to delete the retrieval or verification steps that the answer depended on to meet the number. Then a measured problem becomes an unmeasured problem. If the chain dropped its price lookup, replies would be faster and wrong prices would be unmeasured.
Quality Is a Measured Claim
An accuracy commitment needs the same six elements. "It will be very accurate" fails in the same way as a duration on its own. Where the shortfall is concentrated in a segment that you can identify, say so. Commit on the population that you genuinely measured. Give that segment its own improvement plan, evidence and owner. One such segment would be order questions from clinics, where answers miss the quality floor while answers to individual customers meet it.
The Trap That Looks Like Rigour
Narrowing the population until the main number becomes true leaves the excluded cases in production. They have no threshold, no review path and nobody accountable. The precise schedule (the contract section that lists the service levels) hides the carve-out: the excluded cases. Narrow only where the exclusion is visible and nobody is left uncovered by accident.
Feedback Is Instrumentation
A feedback channel must produce evidence and a disposition (a decision about what happens to the report). It turns a report into a trace, a case identifier or a reproducible input. A pharmacist's "it gets things wrong sometimes" cannot be investigated; the trace of the wrong price can. Triage needs a visible outcome. Otherwise the channel teaches everyone to stop reporting, and silence is mistaken for quality. Confirmed failures route into Domain 4's regression set. Anecdotes routed straight into prompt edits make a system worse over time. Each edit fixes the reported case and changes another case that nobody mentioned.
Reports start with the success criteria that were agreed at discovery. Engineering measurements support those criteria. Scope changes need the same discipline. Weigh each request, such as adding invoices for clinics, against the agreed criteria and baseline. Present its cost, date and risk to the sponsor before anyone commits. Then reset the plan with the sponsor's signature.
In the exam
The right answer matches a measured envelope and has all six elements. It also streams responses where a human is waiting. Capitulation and deleting a control are wrong options. Hidden narrowing of the population is the most dangerous option in this objective. Visible narrowing can be the right answer.
Sourcing Notes
The six-element shape is professional practice, not Anthropic documentation. The tier details are a snapshot from one date, for the quick reference.
Exam Traps
Writing a bare number into a schedule as the service level
A figure with no percentile, measurement point, window, population or breach behaviour cannot be assessed or operated. Commit on the measured envelope, with all six elements stated.
Meeting a latency target by dropping a retrieval or verification stage
The time target is met while the error rate changes without being measured, so the promise that is met is not the promise that was made. Show the decomposition and commit to the envelope the full pipeline holds.
Offering streaming or partial results where the consumer is a pipeline rather than a person
Those methods change perceived responsiveness, not total completion time, so with nobody watching the measured figure is unchanged. Negotiate the number against the measured decomposition; keep streaming for surfaces where a person waits.
Quietly narrowing the covered population until the main figure becomes true
The excluded cases stay in production with no threshold, no review path and nobody accountable. Hidden narrowing is the most dangerous option in this objective; state any exclusion and give that segment its own plan and owner.
Practice Scenario
An insurer's service-centre sponsor wants a flat maximum response time written into the schedule for a correspondence-drafting assistant: every draft back inside the same short window, no exceptions. Measurement on the real pipeline holds that window for routine cases but runs well past it on multi-policy customers, a small and permanent share of the queue. Service staff wait at the screen while the draft is produced. Any promise must rest on what the pipeline has been shown to do, cover the whole queue and leave draft quality as it is. What should the architect commit to?
Build Exercise
Write the Service-Level Schedule and the Sponsor Page
What you'll learn
- How to derive a commitment from a measured envelope rather than a demonstration
- How to write a latency commitment carrying all six elements
- How to check a promise against the service tier underneath it
- How to reduce a perceived delay with experience design rather than by deleting a control
- How to lead a sponsor report with the criteria that were agreed at the start
- Measure or estimate the envelope for a Claude system you know, then write one latency commitment with all six elements: metric, percentile, measurement point, window, population, breach behaviour
Why: A commitment derived from an envelope is still met after the first real week. One derived from a demonstration is a problem that will appear later.
You should see: One sentence a lawyer could put in a schedule and an engineer could alert on.
- Write one quality commitment to the same standard, including who reviews the residual errors, within what time, and at what cost
Why: An accuracy promise fails exactly the way a latency promise fails, and the residual errors are certain rather than hypothetical.
You should see: A threshold on a named population, plus a review path with an owner and a cost.
- Check both commitments against your dependencies — the service tier the system runs on and what it actually promises — and mark anywhere your promise exceeds the limits that those dependencies promise
Why: An availability or tail-latency clause written above the platform underneath it depends on somebody else’s best effort.
You should see: A dependency line per commitment, and at least one honest note where the promise is weaker than it looked.
- Design the experience that reduces the perceived delay for the request you could not meet: streaming, progressive disclosure, partial results or an asynchronous path with notification
Why: Perceived responsiveness is useful on interactive interfaces and irrelevant everywhere else, so the choice has to be justified by who is waiting.
You should see: One method chosen, with the sentence naming whether a human is watching the response arrive.
- Draft the sponsor page led by the success criteria discovery agreed, with the engineering measurements underneath as support, then delete anything that is on the page only because it was easy to measure
Why: A sponsor who agreed to a business outcome and receives infrastructure telemetry is no longer getting the information they need, and more of it makes that worse.
You should see: A page whose first section is an agreed criterion with its trend, and a shorter appendix than you started with.
Sources
- Reducing latency — Anthropic
- Service tiers — Anthropic
- Define success criteria and build evaluations — Anthropic
- Claude Certified Architect Professional Exam Guide (v1.0) — Domain 6 — Anthropic