What You Need to Know
A guardrail is a control that stops a Claude system causing harm. Guardrails sit on five control surfaces. Each surface is at a different position on the path of a request. Each risk belongs on the one surface that can control it.
This lesson uses one made-up example throughout: an insurance company whose customer assistant reads claim documents that customers upload and has a tool that cancels policies.
Five Control Surfaces, Read by Position
Input controls screen what enters the system: injection and jailbreak patterns, personal data that should never have been submitted, and out-of-scope or abusive requests. A jailbreak is an attempt to make the model ignore its rules. Only input controls can stop a payload before it reaches anything. Anthropic advises that a lightweight model pre-screens the input and gives its verdict as a structured output.
Instruction-layer controls are the scope, refusal policy and escape hatch in the system prompt. They shape behaviour, and they are worth having. The insurance company's prompt limits the assistant to questions about policies and claims.
Capability controls decide what the system may do. They decide which tools exist, which credentials those tools have, and which actions need a second party. Approval by staff for each cancellation is one capability control.
Output controls validate the response before it leaves: schema and policy checks, blocked types of claim, required citations, and redaction of outgoing text.
Architectural removal means not building the capability at all. A tool that was never built cannot be misconfigured, degrade or need an audit. The insurance company could decide not to build the cancel tool.
An Instruction Is Not an Access Control
A system prompt is text that the model reads. If a capability is connected, an instruction not to use it is a request. Anthropic's hardening guidance puts prompt policy beside input validation and least privilege (the system has only the capabilities that its role requires). It never puts prompt policy in their place.
Test any design with one question: what stops a confused or compromised turn from reaching the connected capability? If the answer is "the prompt says not to", no control is responsible for the risk. When the cancel tool is connected, the insurance company's prompt line "never cancel a policy" is only a request.
Untrusted Input Is Everything the System Did Not Author
Two threat models apply. In a direct jailbreak, the user is the adversary. In an indirect injection, the user is trusted. The adversarial instruction arrives inside third-party content: a retrieved document, a tool result, a connector payload (the data that a connected service returns) or an uploaded file. An honest customer's claim document is third-party content too.
The mitigations that work are structural, not persuasive:
- Untrusted content arrives inside tool results and nowhere else. The result states where the content came from. Its encoding leaves no doubt about where the content starts and ends. Then an attacker's text stays data and cannot pass as an instruction.
- The system prompt states that tool content is data to report, not commands to follow.
- Tool output is screened before the model acts on it.
- Access is scoped so that a successful injection has limited effects.
Screening reduces how often injections pass. Scoping limits the consequence. Detection helps, but least privilege is reliable. If the cancel tool had never been built, a planted instruction that passes the screen could not use it.
A Guarantee Has Only Two Homes
"This must never reach a customer" is a guarantee. Wording cannot guarantee an outcome. A guarantee exists at the output gate or in a capability that you never built. Where a property must be true every time, Anthropic's consistency guidance recommends structured outputs instead of prompt wording.
The gate covers the claim classes that the organisation has already decided about: financial commitments, clinical or legal advice, and guarantees of outcome. For some types of claim, you do not make the decision. A provider usage policy sets minimum requirements for high-risk uses. The gate blocks a reply, redacts it or routes it to a person. The insurance company's gate would send to staff any reply that promises to pay a claim. A gate that only records is a detective control (one that finds harm after it happens), even if it is called a preventive control.
Defence in Depth, and Why Removal Still Wins
The layers are independent, so that the failure of one layer does not let harm reach a user. For that reason, Anthropic's advanced guardrail material uses a harmlessness screen, a system prompt that states values, and a policy check in a chain. The harmlessness screen is a lightweight model that pre-screens the input.
Defence in depth does not permit you to use the wrong layer to control a consequence. Capability scoping or a human gate (a person who must approve the action first) governs an irreversible external action, never firmer wording plus a weekly sample. A cancelled policy cannot be undone, so cancellation needs staff approval or no cancel tool.
In the exam
In the Domain 3 sample in the guide, a support agent has refund and delete tools that its users never need. The right answer removes those tools. Logging is a wrong option: it is a detective control. A confirmation prompt is a wrong option too. It is a compensating control: it lowers the risk but leaves the unneeded tools in place. A bigger model is a wrong option that is irrelevant to the risk. Every option is a real control, and one option is the control that the risk needs. Nearly every guardrail question has that pattern.
Find the phrase in the question that decides the surface: never reaches, before it is sent, cannot be undone, after the fact. That phrase usually decides the answer. The prompt instruction is the most common wrong option in this domain. It appears again in Domains 3 and 7. Domain 3 implements capability controls. Domain 5 decides that they must exist.
Extra Checks on an Unnecessary Privilege
When a question says that the role never needs a capability, some options add a confirmation step, an alert or an approval. Those options are real controls, but they solve a problem that the question does not describe. Extra checks on an unnecessary privilege leave the privilege in place. The answer key of the guide ranks removal above every guard in front of that privilege.
The guide publishes no Domain 5 sample. The Domain 3 sample is the closest example of how Domain 5 questions are written.
Exam Traps
Strengthening the system prompt to forbid a behaviour the system is still able to perform
The prompt is context, not enforcement. A connected capability stays reachable through error, manipulation or injection. Anthropic's own injection guidance treats prompt policy as one layer beside screening and least privilege. It does not treat prompt policy as the enforcement point.
Moving to a more capable model to stop policy violations or resist injection
Model size is unrelated to authorisation scope. A stronger model changes nothing about what the configuration permits, so the capability stays wired. Scope the capability or remove it.
Logging every response, or scheduling a periodic review, against a requirement that the harm must never reach a user
A detective control finds the harm after it has happened. A requirement written as a guarantee is satisfied at the output gate or by removing the capability. When the review team eventually reads the sample, the sample is evidence that the control failed.
Adding a confirmation prompt, approval or alert in front of a capability the role never needs
A confirmation is a compensating control, so the unnecessary privilege stays in place. Removing the capability the role does not require eliminates the attack surface, which is why the exam guide's sample answer chooses removal over a confirmation.
Practice Scenario
A pharmacy chain's customer assistant answers questions about the medicines it stocks. Clinical governance requires that no reply ever states a dose, a frequency or a duration; those questions must reach a pharmacist. Which change satisfies the requirement as stated?
Build Exercise
Map the Control Surfaces for One Claude System
What you'll learn
- How to state a risk in the language a user or regulator would use rather than in control jargon
- How to assign each risk to the one surface that can actually control it
- How to test whether a second, independent layer really exists
- How to find every risk currently controlled only by wording
- How to red-team a design so scoping, not phrasing, is what stops the payload
- Take a Claude system you operate, or design one, and write its five named risks in the language a user or a regulator would use
Why: If you write a risk in control jargon, the wording already names a control. Plain language leaves you free to choose the surface.
You should see: Five sentences a non-engineer would recognise, for example: a customer is told something about their medication that only a pharmacist should say.
- Assign each risk to exactly one owning surface: input screening, instruction layer, capability scoping, output validation, or removal
Why: Choosing one owning surface makes you use the judgement that the exam tests. If a risk has two owning surfaces, usually neither control has been designed.
You should see: Five assignments with a one-line justification tied to how the risk actually manifests.
- For each assignment, write what happens if that layer fails, and name the independent layer that catches it
Why: Defence in depth is a claim about independence. Writing the failure case is how you find out whether the second layer exists or was assumed.
You should see: A failure sentence per risk, and a blank, not an assumed layer, where there is no second layer.
- Find every risk currently carried by an instruction alone, then either move it to an enforcing surface or record it as an accepted risk with a named owner
Why: Wording that nobody has agreed to accept is the normal state of most systems, and nobody notices it. It is the wrong answer that questions in this domain include most often, here in a real system.
You should see: A list of instruction-only risks, each now moved to an enforcing surface or accepted in writing by a named person or role.
- Red-team the result: write three untrusted payloads, one in a retrieved document, one in a tool result and one in an uploaded file, that try to reach a consequential action
Why: Anthropic recommends testing your own agent with deliberately adversarial content before deployment, and it is the fastest way to see which layer actually stops the attack.
You should see: Three payloads, each with a note saying whether scoping or wording stopped it.
Sources
- Mitigate jailbreaks and prompt injections — Anthropic
- Increase output consistency — Anthropic
- Usage Policy — Anthropic
- Claude Certified Architect Professional Exam Guide (v1.0) — Domain 5 — Anthropic