What You Need to Know
A risk register has one row for each way a system can fail. It is an architecture artefact as well as a document: each row changes the build. Every row has five fields.
- Mechanism. What goes wrong, in the language of the system rather than of assurance (statements that a risk is under control).
- Blast radius. How far the consequence reaches, who bears it, and whether it can be undone.
- Detection signal. What changes in the telemetry (the logs and metrics that the running system records) while it is happening.
- Owning control. Which control owns the risk, and which of the surfaces in objective 5.1 that control sits on.
- Residual, and who accepted it. A risk with no owning control is an accepted risk. Acceptance is a decision that somebody signs. Domain 6 takes the risk to that person.
This lesson uses one made-up example throughout: a retail bank. Its Claude agent answers customers, books branch appointments and refunds fees. One row in the bank's register is this: the agent gives a customer a wrong deadline for disputing a card payment. The customer misses the real deadline, and that cannot be undone.
Anthropic's Responsible Scaling Policy has the same structure at a lower layer. It defines capability thresholds and commits to upgrading safeguards when one is reached. The policy is like a register row with a threshold and a pre-agreed response, not a promise to be careful.
Confident Fabrication Does Not Fail Loudly
The characteristic failure of these systems is fluent, plausible and unflagged. This failure never shows in an error rate, because nothing errored.
Anthropic's hallucination guidance says that its techniques significantly reduce hallucinations but do not eliminate them. It also says that critical information must always be validated. So any consequential output, like the bank's dispute deadline, needs a verifiable basis, not a plausible one. This need for a verifiable basis is why "we tested it thoroughly and it worked" is not a risk assessment.
Non-Determinism, and the Assurances It Breaks
The same input can produce different outputs. Three consequences follow. One passing test proves nothing. One clean re-run does not exclude a hypothesis. An audit that reproduces a decision months later needs the recorded trace. It needs the trace because a re-run of the prompt produces a fresh answer, not the original answer. A customer complaint about a wrong dispute deadline needs the stored conversation.
Injection, Restated as a Risk
Objective 5.1 treats untrusted input as a control problem. On the register, untrusted input becomes a row. The blast radius of that row is the permission set of the agent. A compromised turn on an agent that can only read and summarise is a contained incident. On an agent that has a payment tool, like the bank's refund tool, a compromised turn is a breach.
So the row names the permission set, which you control. It does not name the injection technique, which will change.
Autonomy Compounds Error
In an agent loop, a modest error rate per step compounds. Every additional autonomous action widens the blast radius. Anthropic says that autonomy means higher costs and the potential for compounding errors. Anthropic recommends extensive testing in sandboxed environments with guardrails. It says that agents suit tasks with clear success criteria and meaningful oversight. It also says that agents can pause for feedback at checkpoints or when blocked.
So the factors that you can control are scope, step count, reversibility and checkpoints. Accuracy is not one of them. The bank's agent should pause before refunds.
The Three Rows Registers Usually Omit
Over-refusal. The cost of every safety control is the legitimate requests that it denies. A safety programme with no over-refusal metric improves only one measure and reports that as progress. That is why the safety metric in Domain 4 measures both directions.
Over-reliance downstream. A reviewer who approves everything is not a control. The Information Commissioner's Office (ICO) is the data protection regulator of the United Kingdom. The ICO names automation bias: users rely routinely on a system's output and stop applying their own judgement. So rubber-stamping (approval without a check) is the default unless the gate is designed against it. Objective 5.3 covers that design.
Third-party surface. Every connector, MCP server, subprocessor (a company that a vendor uses to process your data) and analytics vendor in the path can see data and can sometimes act. Anthropic's retention documentation puts third-party integrations entirely outside its arrangements. It tells customers to review the data handling of each service. The access of these third parties is your risk posture. Their agreements are your compliance posture. Objective 5.4 covers that compliance posture. The bank's appointment booking connector gets a row in the register.
Rank by Consequence and Detectability
Rank risks by consequence and detectability, not by likelihood. A rare irreversible action outranks a frequent reversible one. A silent failure in an uninstrumented path outranks a loud failure that anybody would notice quickly. The wrong dispute deadline outranks a double-booked appointment.
The Column That Most Registers Get Wrong
"How would we notice?" is a required field, and it is the field that teams skip. Any row whose honest detection signal is "a customer would complain" has no detection. That signal fires after the harm. It comes through a channel that only detects the harms that people report. Mark those rows. The next incident comes from those rows.
In the exam
A per-item accuracy figure quoted for a multi-step autonomous workflow tells you almost nothing.
Exam Traps
Accepting a high accuracy figure as the risk posture for a system that takes actions
An accuracy figure measures one thing. It says nothing about blast radius or the detection signal, so what a wrong action costs and how the team would notice one stay unknown. Register each irreversible action with a control, an owner and a detection signal.
Ordering register rows by how often each failure is expected to fire
Likelihood ranks a frequent, loud failure above a rare, irreversible, silent one. Rank by consequence and detectability, so the irreversible action with no detection signal sits first.
Adding a disclaimer or a notice to an action that has already left the organisation
A surface-only fix placed after the point of harm. The action has already reached the recipient; the wording neither prevents the error nor detects it, and transfers no consequence away from them. Put a checkpoint before the action and add a detection signal.
Recording a human review step as the owning control with no override rate behind it
Automation bias makes an unmeasured gate a rubber stamp by default. A control whose effectiveness has never been observed is an assumption on the register, and the residual it is supposed to close is still open. Measure the override rate before relying on it.
Practice Scenario
An agent will chase overdue invoices: it reads the ledger, drafts an email, sends it, and can apply a late-payment charge. The team's risk assessment reads, in full, tested on 200 historical invoices, 98% accuracy. The sponsor wants to know what a wrong email or charge would cost and how the team would notice one. What should the architect require before launch?
Build Exercise
Write the Risk Register for One Claude System
What you'll learn
- How to separate irreversible actions from reversible ones and name who bears each consequence
- How to turn the lasting classes of limitation into concrete rows for a specific system
- How to give every row a mechanism, a blast radius, a detection signal, an owning control and a named accepter
- How to rank by consequence and detectability instead of likelihood
- How to test a register against the question of what would change in the telemetry first
- List the actions your system can take that cannot be undone, and write next to each who bears the consequence and what their recourse is
Why: Irreversibility and recourse decide the design. Starting here stops the register from becoming a list of engineering worries.
You should see: A short list of actions with a named bearer and a recourse route, including at least one row where the recourse is honestly none.
- Work the limitation classes one at a time and write a concrete row for each: non-determinism, confident fabrication, corpus and knowledge boundaries, untrusted input, over-refusal, compounding error under autonomy, over-reliance downstream, third-party access
Why: The classes are the part of this objective that you must remember. Writing a row for each one makes you use that knowledge in the design.
You should see: Eight rows, or fewer with an explicit written reason why a class does not apply to this system.
- Give each row its five fields: mechanism, blast radius, detection signal, owning control with its 5.1 surface, and the name or role that accepted the residual
Why: A row missing the owner or the signal has no effect, and those are the two fields teams leave blank.
You should see: A complete table with no empty cells in the owner column.
- Rank the rows by consequence and detectability rather than likelihood, then mark every row whose real detection signal is that a user would complain
Why: The risks that cause the most harm fail with no signal in paths that nobody instrumented. The ranking makes that visible.
You should see: A reordered register with the silent, irreversible rows at the top and the complaint-driven rows flagged.
- Test the register one row at a time against a single question: if this happened tonight, what would change in the telemetry before anyone told us?
Why: It is the fastest test of whether the detection column is real, and it usually shows that two or three rows need instrumentation work from Domain 3.
You should see: A named metric, log or alert per row, and a list of the rows that needed more instrumentation.
Sources
- Reduce hallucinations — Anthropic
- Building effective agents — Anthropic
- Responsible Scaling Policy — Anthropic
- API and data retention — Anthropic
- How do we ensure individual rights in our AI systems? — Information Commissioner's Office
- Claude Certified Architect Professional Exam Guide (v1.0) — Domain 5 — Anthropic