What You Need to Know
A human gate is where a person checks or approves an AI action before it takes effect. Placement is where the gate sits. Pattern is the form oversight takes. Blast radius is how far a wrong action's harm spreads. Placement comes before pattern. The gate belongs where the consequence is high and the action is hard to reverse.
This lesson uses one made-up example throughout: an insurance company. Its agent tags overdue accounts, updates customer contact details, issues refunds and cancels policies that are unpaid. A wrong tag can be corrected. A paid refund or a cancelled policy cannot be corrected: the money has left, and the customer has no cover.
If you gate all of it, every tag waits for a person. Then the system delivers nothing that anyone will continue to pay for. If you gate nothing, a probability decides whether irreversible harm happens. Reversibility and blast radius decide the placement, not seniority, discomfort or a fixed percentage.
Anthropic's agent guidance agrees: an agent pauses for feedback at checkpoints or when it meets a blocker. That guidance places the pause. It does not put a pause everywhere.
The Five Patterns
Approve-before-execute. The system proposes an action. A human authorises it. Then the action runs. This pattern is for irreversible external effects: payments, deletions, outbound customer messages, production changes, and anything with a legal or financial consequence. The company's refunds and policy cancellations use this pattern.
Review-before-release. The system drafts, a human approves, and the output is released, and that person is accountable for it. It suits consequential content: regulated advice, clinical documentation, statements that commit the organisation.
Triggered escalation. The system handles routine cases. It routes the uncertain, out-of-policy or high-value case to a person. The pattern depends on the trigger, because only the cases that the trigger selects reach anyone. An unvalidated confidence score only looks like a control. So validate it like any other classifier.
Post-hoc sampled audit. Nothing is gated. A sample is checked afterwards, and that check shows whether the quality claim is true. This pattern fits work that is high-volume, reversible and low-severity per item, such as the company's overdue-account tags. The audit also shows when reviewers stop reading. That stops the other patterns from becoming less effective.
Automate with a reversibility guarantee. This pattern has no gate. An audit trail, an undo path or a compensating transaction makes the action cheap to undo. A compensating transaction is a second transaction that cancels the effect of the first. The company's logged contact-detail updates fit this pattern.
Some Gates Are Not Discretionary
Anthropic's Usage Policy decides some placements before any architecture work. It names eleven High-risk Areas. Among them are legal and medical advice, and decisions on credit, insurance, housing and employment. A recommendation that Claude makes in one of these areas is a High-risk AI Recommendation. The policy requires two controls for it. Before the recommendation is delivered as advice or used to make a decision, a qualified person must review it properly. That person must have the authority to change the recommendation. The person that the recommendation concerns must be clearly told that AI was used to produce it.
Both controls apply to a recommendation about one person: advice that the person might rely on, or a decision about that person. The policy also lists exclusions: general educational information, and wellness content on sleep, stress, nutrition or exercise. Internal drafting, research, summarisation or analysis that is not the final recommendation delivered is also excluded. So are business operations that do not advise or decide about anyone in particular. So check what the output does, and to whom, before you assume that the policy requires a human review. A summary for the company's managers decides nothing about any one customer, so the policy does not apply to the summary. A reply about one customer's account is about a person. The next question is whether the subject of that reply is in one of the areas.
Under Anthropic's commercial terms, the customer is responsible for judging whether outputs are appropriate. This responsibility includes judging where human review is appropriate.
Making the Reviewer Effective Rather Than Decorative
The Information Commissioner's Office (ICO) is the data protection regulator of the UK. UK data protection law has rules for decisions that are solely automated: no person takes a real part in them. The ICO's test: a decision does not fall outside the automated-decision rules just because a human has approved it without checking it. Reviewers must check the recommendation rather than routinely apply it. They must have the authority and competence to overrule the system.
Most gates fail one constraint. If reviewers see only the data that the system saw, they weigh no additional factors. Then the decision may be treated as solely automated. So the reviewer gets the trace, the sources, the flagged uncertainty and at least one input that the model never had. The reviewer also gets enough time to use them. Before a cancellation, the company's manager also reads the customer's last call notes.
A Gate Is a Control With a Performance Number
Three numbers matter: override rate, reviewer agreement and time per item. An override rate that rounds to zero at high daily volume shows that reviewers approve without checking. Either reviewers cannot see enough, or the gate sits in the wrong place. A manager who passes hundreds of refunds a day without changes is not checking them.
The ICO says explicitly that training reviewers is not the main fix for automation bias. Controls start at scoping and design. At that stage, the business, the data scientists and the oversight function agree in advance what the system considers and what the human weighs.
The person who signs is accountable. The person who approves owns the outcome, so they must see the basis for it. This is 5.5's explainability obligation from the reviewer's side.
In the exam
Questions test which of the five patterns to select. Automate with a reversibility guarantee is a legitimate right answer wherever the question says the action is low-impact and reversible. In a High-risk Area, the question changes "should there be a human here" into "where in the flow does the qualified person sit, and what do they see". The second-best answers are about reviewer design.
The Right Answer Usually Gives Each Path Its Own Pattern
When a question gives both a throughput constraint and a reversibility distinction, the right answer splits the flow. If one pattern applies uniformly, it fails half the requirement. If you gate everything, the backlog stays. If you gate nothing, the irreversible path is unguarded. Pick the option that gives each path its own pattern. The company does this when it separates contact-detail updates from policy cancellations.
The list of High-risk Areas changes. Read the current list instead of memorising it.
Exam Traps
Requiring human approval for every action the system proposes
If the scenario gives a throughput limit, volume forces reviewers into routine approval, so the gate produces the rubber-stamping the ICO warns about. Gate only the irreversible path and let the reversible path run on an undo guarantee or a sample.
Placing the human check at a periodic retrospective review when the harm has already left the organisation
Post-hoc sampling suits high-volume reversible work. A sample shows whether a quality claim is true; it does not recall a payment, a deletion or a message to a customer. Where the effect leaves the organisation, approve before the action executes.
Strengthening a sign-off gate by adding a second reviewer or a more senior one
An added reviewer still sees only the drafted output, so two reviewers give two rubber stamps, and seniority adds authority, not information. Give the reviewer the trace, the sources, an input the model never had and time per item, then measure the override rate.
Escalating to a human whenever a model-reported confidence score falls below a threshold nobody validated
The pattern depends on the trigger. An unvalidated score is a classifier with unknown error rates choosing which cases a person sees, and its misses go unseen. Validate it against outcomes on labelled cases, counting the escalations it missed.
Practice Scenario
A payroll assistant investigates pay queries and can issue corrections. Small corrections are added to the next monthly pay run and stay fully reversible until it closes. A correction paid immediately, outside the monthly run, leaves the organisation, and the only way to recover it is to ask the employee to return it. The team currently answers queries three weeks late. The single payroll manager cannot check every correction, but no payment may leave the organisation unchecked. What is the right oversight design?
Build Exercise
Design and Stress-Test the Human-Oversight Layer
What you'll learn
- How to classify every action by reversibility and by the window in which reversal is still possible
- How to select between the five oversight patterns on evidence rather than on what feels safe
- How to recognise when a qualified reviewer is required rather than optional
- How to specify the evidence and the time budget a reviewer needs to be effective
- How to set the override rate at which you would conclude the gate has no real effect
- List every action your system can take and mark each reversible or irreversible, with the window during which reversal is still possible
Why: The window is where teams get this wrong. Plenty of actions are reversible for an hour and permanent afterwards.
You should see: An action inventory with a reversal window per row, including the rows where the window is zero.
- Assign one of the five patterns to each action and justify it from reversibility and blast radius rather than from seniority or what feels safe
Why: Pattern selection is the skill the exam tests, and writing the justification exposes the assignments made out of nervousness.
You should see: One pattern per action with a one-line justification that mentions reversibility or blast radius by name.
- Check the regulatory requirements: if the use case makes High-risk AI Recommendations, a qualified person must review each one before it is delivered or acted on, so place that person explicitly and write what they see
Why: In those areas the reviewer is not a design option, so an architecture that leaves them out is wrong before the trade-off is even weighed.
You should see: Either a named reviewer role positioned in the flow with their evidence pack, or a written finding that the use case falls outside those areas or within an exclusion.
- For every gate, specify the evidence the reviewer receives, including at least one input the model did not have, and the time budget per item
Why: The ICO test is that a reviewer restricted to the system data is arguably not weighing additional factors, which makes the decision effectively automated.
You should see: An evidence list per gate, with the additional input named, and a seconds-or-minutes budget that is still possible at the daily volume.
- State the override rate you expect if the gate is real, name the rate at which you would call it a rubber stamp, and write what you would change first when you see that number
Why: A gate with no expected number cannot be audited, and deciding the response in advance stops people dismissing the finding later with an excuse.
You should see: Two numbers and one pre-committed action, recorded where the team that runs the gate can see them.
Sources
- Usage Policy — Anthropic
- Commercial Terms of Service — Anthropic
- How do we ensure individual rights in our AI systems? — Information Commissioner's Office
- Building effective agents — Anthropic
- Claude Certified Architect Professional Exam Guide (v1.0) — Domain 5 — Anthropic