Skip to content
Domain 2
21%

Glossary: Output Evaluation and Validation

Quick-lookup definitions for the 21% exam domain. Each entry includes a concise definition and exam context. Follow the lesson links to dive deeper.

Reviewer's Stance

The working attitude toward every Claude output: a strong first draft from a brilliant colleague who can be confidently wrong. Fluency and confidence are properties of the writing, not evidence about the facts, and the professional who sends the work is responsible for it.

Exam context: The foundation of the exam's heaviest-weighted domain. Right answers check the output against its source before use; wrong options send it because it reads well or because Claude sounded sure.

See also: 2.1 Evaluating Outputs for Accuracy and Completeness


Accuracy Pass

The first review question: are the specifics right? Every number, name, date, title, quoted figure and reference gets checked against the source material. The checks are ordered by consequence. The figure a decision depends on is checked first.

Exam context: A draft with twenty accurate sentences and one wrong subtotal is a wrong draft if the subtotal is what the meeting decides on. Under time pressure, the right answer checks the most costly details first, not the easiest ones.

See also: 2.1 Evaluating Outputs for Accuracy and Completeness


Completeness Pass

The second review question: is anything missing? It is checked by following the source material's own structure, item by item: its section list, funds, themes or line items. Each element is confirmed as represented in the output or knowingly excluded.

Exam context: Omissions are not visible in the prose, so completeness can never be checked by reading the output alone. Wrong options verify one named figure while the question hints at a dropped section; the right answer opens the source.

See also: 2.1 Evaluating Outputs for Accuracy and Completeness


Traceability

The property of an output whose claims can be followed back to their origins. Built by asking Claude to indicate where in the attached material each figure or claim came from, then spot-checking the important claims.

Exam context: A traceable output is checkable in minutes; an untraceable one is not assumed to be correct. Anthropic's prompting guidance backs the habit: for long documents, ask Claude to quote the relevant parts of the source before carrying out the task.

See also: 2.1 Evaluating Outputs for Accuracy and Completeness, 2.3 Fact-Checking and Validation Techniques


Hallucination

Content that is fluent, specific and fabricated: quotes, figures or instructions that look authoritative or sound convincing but are not grounded in fact. An invented detail arrives with precisely the same confidence as a real one.

Exam context: Specificity is not evidence. The question to ask is never "does this sound right?" but "where would this have come from?", and the right answer verifies against an authoritative source.

See also: 2.2 Identifying Hallucinations, Inconsistencies and Bias


Capability Hallucination

Claude falsely claiming to have done something outside the conversation: sent an email, saved a file elsewhere, produced a working external link. Claude has no access to tools that are not explicitly integrated.

Exam context: A completion claim is a claim to check, not a delivery confirmation. Wrong options treat "the email has been sent" as evidence the email exists; the right answer verifies that the action actually happened.

See also: 2.2 Identifying Hallucinations, Inconsistencies and Bias


Grounding

The material Claude actually had to work from. Hallucination risk depends on grounding: it is highest for private internal systems Claude was never given documentation for, events after its training data, and precise figures that were not in the source material.

Exam context: Confident numbered instructions for an organisation's own software are the highest-risk kind of all: fluent, specific and based on no source material. The less grounding Claude could possibly have, the less its specificity means.

See also: 2.2 Identifying Hallucinations, Inconsistencies and Bias


Inconsistency

The output disagreeing with itself: a number in the prose that contradicts the table, two sections giving different dates for the same event, a recommendation that contradicts the findings.

Exam context: An internal contradiction is a reliability warning about the whole output, not a typo to patch. The right answer re-verifies the entire output against its source; fixing the one mismatch you noticed and sending the rest is the tested wrong answer.

See also: 2.2 Identifying Hallucinations, Inconsistencies and Bias


Bias

Skewed framing, assumptions or language, most consequential in outputs about people. It is found in the pattern across descriptions, not in any single sentence: who gets caveats, whose achievements get qualifiers, which groups are framed in different terms.

Exam context: Read across descriptions rather than sentence by sentence, and apply the read-aloud test: would the framing still be acceptable if read aloud to the group it describes? The right answer reworks the pattern; deleting one biased word leaves the skew intact.

See also: 2.2 Identifying Hallucinations, Inconsistencies and Bias


Independent Validation

Checking a claim against a source that does not share the output's errors: the primary document, the official publication, the regulator's text, the organisation's own records. Independence is the property that makes validation mean anything.

Exam context: Web search is a legitimate means of reaching an authoritative source, but the source does the validating, never the search. Citations exist to be checked, not to make the output look better.

See also: 2.3 Fact-Checking and Validation Techniques


Circular Verification

Checking a claim against something that inherits the errors of the thing being checked: a second AI summary, a forum post, or Claude's own earlier output in the same conversation.

Exam context: A source that seems independent but is not is a common wrong option: an option that "checks" the draft against the summary Claude produced earlier. It proves nothing, because no independent information enters the check.

See also: 2.3 Fact-Checking and Validation Techniques


Self-Check

Asking Claude to check its own work. It can be a direct request, or a re-run of the same request that you compare with the first output. It can also be a follow-up that asks Claude to verify its earlier statements. It can find arithmetic slips, contradictions and figures that differ between runs. It gives you a list of items to check against the source. It cannot confirm a fact, because no new information enters the check.

Exam context: A self-check offered as verification is a common wrong option. The guide's own sample rationale says that self-reported confidence is not a reliable accuracy signal. So a confident "yes, it's correct" is never the right validation answer. A self-check whose findings you then check against the source can be part of a right answer.

See also: 2.3 Fact-Checking and Validation Techniques


Calculated Figure

A figure that Claude worked out rather than took from the source: a total, a percentage, an average, a change from last year. The source does not contain it, so the check against the source cannot find an error in it.

Exam context: Correct input numbers do not prove a correct result. The right answer checks the input numbers against the source. Then it does the calculation again. It does not accept the figure or ask Claude to confirm it.

See also: 2.3 Fact-Checking and Validation Techniques


Proportionality

Matching validation effort to consequence: the statistic in a public funding bid gets independently verified in two places, the phrasing in an internal ideas list gets a read-through.

Exam context: Tested in both directions. Too little checking on the claim a decision depends on is the obvious failure, but verifying everything to the maximum is a real wrong answer too, not a safe one: it spends limited checking time on prose nothing depends on.

See also: 2.3 Fact-Checking and Validation Techniques


Escalation Triggers

The properties that make an output need review by a person qualified in its subject before anyone uses it. Any one is enough:

  • an external audience for claims in the organisation's name, where an error reaches many readers at once
  • legal, financial or contractual consequence
  • decisions about people, including references
  • safety or health implications
  • irreversibility once sent

A routine reply to one customer, with facts the author can check, does not have the external-audience trigger.

Exam context: Notice what is absent: how confident the output sounds and how well it reads. Polish is the one property every Claude draft already has, so it never appears among the triggers.

See also: 2.4 When Human Review or Additional Verification Is Required


Qualified Reviewer

The person competent in the output's subject: legal competence for contract language, the accountant for tax figures, the clinician for clinical claims. Anthropic's Usage Policy sets the standard for High-risk AI Recommendations: a qualified person with the training or experience to evaluate the output in that field, and the authority to change it.

Exam context: Seniority and availability are neither necessary nor sufficient. When an option names a reviewer, check their qualifications, not their rank; a busy director skim-reading a lease clause is comfort, not review.

See also: 2.4 When Human Review or Additional Verification Is Required


Over-Escalation

Routing routine, low-stakes drafts through formal review: the committee for a note about the staff schedule, the lawyer for every internal email. It gives reviewers too much work, delays routine work and weakens the escalation channel that high-stakes outputs genuinely need.

Exam context: A tested wrong answer, not a cautious right one, because candidates rarely expect caution to be penalised. The skill being certified is triage: deciding how much review each output needs; the right answer lets careful self-review suffice for genuinely low-stakes material.

See also: 2.4 When Human Review or Additional Verification Is Required


Audience Adaptation

Reshaping an accurate output for its named reader: vocabulary down to the reader's level, sentences simplified, ordering changed so the urgent item leads. The substance, findings, recommendations and costs, must remain after the reshaping.

Exam context: Both failure directions are tested: sending output that is hard for the reader because its content is correct, and simplifying a serious point until the reader cannot see it. The right answer adapts delivery, then checks the new version against the original for anything lost.

See also: 2.5 Editing, Adapting and Comparing Outputs for the Audience


Criteria-First Comparison

Fixing the judging criteria before reading any version: nothing factual lost, the right register for the named reader, urgent items first. Then reading against the list and defending the choice against the criteria, not the style.

Exam context: Comparison without criteria favours the best-sounding prose, which is how a fluent version with less content is chosen over a correct one. Fluency is never a criterion, because it is the property both versions already have.

See also: 2.5 Editing, Adapting and Comparing Outputs for the Audience


Generator, Not Judge

The division of labour for variants: Claude produces the versions and articulates how they differ; the human applies the criteria and picks. Asking Claude which of its versions is better passes to Claude the exact judgement the objective certifies.

Exam context: Wrong options have Claude recommend its own best version and the user send it. The right answer keeps the choosing with the human, for the same reason self-check fails in validation: self-assessment is not a reliable signal.

See also: 2.5 Editing, Adapting and Comparing Outputs for the Audience


Curation

The deliberate step from working material to work product: starting from what the deliverable must give its reader, extracting exactly that from the conversation, and shaping it into its final container. Most of a working conversation is scaffolding; the deliverable must give its reader the decisions, not the steps that led to them.

Exam context: Forwarding the whole conversation delegates the curation to a reader who will not do it. The right answer extracts what each reader needs into the container that fits their use.

See also: 2.6 Curating Information and Selecting Output Formats


Three Format Homes

The selection rule for output shape, driven by one question: what does the reader do with this? Ask-and-move-on wants an inline answer, compare-and-scan wants a structured table, reuse-and-revise wants an artifact.

Exam context: Options that use one format for everything (everything a table, everything an artifact) sound systematic and are wrong in both directions. Feature and model choice in a format question belongs to Domain 3 and is a wrong option; the right answer is always about the container the reader receives.

See also: 2.6 Curating Information and Selecting Output Formats


Artifact

Claude's dedicated window for substantial, standalone content: a document, plan or reference page that has a use outside the conversation. It is changed by asking Claude or by editing it directly. Editing an earlier message branches it. It can be shared as a link when it must be sent outside the conversation.

Exam context: The artifact suits content you will edit, iterate on or reuse outside the conversation. The reusable procedure left as plain text in the chat gets lost and re-created; the one-line answer put in an artifact gains needless formality.

See also: 2.6 Curating Information and Selecting Output Formats