What You Need to Know
Decomposition splits one task into subtasks. Each subtask gets its own model pass. A pipeline chains those subtasks as stages. Decompose a task that is too large, too varied, or too important to trust to one pass.
This lesson uses one made-up example throughout: a payments company that processes card payments for online shops. Customers dispute some payments. The company wants a draft answer for each dispute, based on the dispute's file of payment records and customer messages. It also wants a monthly check of all its dispute files.
Fixed vs Dynamic Decomposition
Fixed decomposition happens at design time: the architect splits the task into known stages and builds a pipeline. The stages are known, so the pipeline is predictable and you can test it stage by stage. Fixed decomposition suits structured, repeatable work, such as processing documents, review pipelines and generating reports. Its workflow pattern is prompt chaining.
Dynamic decomposition happens at run time: the model discovers the task's structure as it works. The model generates subtasks from its findings. Dynamic decomposition suits open-ended work whose structure appears during the work: investigation, research, exploratory analysis. Its workflow pattern is orchestrator-workers (objective 1.4).
The test is this: can you write the steps in advance? If yes, fix the steps in code, and you gain predictability. If no, let the model decompose the task. You gain adaptability, but you lose predictability. The company's dispute answer passes this test. A question about why disputes rose last month fails the test, because each finding decides the next step.
The Four Decomposition Axes
The real design decision is how to split the task. That decision determines the workflow pattern.
- By stage: sequential phases with clear hand-offs, such as extract, analyse, draft, review. Use this axis when later work depends on earlier output. The dispute answer splits this way, because the decision needs the list of facts first.
- By section: independent units that are processed in parallel, such as one unit per document, per file or per region. Use this axis when units do not interact. The monthly check splits by dispute file.
- By perspective: the same input analysed from different viewpoints (security review plus performance review), or analysed repeatedly, with voting for confidence (a finding counts when most runs report it). Use this axis when one viewpoint systematically misses what another catches. The company could check one dispute file once for fraud and once for a shop error.
- By specialism: subtasks that need different tools, prompts or models. Use this axis when one configuration cannot serve all subtasks well. Reading a photo of a receipt and reading payment records could need different prompts.
Attention Dilution
Attention dilution happens when one pass covers too much material. The depth of the pass becomes inconsistent. It gives detailed findings on some items and makes obvious misses on others. It gives contradictory judgements on identical patterns in different places. Consider one monthly pass over fourteen dispute files that is thorough on three files and misses everything in five files. That pass has a decomposition problem, not a model capability problem. It might flag a missing delivery record on one dispute and miss it on another.
The fix is structural, and it has two mandatory parts. Neither part is enough alone.
- Per-unit local passes: the model analyses each file, section or document alone. Every unit gets a full pass, so depth is consistent on every unit. Each dispute file gets its own pass. Without the integration pass, these local passes miss everything between units.
- A separate integration pass: this pass searches for the cross-unit issues (data flow between files, rules that span contract sections). Local passes cannot see these issues by design, because each one sees only its own unit. For the company, such an issue is ten disputes that describe the same late delivery from one shop in different words. Without local passes, a single integrated pass dilutes again.
The Bigger-Model Wrong Answer
Exam questions that show inconsistent depth offer a larger model or bigger context window. The problem was never that the material did not fit. The problem was uneven attention. A model that holds 400 pages still dilutes its attention across 400 pages. The answer is always local passes plus integration.
Interface Discipline
Between stages, intermediate outputs should be structured against a defined schema, not free prose. A schema has three effects.
- you can test each stage independently, because you can check its output against the schema
- downstream stages stop re-interpreting prose
- attribution (source locations, clause references) passes through the pipeline as fields, such as the message reference on each of the company's fact records
Add acceptance gates between stages: programmatic checks that stop bad intermediate output from spreading. Unchecked errors grow worse across stages. One hallucinated field (a value the model invented) in stage two becomes a confident conclusion in stage five. The company's gate checks in code that every payment ID exists before the decision stage starts.
When Not to Decompose
Every split adds latency, cost and an interface to maintain. Errors also propagate across stage boundaries. Decompose a task when one of these conditions is true:
- quality measurably improves
- the task cannot fit one context
- stages need independent verification
A task that fits comfortably in one call and shows no symptoms of dilution should stay in one call. An example is the company's answer to a dispute about one payment that was charged twice. A five-stage pipeline for that task is pure overhead.
In the exam
The Professional exam tests decomposition as a design skill with named failure modes. The exam tests attention dilution most heavily. The exam scores a five-stage pipeline for a one-call task as over-engineering.
Exam Traps
Splitting a task that already works in one call into a multi-stage pipeline because staging looks more rigorous
Every split adds latency, cost and an interface to maintain, and errors propagate across stage boundaries. A smaller pipeline is still unnecessary if no evaluation shows its calls are needed. Decompose when quality measurably improves.
Fixing inconsistent-depth symptoms with a larger model or bigger context window
The problem was never that the material did not fit. A model that holds the whole corpus still dilutes across it. The fix is structural decomposition, and the exam offers the bigger-model option as a favourite wrong answer here.
Decomposing into local passes only, with no integration pass
Cross-unit issues, such as data flow between files or rules spanning sections, are invisible to local passes because of how such passes work. The two-part fix is mandatory: local depth plus cross-unit integration.
Passing free-text summaries between pipeline stages
Downstream stages re-interpret prose, attribution is lost, and stages cannot be tested independently. Structured intermediate outputs with acceptance gates fix all three at once.
Practice Scenario
A compliance team must check 400-page loan agreements against 60 policy rules. A single-prompt approach misses violations inconsistently — some sections are analysed thoroughly, others barely. What should the architect design?
Build Exercise
Decompose a Document-Analysis Pipeline
What you'll learn
- How to choose a decomposition axis and defend it against alternatives
- How to design local passes plus an integration pass against attention dilution
- How to define structured intermediate outputs that preserve attribution
- How to write acceptance gates that stop error propagation between stages
- Choose a large-document analysis task: contract review, codebase audit, or long-report QA
Why: Attention dilution needs scale to appear, and the fix needs scale to matter. Pick something a single pass genuinely struggles with.
You should see: A one-line task statement including the size of the material and what a finding looks like.
- Pick the decomposition axis — stage, section, perspective, or specialism — and defend it against the other three in one sentence each
Why: The axis is the design decision; the pattern follows from it. Defending against alternatives is what makes the choice deliberate rather than habitual.
You should see: Four sentences: one justification and three rejections, each naming a property of the task.
- Define the intermediate output schema for the local passes, including source-location fields
Why: Structured interfaces make stages independently testable and carry attribution through the pipeline. Prose interfaces lose both.
You should see: A JSON schema for one finding: what was found, where (section, clause, or file and line), against which rule, at what confidence.
- Write the acceptance gate that sits between local passes and integration: what makes a local finding admissible
Why: Acceptance gates stop bad intermediate output propagating. Errors compound across stages if unchecked.
You should see: Two or three programmatic checks: schema validity, source location resolves to real text, rule identifier exists.
- Specify the integration pass: which cross-unit issues it searches for and why local passes cannot catch them
Why: The integration pass is the half of the fix designs forget. Its scope must be explicit or it becomes a second large pass over everything, with diluted attention.
You should see: A short list of cross-unit issue types — rules spanning sections, contradictions between sections, aggregate thresholds — each with a line on why it is invisible locally.
Sources
- Building Effective AI Agents — Anthropic
- How we built our multi-agent research system — Anthropic