DOMAIN 416% of the exam
Evaluation, Testing & Optimisation
Define evaluation metrics, build datasets and test frameworks, run A/B tests, and diagnose and optimise live Claude systems.
- 4.1Defining Evaluation Metrics
- 4.2Designing Evaluation Datasets and Test Frameworks
- 4.3A/B Testing and Iterative Improvement
- 4.4Diagnosing Prompt Failure, Hallucination, and Model Mismatch
- 4.5Optimising Token Usage, Latency, and Cost-Performance Trade-offs
- 4.6Monitoring System Performance in Production
How others did on this domain
56%
The median score on this domain across 409 first attempts at the CCAR-P mocks. Half scored between 0% and 78%.
See where you would land
5 full-length mocks of 63 questions in 120 minutes, each scored out of 1,000 against the real pass mark.