Reliable use of Claude requires more than asking for a response in the right format. Production systems need outputs that mean the right thing, represent uncertainty honestly, and can be checked before another program acts on them. That challenge grows when hundreds of documents must be processed consistently or when a single review pass misses important errors. Few-shot examples, asynchronous batches, and multiple review passes solve different parts of the problem; combining them indiscriminately adds cost without ensuring accuracy.
The strongest architecture begins by defining what correct work looks like. For a contract-classification system, the desired output may include the contract type, renewal date, monetary obligations, and evidence quotes. Those fields require a schema, but each field also needs a semantic rule: how is a renewal date distinguished from an expiry date, and what should happen when the contract uses both?
A structured-output schema can define the field and type; it cannot decide whether a renewal date describes an optional extension or an existing legal obligation. That interpretation needs examples that expose edge cases and verification against the original document.
Write task criteria before collecting examples
Examples are useful only if they illustrate a rule the model can apply to new input. Start with a rubric stating what must be extracted, how ambiguous evidence should be handled, what is out of scope, and when the correct response is “unknown.” A vague instruction such as “identify the most important date” invites the model to choose whichever date is prominent; an explicit instruction can prioritize the cancellation deadline while reporting renewal and termination dates separately.
Define edge cases before writing the happy-path prompt. A document might include multiple currencies, a superseded clause, a handwritten amendment, or a conflicting schedule. The correct output may need to preserve both candidate values with their supporting locations rather than quietly select one. When the application has a definitive business rule, put it in the task specification. When only a human can resolve the ambiguity, make escalation part of the expected output.
Task criteria should include provenance. If a model extracts a penalty amount, require the source paragraph, document identifier, and confidence expressed as evidence quality rather than a fabricated probability. Reviewers can then verify claims without reading a full model transcript. Clear criteria also create the baseline for automatic evaluation later.
Use few-shot examples to teach distinctions, not style alone
Few-shot prompting works best when examples cover failure-prone differences: a true renewal clause versus a routine date reference, a refundable deposit versus a nonrefundable fee, or a customer request versus an instruction found inside the customer’s attached document. Show the relevant input, the correct structured answer, and enough explanation to reveal the criterion being applied. Repeating several near-identical happy-path examples rarely offers the same value.
Examples should reflect the data likely to appear in production. If a contract system handles scanned documents, examples taken solely from perfect digital text can hide OCR errors, unusual tables, and ambiguous headings. Include a case with missing information, one where two candidate values conflict, and one where the answer must be left blank. In many workflows, demonstrating appropriate non-answering is more useful than demonstrating fluent completion.
Do not embed secrets, actual customer identifiers, or private contractual terms in reusable examples. Create realistic synthetic examples or properly approved, de-identified fixtures. Keep them versioned with the prompt and schema; an example that teaches an older classification rule can silently undermine an updated instruction. Changing a few-shot set should therefore trigger regression tests just like changing application code.
Separate output format validation from factual validation
A JSON schema can reject a missing property or wrong value type. It cannot prove that a quoted contract passage actually says what the model claims. After schema validation, run deterministic checks for allowed currency codes, date relationships, required evidence locations, and rules that can be evaluated without a model. Then review claims that require interpretation, either through a second bounded model pass or human expertise according to the risk of the decision.
A useful extraction schema can include value, source_span, status, and reason_unresolved fields. The status might distinguish observed, inferred, conflicting, and absent values. Downstream services can then reject inferred or conflicted values for sensitive operations while displaying them for human review. Avoid a single yes/no confidence switch that hides the distinction between unavailable evidence and a low-quality inference.
Structured output does not grant authority. An extracted payment instruction is untrusted information until the application verifies the customer, relevant account, allowed action, and approval. If a workflow proposes changing records, those tool and permission boundaries must be enforced outside the model even when the output passes every schema rule.
Choose asynchronous batching when immediacy is not required
Many applications process large document collections rather than individual live requests. Anthropic’s Message Batches API supports asynchronous groups of independently specified requests, with a request identifier used to associate each result with the correct source item. This can be appropriate for nightly classification, historical document processing, and evaluation runs that do not need an immediate response. It is not the right mechanism for an interactive request whose user expects a result before continuing.
Each item should have a stable application ID separate from its position in the submitted array. Batches may complete out of order, and individual requests may succeed, fail, or expire independently. The ingest pipeline should therefore persist the mapping from source record to request identifier and record a per-item outcome. Never assume that the tenth returned result belongs to the tenth original document. A failed item can be retried deliberately after inspection without replaying everything.
Batching does not eliminate input security or governance requirements. Apply tenant access rules before collecting documents, respect the applicable retention conditions, and avoid mixing unapproved sensitive data merely because the API supports high throughput. Track processing status and retrieve results only through approved credentials. Message Batches have service-defined size limits, expiration behavior and result retrieval semantics; verify current values before promising processing times or costs.
Design multi-pass review around independent error detection
One pass can extract candidate facts; a second can verify each candidate against source spans; a third may adjudicate disagreements or risk. These passes should have distinct objectives. Asking the same model to “look again” with the same context is not automatically independent review. It may reproduce the first answer, including its hidden assumptions. Better verification uses original evidence, explicit test criteria, and a check designed to detect specific failure classes.
For example, a primary extractor produces fields with citations. A validator confirms that each cited passage exists and supports the field, while deterministic checks identify impossible dates or missing mandatory values. A policy reviewer evaluates whether the document type allows automatic processing. The system then escalates cases involving conflicting evidence, high financial consequence, or a missing clause. If the first pass is wrong, later stages have a chance to catch it because they are not rewarded merely for agreeing.
Multi-pass systems should capture disagreement instead of forcing consensus. If two reviewers interpret a clause differently, record their competing readings and the evidence behind each. For high-risk cases, a human with relevant expertise decides. For low-risk cases, the system may apply a predefined tie-breaking rule or request a more precise source. The goal is accountable accuracy, not generating a final answer at any cost.
Evaluate prompts and reviewers with realistic failures
Build an evaluation set around difficult cases: contradictory clauses, malformed OCR, a misleading heading, missing context, a malicious instruction embedded in a document, a wrong citation, and a value that is numerically valid but belongs to the wrong record. Annotate expected outputs using the same schema as production. Where multiple legitimate interpretations exist, label the ambiguity rather than manufacture one ground-truth answer.
Separate metrics for extraction, citation support, abstention, policy correctness, and operational reliability. A system can have a high percentage of valid JSON while fabricating half its source citations. Another may be conservative and leave difficult fields unresolved, which can be preferable for financial or legal documents. Report false positives and false negatives for each important field instead of collapsing performance into a single headline score.
Review the results by document type, customer segment, source format, and version of the model or prompt. A change that improves English digital contracts may reduce accuracy on scanned multilingual material. Run regression samples after any change to examples, schema, retrieval, or model. Distinguish deployment quality from answer fluency; evaluation should answer what went wrong, why, and which component owns the fix.
Use cost and latency as design constraints, not goals
Few-shot examples consume context. Review passes add requests. Batches can reduce per-request operational cost, but asynchronous completion introduces latency and monitoring work. Select a design by matching processing mode to the business requirement. A small, bounded classifier might need one model call plus deterministic checks. A large-scale, high-stakes extraction pipeline may justify a second verifier and human review on flagged records.
Measure completed, validated tasks per dollar and per unit of elapsed time, not just average tokens per response. One extra review call may be economical if it prevents costly human remediation. Conversely, several repeated passes with identical instructions are expensive if they add no detection power. Track the reasons records are escalated, and use them to improve the rubric or examples rather than hiding uncertainty behind more generated text.
A reliable batch-processing pipeline separates its guarantees. Few-shot examples teach the model how to distinguish cases; schemas define the output shape; independent validators check claims and policy constraints; and reviewers resolve cases for which evidence remains insufficient. Batching is worthwhile only when these checks still apply to each item and a partial failure can be retried without duplicating accepted results. Report errors by type and severity, not only as an aggregate completion percentage.