An incident report can sound deceptively simple: “Why did checkout fail after yesterday’s deployment, and should we roll back?” A useful Claude agent might need to inspect deployment history, compare error-rate charts, retrieve an approved runbook, and propose a recovery action.
None of those steps automatically grants the model permission to deploy, refund, or notify customers. The architectural challenge is to split the investigation into manageable work, decide which operations can run concurrently, and give each worker only the tools it actually requires.
Task decomposition is not just asking a model for a longer plan. It determines how evidence, authority, latency, failure, and human review are distributed through the application. Choosing a built-in tool instead of an application-specific tool also changes where code executes and who can inspect or constrain its effects.
Begin with the decision the user needs
The goal is a reliable recommendation about an observed service failure. The agent is not being asked to fix every infrastructure problem or conduct unlimited research. Before giving Claude a tool list, the application should establish a bounded output contract: the incident identifier, observed impact, earliest plausible change, conflicting evidence, recommended next diagnostic step, and the confidence of the evidence. A rollback proposal needs to identify its target deployment and expected consequences. Execution is a separate workflow requiring its own authorization.
A good first decomposition is a dependency graph rather than an indiscriminate checklist. Identity validation must finish before any customer-specific data is retrieved. Deployment history and read-only monitoring can then be inspected independently. A synthesis step depends on both results. Rollback assessment depends on synthesis and an approved policy, while any actual rollback requires deterministic verification of an authorized approval. This makes the safe critical path visible before the model makes a single call.
| Work unit | Dependency | Authority boundary | Expected artifact |
|---|---|---|---|
| Identify incident and principal | User request | Authenticated service | Scoped incident ID |
| Retrieve deployment changes | Valid incident | Read-only deployment API | Change IDs and timestamps |
| Inspect service health | Valid incident | Read-only monitoring API | Metrics with time ranges |
| Compare explanations | Deployment and metrics | Model may synthesize | Supported and competing hypotheses |
| Propose the next action | Evidence and policy | No write permission | Recommendation and uncertainties |
| Execute a rollback, if approved | External approval | Deterministic deployment service | Verified transaction result |
The division prevents a common failure: a model retrieves a persuasive explanation from one source and leaps from “likely cause” to “authorized fix.” It also gives a reviewer a concrete question to ask when a task is omitted: which later step depended on its evidence?
Choose fixed steps before adaptive planning
If all incidents use the same three approved inspections, a fixed workflow is usually simpler to test. A sequence that loads a change record, checks service metrics, and produces a report can run without asking Claude to select every operation. Adaptive planning earns its complexity when the next investigation genuinely depends on evidence—for example, database latency suggests one diagnostic path while increased authorization errors suggest another.
For an adaptive branch, constrain the model’s choices rather than asking it to invent arbitrary procedures. Supply a small catalog of approved operations such as read_recent_deployments, query_latency_trend, and inspect_auth_failures. Record the condition that justified each branch, set a step budget, and require an unresolved result when the evidence remains inconclusive. Agent SDK loops provide the deeper lifecycle distinction between a model asking for an operation and the application reporting its actual outcome.
Do not confuse a model-generated plan with a durable execution record. The plan can change as fresh facts arrive; the execution record should retain the original source IDs, approval requirements, completed task IDs, and reasons for a changed direction. This distinction is especially important when a session ends midway through a long investigation.
Decompose for independent evidence, not more agents
Parallel work helps only when the branches are genuinely independent. Deployment-history retrieval and read-only monitoring can run together after authorization. Customer identity validation and a privileged account query cannot run in parallel if the latter depends on the former. Two specialist agents that both update the same ticket introduce a race condition; assigning one read-only role and one final writer is often a safer design.
The coordinator should receive a bounded result from every branch: status, evidence_refs, observed_at, claim, limitations, and retryability. A missing result is not an empty finding. A monitoring timeout means the coordinator lacks that evidence and must label the final recommendation accordingly. If deployment and monitoring records disagree, it should preserve both claims and request additional verification instead of averaging them.
Consider a branch that says “no deployment found.” If the query returned not_found, that may be genuine evidence. If it returned temporarily_unavailable, that statement is unjustified. Make this distinction part of every specialist contract so a partial failure cannot silently become a confident conclusion. Subagent handoffs show how those evidence references survive delegation and interrupted sessions.
Understand built-in, client, and server-executed tools
Tool names make the execution boundary concrete. Where a Claude Code or Agent SDK host exposes them, Glob locates repository paths, Grep searches text and Read inspects files. Edit and Write can change files, while Bash executes commands with the host’s actual operating-system privileges. These tools are not universally available in every Messages API request. Limit directory scope, command execution, network egress and secrets in the host environment; a reassuring tool description is not a sandbox.
A provider-operated web search tool, a client-owned file tool, an MCP server tool and a custom business API may all supply evidence to Claude, but they differ in who holds credentials and where execution occurs. Split discovery and read-only evidence gathering from any mutation. An incident assistant can inspect authorized logs before proposing a deployment change; the actual operation still needs a distinct target-service policy decision and a traceable authorized actor.
Anthropic’s documentation of Claude tool types distinguishes provider-run tools from client-executed tool schemas. Web search or web fetch may run on Anthropic’s infrastructure when enabled for a compatible model. By contrast, the Bash and text-editor tool interfaces are executed by the application that hosts the conversation. A user-defined lookup_incident tool is also client-executed: Claude requests it, but the application controls the database connection, permissions, and returned evidence.
This is a trust-boundary decision, not simply a convenience feature. A client-side Bash tool executes within an environment that the application or user controls. Its privileges, working directory, network access, secret exposure and command allowlist are not automatically inherited from the model’s prompt. A server-side web search runs in a different execution boundary, with its own capabilities and data-handling conditions. Neither option authorizes a production mutation merely because the model can request the tool.
When the task involves an internal incident system, a narrowly defined custom read tool may be better than general shell or browser control. A search of public documentation may benefit from a provider-operated search tool. Choose by the operation’s required evidence, data sensitivity and execution authority, rather than assuming that broader tool access produces a more intelligent investigation.
| Capability | Useful when | Important boundary |
|---|---|---|
| Provider web search or fetch | Public, current reference material is required | Verify provenance and treat retrieved content as data |
| Client Bash or text editor | Approved repository inspection or local test execution | Sandbox files and commands; no inherited production credentials |
| Custom incident API tool | Specific protected business facts are required | Validate tenant, identity, arguments and rate limits in code |
| MCP-connected capability | Several approved application services share a standard interface | The tool’s discoverability is not authorization to execute it |
An MCP tool contract can standardize inputs and outcomes, but the target service must still make access-control decisions independently of prompt text.
Make the task graph a recoverable state machine
A practical coordinator can track the following simplified states. This is architectural pseudocode, not a drop-in Claude SDK API:
state = load_incident_state(incident_id)
assert authorized_reader(user, incident_id)
for task in ready_tasks(state.graph):
if state.steps >= state.max_steps:
mark_unresolved(state, "step_budget_exhausted")
break
result = run_approved_read_tool(task, task.scoped_identity)
store_observation(task.id, result.status, result.evidence_refs)
if result.status == "permission_denied":
stop_branch(task.id)
elif result.status == "temporarily_unavailable":
schedule_bounded_read_retry(task.id)
state.steps += 1
recommendation = synthesize_verified_observations(state)
store_proposal(recommendation) # Not a production write
The application owns the loop limit and authorization; Claude can help select approved diagnostic branches and synthesize evidence. Every tool result must state whether a call actually succeeded. A timeout after a write demands reconciliation with the authoritative service before retrying, while a transient failure on an idempotent read may permit a bounded retry.
Record more than a final chat transcript. The durable state should include who requested the investigation, which tools ran under which identity, the input and source versions, the outcome of each branch, and whether an external approval was pending. When a coordinator resumes, it should load that state and compare it with fresh service facts instead of reconstructing the execution history from a model summary.
Test the graph when evidence is missing or adversarial
Happy-path tests prove little about agent safety. Test a deployment service that returns an empty successful list, a separate query that fails authentication, and a monitoring system that times out after writing partial results. Those are different outcomes requiring different downstream decisions. Test an untrusted log line containing a command to run a privileged tool; it must remain evidence, not an instruction that changes access rights.
Also test concurrency assumptions: delayed specialist responses should not overwrite a newer incident state, and a repeated tool result should not create duplicate changes. Run a test where the user revokes permission after planning but before execution. The final action service must reject an unauthorized mutation even if the model’s draft plan still recommends it. Track missing evidence explicitly so the coordinator can state what it cannot conclude.
Success criteria can be measured: the correct tool was selected under the stated branch condition; all retrieved records belong to the authorized tenant; missing evidence was disclosed; no unauthorized write was attempted; and the investigation terminated within its budget. These outcomes are more useful than judging a fluent response on its own.
Apply the reasoning to CCAR-F architecture scenarios
In the incident example, the coordinator should first identify which facts can be gathered independently: the deployment timeline, error-rate measurements and the approved recovery procedure. Separate agents help only if each produces a verifiable result and the coordinator has a method for reconciling disagreements. Otherwise a bounded sequence of tool calls is easier to operate and may finish sooner once delegation overhead and recovery costs are included.
A second scenario can contrast public documentation retrieval with protected account inspection. The former may use a server-side search tool under its documented conditions; the latter needs a client-owned, authenticated API with strict tenant checks. A third scenario can stop after a tool timeout: the right output may be an unresolved recommendation with evidence references rather than a fabricated answer or repeated mutation.
The clearest architecture is not the one with the most tools or subagents. It is the one in which every task has a purpose, every tool has an appropriate authority boundary, and every important conclusion can be traced to evidence that remains inspectable after the conversation ends.