A successful request is not necessarily a correct answer
An AI support application can return HTTP 200 while telling a customer to follow an expired policy. It can produce fluent text while calling a tool that should have required approval. Traditional availability monitoring would label both interactions healthy. Microsoft AI-103 expects developers to evaluate generative outputs and agents, monitor drift and grounding quality, instrument responsible AI, and manage runtime cost, latency, safety, and recovery. These responsibilities require a distinction between service health and behavior quality.
Use a practical application that searches approved service policies and proposes support tickets. Its service-level indicators include successful inference requests, search availability, p95 response time, tool error rate, token consumption, and index freshness. Its behavioral indicators include accurate source citations, grounded claims, correct refusal, appropriate tool choice, authorization compliance, and successful task completion. Put them side by side but do not combine them into one number that hides critical safety failures.
Microsoft’s Foundry tracing documentation describes server-side traces and client instrumentation, including OpenTelemetry integration. Treat tracing as a means of understanding operations, not evidence that a tool call was justified. A trace shows what the system attempted; a separate evaluation or policy check determines whether the attempt should have been allowed.
Create a small, deliberately difficult evaluation set
Collect representative questions before optimizing a prompt. Include ordinary supported requests, a request with no authoritative answer, two conflicting policy revisions, a person asking for another user’s data, an ambiguous product identifier, and a malformed scanned document. Include adversarial text embedded in a retrieved page and a tool response that falsely claims to change the application’s instructions. Cover multilingual or multimodal inputs when the application handles them.
Label expected outcomes explicitly. For a warranty question, the expected result could be a cited public explanation and no ticket action. For a protected exception, it could be denial of retrieval and a safe redirect to a human review process. For an uncertain invoice total, it could be an ambiguity flag and a request for clarification. A test set whose only labels are “good” and “bad” cannot explain which engineering layer failed.
Separate the benchmark into retrieval, generation, tool behavior, and end-to-end task completion. Retrieval tests ask whether the right authorized evidence appeared. Generation tests ask whether every factual claim is supported by it. Tool tests check schemas, approval, idempotency, and permissions. End-to-end tests ask whether the user achieved the legitimate goal without an unauthorized side effect. This separation prevents a prompt rewrite from being blamed for an index permission bug.
Evaluate groundedness, relevance, and action choice
Groundedness measures whether a response is supported by its supplied evidence. Answer relevance measures whether the content addresses the user’s question. Both matter: a response can be fully grounded but irrelevant, or relevant-sounding but fabricated. Citation correctness adds another dimension because a model may cite a plausible document that does not contain the quoted fact. Evaluate evidence spans and effective dates rather than checking whether any link appears.
For agents, measure whether the selected tool was necessary, appropriately scoped, and authorized. A policy question should not trigger a write tool. A ticket request may require user identity, validated fields, and manager approval. Wrong-tool execution can be an unacceptable defect even when the final text looks fine. Add typed expected tool traces and compare them with actual calls at a decision level.
AI-assisted evaluators can help screen a larger dataset, but they also have limitations and sometimes disagree with human judgments. Calibrate them on manually inspected examples, explain criteria, and route high-risk or uncertain cases to reviewers. Microsoft’s observability guidance connects evaluations to development and production monitoring; do not confuse an automated evaluator’s score with audited business correctness.
Reflection and self-critique loops can improve a draft when they force a model to re-check claims against independent sources or a specified acceptance contract. They can also reinforce an unsupported claim if the same model merely declares its own answer correct. Set a bounded retry budget, require external evidence for any factual correction, and compare citation accuracy and task success with an ordinary single-pass baseline.
For examination discussions of chain-of-thought evaluation, assess observable decisions, tool traces, and verified outputs rather than demanding or publishing a model’s private internal reasoning as an audit record. Store concise evidence-backed correction notes, not purported hidden thought transcripts. A reflection loop is useful only if it demonstrably reduces errors without causing excessive latency, cost, or unsafe tool calls.
Trace retrieval, model steps, and tool execution
Capture a correlation ID from user request through policy retrieval, model inference, specialist handoff, tool proposal, authorization, and response. Attach model and prompt version, search index revision, selected document identifiers, safe tool arguments, execution duration, and outcome. When the answer is wrong, the trace should help determine whether evidence was stale, permissions were missing, the model misinterpreted evidence, or a business API returned an incorrect status.
Avoid logging secrets or full customer records merely to make debugging convenient. Decide which span attributes are safe, what must be masked, who can see traces, and how long data is retained. A generated summary may contain information from sensitive documents even when the user’s own input was harmless. Logging is a data-processing operation that needs the same access and retention scrutiny as the application itself.
Use parent and child spans to understand multi-agent behavior. A coordinator may dispatch three specialists concurrently; one may wait on search, one may repeatedly retry a failing API, and the last may return quickly. Aggregate request latency alone does not identify the bottleneck. The multi-agent workflow keeps task IDs and partial-result states so trace data can be correlated with durable application records.
Test prompt injection and tool boundaries
Retrieved content and user uploads are not trusted instructions. An attacker can insert a sentence into a searchable policy file instructing the assistant to reveal secrets or bypass approval. The right test has two parts: the model should resist promoting the sentence to a system instruction, and the tool service should independently reject any unauthorized operation even if the model attempts one. Preventing the final answer from mentioning the attack does not prove the second boundary is secure.
Microsoft Prompt Shields can detect adversarial inputs, but it is a layer in a defense-in-depth system. Validate the identity on each tool request, maintain allowlisted destinations, enforce schema limits, and keep business approval state outside model text. Test audio transcripts and extracted document content as possible injection carriers, not only ordinary chat messages.
Include refusal correctness in scoring. A system that refuses every question might prevent some dangerous actions yet fail its intended service. Measure false refusals on authorized questions and false allowances on restricted ones. Require human review for high-impact uncertain decisions, and define which outcomes are safe to automate. Production readiness is a decision across usefulness and risk, not a contest to minimize one metric.
Monitor production drift and resource economics
A model deployment can remain unchanged while retrieval quality degrades because source documents, embeddings, or index configurations changed. Monitor ingestion failures, lag relative to source updates, access metadata validity, changes in query terms, relevance patterns, and the proportion of no-evidence answers. Compare live samples with a dated benchmark while respecting data consent and confidentiality.
Monitor token demand, per-request latency, retries, rate-limit incidents, hosted-agent compute where relevant, and downstream search costs. Estimate cost per successfully resolved user task instead of cost per generation. Repeated tool calls and unnecessary agent branches can quietly dominate spending. Segment performance by model, deployment, request type, source index, and release version so an expensive change is visible before it spreads.
Use alerts that identify actionable system states. An increase in p95 latency may call for capacity or tool-timeout investigation; a rise in inaccurate citations requires retrieval or generation review; unauthorized-tool attempts require security attention. Sending every alert to one generic queue dilutes accountability. Define owners and thresholds, then test one alert with a controlled failure.
Release, roll back, and audit decisions
Tie each release to a reproducible manifest: model deployment, prompts, tool schema, index generation, connection configuration, evaluation dataset revision, and infrastructure changes. Run preproduction tests under permissions comparable to real users. A staging service with full administrator access might produce excellent answers that would be impossible or unsafe for ordinary users.
Canary a risky update on an allowed subset of traffic, compare key behavior and operational metrics, and roll back when a predefined guardrail is breached. Reverting a prompt is insufficient if a tool schema or search index changed incompatibly. Include compatibility and recovery procedures for all connected resources. The Foundry deployment design connects these release controls to quota and identity planning.
Preserve the rationale for human review and approvals. If a case is escalated because it involves policy uncertainty, show the source disagreement and the approval required. An audit that contains only a final decision without input lineage is hard to investigate. Avoid reporting improvements in real-world performance merely because a local evaluation suite passed; live outcomes require separate measurement.
An end-to-end negative-test exercise
Start with a correct policy answer and capture its trace. Then replace one source with an expired revision, remove access to a supervisor-only document, make the search tool return an error, and cause a timeout after a ticket was committed. For every case, record the expected final message, retrieval evidence, tool calls, authorization result, latency, and whether business state changed. Use the same test set after a prompt or deployment update.
Add one malicious policy paragraph telling the assistant to skip authorization. The service should treat it as data; the tool layer must deny a forbidden write; and the evaluation must show the precise point of enforcement. Next, create an ordinary legitimate request and verify that the system still resolves it without over-refusal. These two tests together reveal whether safety controls are effective and proportionate.
Release readiness requires more than a high evaluation score. Review whether citations identify current authorized sources, whether alerts expose actual model or tool failures, and whether a failed write can be retried without changing business state twice. A trace should allow a reviewer to locate the source, tool result, policy decision and deployment version behind a disputed answer. If any of those links in the evidence chain is missing, postpone the rollout or limit it to a bounded, monitored audience.