A Microsoft Foundry application is not one box containing a model. Consider a customer service system that retrieves product entitlements, drafts an answer, and occasionally initiates a warranty request. A user-facing API authenticates the caller; an orchestration service invokes model and retrieval endpoints; a search index returns documents; a business API enforces warranty rules; and telemetry captures what happened. The architecture becomes reliable when each component has an explicit responsibility and can fail without accidentally granting another component more power.<\/p>\n
Microsoft AI-103<\/a> expects developers to choose Foundry services, design Azure infrastructure, configure model and agent deployments, and manage access, quotas, cost, and monitoring. A useful design artifact is a short decision record with a diagram, input classes, approved output operations, identities, data residency, deployment regions, and a recovery objective. It should explain whether a simple model call, a prompt agent, or a hosted agent is appropriate for the required behavior.<\/p>\n Do not begin by adding agents to every path. A deterministic lookup followed by a generated answer can be easier to test and less expensive than an autonomous workflow. A hosted agent makes sense when custom orchestration logic genuinely requires code and an independently scaled runtime. A prompt agent can reduce infrastructure effort when instructions, tools, and platform capabilities suffice. Microsoft’s Agent Service<\/a> documentation differentiates these approaches and their operational cost.<\/p>\n A Foundry resource provides the platform boundary, while projects organize application-facing models, connections, tools, and other assets. A model deployment determines how a selected model can be consumed. An application needs an endpoint, an authentication mechanism, a chosen deployment, appropriate permissions, and logic for parsing responses and errors. Do not confuse having access to the portal with being able to call the deployment from a service principal.<\/p>\n Start by recording the region, supported model deployments, quotas, and available service integrations. Models differ in input modalities, context limits, throughput, compliance conditions, tool support, and pricing. If a service processes employee documents, the requirement to keep data within approved boundaries may restrict what deployment type or associated retrieval service can be used. Region compatibility is an architecture input, not a late deployment surprise.<\/p>\n Use Python with a supported Foundry SDK<\/a> package and the project endpoint format documented for its version. The SDK and endpoint conventions evolve; pin tested dependencies and keep version-specific examples aligned with current quickstarts. Configure external services through managed connections where possible, and make deployment configuration explicit rather than hiding the project name and credentials in application code.<\/p>\n Model instructions are not a security boundary. A search result that says someone is permitted to see a private document does not grant that permission. Validate the end user’s identity at the application layer, propagate only the claims required by a downstream service, and ensure the search\/index authorization path filters content before any model receives it. Agent tools must perform their own authorization and input validation for each business operation.<\/p>\n A useful authorization test uses two accounts: one employee with general access and one supervisor allowed to view protected reimbursement exceptions. Ask both the same question. The system should not merely hide the supervisor’s citation from the employee’s final answer; the employee’s retrieval result must exclude that document. Inspect request-side identity, query filters, and source-level entitlements rather than trusting the model to redact material later.<\/p>\n Managed identities reduce the need for long-lived keys but do not automatically choose correct roles. Map each runtime principal to the few Azure actions it actually needs: read from an approved search index, invoke a model, emit trace data, or access a storage container. Prevent the model-serving identity from having broad write access to the document source. Private endpoints and networking restrictions can support the boundary, while authorization remains necessary even on private networks.<\/p>\n Start with anticipated concurrency, input size, model token demand, latency objectives, and the expected number of agent tool calls. A conversation with three searches and two model calls behaves differently from a single classification request under the same traffic level. Capture these assumptions in a workload model before choosing deployment capacity. Model rate limits, regional quota, and search tier constraints can produce bottlenecks in different components.<\/p>\n Deployment options should be compared along availability, throughput, isolation, geographic needs, scaling behavior, and unit cost. A traffic spike may exhaust token-per-minute capacity long before the API service runs out of CPU. A burst of embedding requests may delay ingestion while interactive search remains responsive. Protect the experience with admission controls, bounded queue lengths, retry limits, and clear degradation paths that do not bypass safety or private data rules.<\/p>\n For CI\/CD, keep configuration and infrastructure changes in reviewable source. A release pipeline should validate schemas, deploy or configure resources, run smoke tests with limited permissions, and measure a small set of quality regressions before promoting traffic. If an agent version changes its tool arguments, a previously deployed business endpoint can reject those calls; version the two sides together and provide a rollback plan.<\/p>\n Application availability, model latency, failed requests, rate-limit responses, token usage, search index health, ingestion freshness, and cost show whether the system is operating. They do not show whether it is giving justified advice. Behavioral evaluations must inspect source grounding, action choice, safety, and correct escalation. Both layers should share trace identifiers so a failed answer can be traced through search, model inference, and tool invocation.<\/p>\n For the warranty assistant, instrument every stage without indiscriminately logging customer account details. Capture the retrieval document identifiers and policy revisions, model deployment version, tool name, authorization decision, and broad error category. Redact sensitive request content when detailed payloads are unnecessary. Review who can access telemetry, how long it is retained, and whether diagnostics propagate data to unintended locations.<\/p>\n The practical evaluation and tracing<\/a> investigation separates service health from response and agent quality. A model that produces persuasive but unsupported claims is a behavioral defect; a correct answer produced after twenty seconds may be an operational failure. A release can fail either gate. Do not report a single blended quality score that conceals where remediation belongs.<\/p>\n A secure system treats prompts, retrieved documents, file contents, and tool responses as different trust levels. External text can contain instructions aimed at the assistant; the application must not promote those words into a system-level policy. Microsoft’s Prompt Shields<\/a> can help detect malicious prompts and document attacks, but server-side tool permissions and data-access controls remain required.<\/p>\n Consider a retrieved product bulletin that includes an instruction to email all client records to an external address. That document may still be relevant evidence for a warranty question, but it has no authority to change a tool call. The retriever should identify its origin, the agent should treat it as data, and the business API must reject unauthorized export operations even if the agent attempts one. This test demonstrates defense in depth rather than confidence in a single content filter.<\/p>\n State-changing workflows should use proposals and approvals. The model may draft a warranty claim, but the backend should validate product identity, eligibility, current user, duplicate requests, and manager approval where policy demands it. Persist approvals and idempotency keys in durable application state. If a call times out after the backend committed the transaction, the retry must return the same result rather than creating a second claim.<\/p>\n Estimate more than inference token prices. A grounded assistant may incur costs from embedding generation, indexing, vector and semantic retrieval, agent tool calls, evaluation runs, hosted compute, tracing storage, and external APIs. Compare cost per resolved legitimate case, not cost per model call. A cheaper response that escalates every question or requires manual correction may raise total service expense.<\/p>\n Watch ingestion lag and index quality alongside model operations. A healthy model can only answer from the evidence it receives. If product bulletins are indexed six hours late, a generated response can be wrong without any model deployment change. Store source version and ingestion timestamp, measure retrieval relevance on current and historical questions, and alert separately when the data pipeline falls behind.<\/p>\nUnderstand resources, projects, models, and endpoints<\/h2>\n
Identity and search permissions are separate controls<\/h2>\n
Choose a deployment and release strategy<\/h2>\n
Monitor the infrastructure and the model behavior<\/h2>\n
Keep guardrails independent of model self-discipline<\/h2>\n
Model cost and search quality are operational concerns<\/h2>\n