Text analysis is more than a prompt that returns sentiment
A multilingual customer-service application may need to understand what a person said, classify the intent, find entities in the message, summarize a case, remove sensitive details, translate an answer, and support a spoken conversation. These are distinct operations with different quality requirements. A classification label can be useful even when an open-ended summary would introduce too much risk; a language model may help with flexible explanation while a dedicated language service provides consistent entity or privacy detection.
Microsoft AI-103 explicitly tests extracting entities, topics, summaries, and structured JSON; detecting sentiment, tone, safety issues, and sensitive content; translation with Azure Translator or language models; and domain customization. It also tests speech-to-text, text-to-speech, speech as an agent modality, custom speech, audio reasoning, and spoken translation. The purpose is to build systems where these outputs can be validated and connected to useful actions, not to memorize a list of API brands.
Use a call-center scenario. Customers ask to change delivery addresses, report damaged shipments, and request refunds. Their messages arrive as typed text, recorded voice, or live audio in two languages. The application must separate the user’s request from incidental statements, detect private identifiers, preserve important dates and amounts, and ask for human confirmation before a refund is processed.
Extract structured information without inventing fields
Define a schema for a support case: intent category, order identifier, requested change, dates, product code, short summary, language, escalation flag, and evidence spans. Set which fields are required and which may be null. A generated JSON object is not enough; validate data types, allowed categories, format constraints, and whether each factual value appears in the actual message.
Consider “My order A-441 arrived yesterday, but the replacement for my older order A-414 has not shipped.” A weak extractor may select the first order number simply because it appears first. A stronger result associates each event with the correct identifier and leaves unrelated fields unset. Test messages with corrected statements, negation, quoted speech, and conflicting numbers. Keep a trace to the original text span for disputed cases.
Compare a structured generative prompt with dedicated language tools when the task justifies it. Microsoft’s Azure Language capabilities include named entity recognition, personally identifiable information detection, sentiment and related text operations. Service versions have changed, so confirm current endpoints and support rather than copying an old Text Analytics request body into production.
Treat sentiment and safety as decision inputs
Sentiment is not the same as urgency or customer entitlement. A calm message may describe a severe product safety incident, while an angry customer may only need a normal status check. Create separate labels for emotional tone, actionable risk, policy category, and escalation condition. Use clear definitions with examples so reviewers understand what an evaluator is scoring.
Test messages where negative words are negated: “I was worried, but nothing is broken now.” A keyword detector can still label the message strongly negative. Also test sarcasm, domain slang, multilingual idioms, and sarcasm mixed with formal details. If the result is used for routing, measure false escalations as well as missed risk. Human review remains important when decisions affect people or finances.
Sensitive content detection requires a privacy plan. A customer may include a passport number, bank information, or account credentials in a support ticket. Decide what data needs to be extracted, masked, deleted, or quarantined. Avoid storing raw sensitive values in model prompts or telemetry when the workflow does not require them. The June 2026 Language updates discuss text PII features and version-specific behavior; production code should pin and test the chosen API.
Translate meaning, not just words
Translation quality is especially sensitive to negation, obligations, legal terms, dates, currencies, and technical product names. A customer may say “do not cancel order A-441” and a translated summary must preserve the prohibition. A polished translation that reverses a request is worse than a slightly awkward one that is accurate. Maintain a source-text reference alongside the translated version for human reviewers.
Azure Translator and LLM-powered translation flows present different trade-offs. A specialized service may provide consistent language support and terminology handling, while a generative model can handle context-rich rewriting and summaries. Select a method using an evaluation set that includes product codes, honorifics, low-resource language pairs, domain jargon, and multilingual messages where the customer switches languages mid-sentence.
Separate translation from action authorization. Even if a translated message appears to request a refund, the backend must authenticate the customer and verify the order. If translation confidence is low on an amount or recipient, ask for confirmation rather than inferring intent. Keep the translation’s model and service version in the trace so an error can be reproduced.
Add speech recognition without losing boundaries
Speech-to-text can process live audio or recordings, but its output is an imperfect representation of what a person said. Recordings may have background noise, channel overlap, accents, domain vocabulary, interrupted speech, and words that sound like model or product identifiers. Evaluate recognition by comparing against labeled transcripts and by checking task-critical terms separately from overall word error rate.
A custom speech model can help when recurring domain terminology is consistently misrecognized. Microsoft’s custom speech documentation describes adapting recognition to domain data. Start with the standard recognizer and an error analysis before investing in customization. Document training-data permissions, supported languages, model stability, and a test set held out from customization.
For a live voice assistant, design turn-taking and interruption handling, timeout behavior, and explicit user confirmation for consequential actions. A transcription error that changes “send replacement details” to “send a replacement” must not initiate a purchase. Persist the transaction request separately from the speech transcript and require the appropriate approval step. Audio is untrusted user input; a spoken instruction cannot bypass service permissions.
Text-to-speech and spoken translation need their own QA
Text-to-speech should pronounce dates, currency values, addresses, acronyms, and account identifiers in ways the listener can understand. An accessible voice experience may need to break long answers into shorter segments and offer a text alternative. Evaluate intelligibility and comprehension, not merely audio naturalness. For sensitive messages, avoid reading an identifier aloud in a public setting unless the user authorized that presentation.
Spoken translation combines recognition, language conversion, and often speech synthesis. Errors can accumulate across stages. Use paired test cases with the source audio, original transcript, translated text, and synthesized response. A failure may originate from speech recognition rather than translation. Track latency and the possibility that streaming output is sent before the complete intent is known.
The speech translation documentation provides product-specific capabilities and setup guidance. Support varies by language, region, and service feature; verify the chosen configuration. Do not promise universal automatic language identification, voice preservation, or custom voice support without checking prerequisites and permissions.
Integrate language and speech into agents carefully
An agent can use a transcription or language-analysis result to decide which approved information to retrieve. It should not treat the analyzer’s free-form summary as an authoritative business state. Pass structured fields with source references and confidence or review indicators; make uncertain fields explicit. When an order identifier is ambiguous, fetch candidates through a read-only lookup before proposing any action.
A useful agent workflow receives a Spanish voicemail about a damaged shipment, transcribes it, detects the language, extracts the order identifier, retrieves the order’s authorized status, and drafts a response in Spanish. The customer then confirms whether a claim should be opened. The agent tool boundary makes sure that uncertainty in speech recognition cannot silently become an approved business operation.
Evaluate the full pipeline at each stage. Store de-identified test audio, expected transcript, extraction labels, retrieved order facts, translated draft, and action decision. Include cases with two order numbers, noise, an incorrect date, and a spoken instruction that tries to override the agent. A strong system can explain uncertainty and route the case for review without losing the legitimate request.
Run a multilingual support lab
Define the workflow’s time boundaries before running the tests. A batch transcript submitted for overnight processing is not interchangeable with a real-time conversation in which the customer interrupts, changes language, or corrects an order number.
For live input, mark an interim transcript as provisional until the final recognition result is available; do not let partial hypotheses trigger business actions. For recorded calls, document retention permissions, speaker-consent requirements when applicable, and whether the agent will be allowed to store a summary after raw audio expires.
Give test cases with repeated numbers, competing speakers, and a speaker saying “not the order ending in forty-one.” Compare the final extracted intent and fields against the actual utterance, not just the transcript’s grammatical fluency. A mistaken order identifier must result in confirmation or escalation before any state-changing workflow continues. This distinction between provisional speech evidence and trusted transaction data is central to safe voice-assisted support.
Prepare 20 support messages with known intent, entities, sensitive data, translation requirements, and escalation outcomes. Include typed and spoken inputs in supported languages. Run text extraction, sentiment or safety detection, and translation; compare field accuracy and key-fact preservation. Review a sample manually instead of treating automatic scores as infallible.
Add five audio clips with intentionally difficult product codes or background speech. Evaluate transcripts and downstream decisions separately. Test a text-to-speech response containing a currency amount, date, and case number. Confirm that the spoken result is understandable and that private identifiers are not exposed unnecessarily. Simulate a missing transcript or denied source lookup to verify safe failure behavior.
Evaluate transcription, translation, entity extraction and downstream action separately. A recognizer that mishears one digit in an order code may have a low overall word error rate but still create a costly ticket on the wrong account.
The workflow should carry confidence and source evidence forward, ask for confirmation when an identifier is uncertain, and block submission until the customer and order are verified. Record language support, recognition errors, latency and sensitive-data handling as independent acceptance criteria rather than relying on a convincing spoken reply.