AAIA: Auditing AI Systems with Defensible Evidence

Auditing artificial intelligence requires the assurance discipline of traditional information systems audit plus a careful understanding of models, training data, operational decisions, and risks that change after deployment. ISACA's Advanced in AI Audit (AAIA) certification addresses this emerging assurance specialty. It is intended for qualified audit and advisory professionals, including CISA holders and holders of certain other designated credentials with appropriate IT audit or advisory focus. It is not an introductory certificate for someone who has never performed an audit; eligibility and examination rules should be confirmed on ISACA's current certification page.

The AAIA exam content outline groups work into AI Governance and Risk, AI Operations, and AI Auditing Tools and Techniques. An auditor must be able to challenge the adequacy of a governance claim using suitable evidence, evaluate how an AI system is developed and operated, and adapt audit methods when behavior is probabilistic or opaque. Merely observing that an organization has an AI policy does not prove it applies to the decisions, data, suppliers, and technical controls of a specific production system.

Define audit scope around the deployed AI system

An AI system is more than a model file. It includes data sources, preprocessing, features or prompts, retrieval services, external model providers, access controls, output handling, human review, monitoring, and interfaces that may take actions. The auditor should identify which parts are within organizational control and which depend on vendors. The system's stated purpose and population matter because the same model architecture can carry very different risks in internal document search and automated eligibility decisions.

Start with objectives and criteria. An audit might assess compliance with approved organizational policy, legal obligations, internal risk standards, data handling requirements, or specific model validation rules. Criteria must be appropriate to the system and sufficiently clear to test. A broad statement such as 'AI must be fair' requires translation into relevant populations, outcomes, metrics, controls, and accountable decisions. Where a standard allows interpretation, record the interpretation used rather than pretending the criteria are self-evident.

Risk-based scoping should prioritize consequential use cases, sensitive information, autonomy, deployment scale, third-party dependencies, and indicators of operational failure. A chatbot drafting low-impact internal messages may require a different audit depth from a decision engine controlling credit access. Risk does not disappear because a vendor marketed the system as responsible AI. Ask who approved the use case, who can suspend it, and how users report problematic outcomes.

Independence and evidence quality remain central. An auditor should not become the designer of the same control being independently assessed. Project documentation, management attestations, data samples, logs, model cards, evaluation results, contracts, and incident records have different reliability. Validate completeness and provenance before reaching conclusions. A beautifully formatted dashboard that shows only selected successful evaluations can be weaker evidence than a reproducible test set with documented limitations.

Evaluate AI governance, accountability, and risk ownership

A governance framework should define intake, risk classification, permitted use, roles, change approval, monitoring, and retirement. The auditor tests whether those mechanisms operate for actual AI assets. Select a sample of systems from independent inventory sources and determine whether every selected system has an owner, approved purpose, risk assessment, data-flow description, and deployment history. If experimental models are operating through production APIs but absent from the inventory, the completeness failure may be more important than an individual missing sign-off.

Accountability can break down across business, model development, security, privacy, and procurement. An AI committee may recommend safeguards while the product team alone changes model providers. Review decision records to see who had authority at each critical gate. When management accepts residual risk, verify that the decision maker had appropriate responsibility, received adequate evidence, and specified compensating measures and reassessment dates. A vague 'approved' status is not an audit conclusion.

Regulatory and standards mapping must be accurate. Laws vary by jurisdiction and use case; voluntary frameworks such as NIST's AI Risk Management Framework or ISO/IEC AI management guidance do not automatically satisfy legal requirements. Audit evidence should connect obligations to concrete controls and tests. An organization may claim that it follows a framework while omitting monitoring after deployment or failing to document how affected people can challenge harmful outcomes. Identify the gap between stated policy and actual operation.

Third-party providers deserve particular attention. Contracts may not disclose training-data provenance, model updates, incident reporting, or retention practices sufficiently. Assurance reports can help but may omit the customer's integration. Evaluate the end-to-end system, not just the provider's infrastructure. The neighboring discipline of AI governance programs supports risk ownership; AAIA concentrates on whether the claimed governance is evidenced and effective.

Examine data, model development, and testing controls

Data quality is a control issue because missing records, inconsistent labels, duplicated examples, and sampling bias can invalidate model evaluations. Identify the source, rights of use, preprocessing steps, versioning, validation controls, and permissions for datasets. Where personal information is involved, the audit should assess whether the organization's privacy controls apply to training, retrieval, fine-tuning, and inference. Documentation may need to distinguish a reference dataset from the data actually used by a production run.

Model validation should address the use case and possible harm. A classifier's overall accuracy can obscure poor performance for a rare but important class. Consider false-positive and false-negative impacts, calibration, confidence, robustness, and performance across relevant subpopulations. Generative models need tests for fabrication, prompt injection, private-data leakage, unsafe tool use, and sensitivity to context. The auditor is not expected to prove perfect model behavior, but should assess whether evaluation is suitable, repeatable, and proportionate to risk.

Reproducibility matters. Record model and prompt versions, data snapshots, evaluation code, test environment, thresholds, reviewers, and exceptions. A provider might update a hosted model without changing the client application's visible release number. The audit should investigate how material changes are detected and reapproved. Test evidence should not be considered current merely because it was produced at project launch when the deployed system has since changed substantially.

Operational monitoring links development controls to actual outcomes. Look for drift indicators, error trends, user complaints, abuse reports, human-override rates, security alerts, and significant environmental changes. Evaluate response triggers: who investigates, how decisions are recorded, and when service is restricted. An assurance conclusion should separate tested effectiveness from areas that are unobservable due to insufficient logging or vendor access.

Adapt audit sampling and evidence collection to AI behavior

Sampling an AI service differs from verifying a deterministic accounting formula. Repeated inputs may yield different outputs, and valid results may depend on context. The auditor should define the population of use cases and how test cases are selected: normal scenarios, edge cases, affected groups, adversarial inputs, and failures with high potential impact. Explain the limits of sample size and statistical confidence. A few persuasive demonstrations are not a substitute for a documented testing methodology.

Combine interviews and document review with direct technical evidence. Walk through one user transaction from data collection to model response and downstream action. Inspect authorization boundaries, system prompts where accessible, retrieval content controls, filtering, human intervention, and log records. If red-team exercises exist, review their coverage and remediation rather than accepting the words 'red team passed.' Where testing might expose personal information or disrupt production, establish safeguards and appropriate authorization first.

AI tools can also assist the audit process by organizing documents, identifying anomalies, or suggesting samples. That creates a second risk: the audit team may rely on incorrect or fabricated summaries. Preserve traceability back to source records, validate tool outputs, and protect confidential evidence. The independence and competence of an auditor do not transfer to an LLM. Automated analysis can improve breadth only when reviewers understand its error modes and retain responsibility for conclusions.

A strong audit finding specifies criteria, condition, cause, effect or risk, and a practical recommendation. 'Model unfair' is too vague. A better finding describes a tested outcome disparity for a defined population, a missing evaluation gate, relevant assumptions, and the risk to affected people. Recommendations should address the root control failure—perhaps absent training-data review or missing release approval—not merely request another policy document.

Conduct an AI vendor and deployment audit case

Suppose an insurer uses a third-party model to prioritize manual claim reviews. Management says that decisions remain human because adjusters approve final outcomes. Audit whether the interface permits meaningful disagreement, how often recommendations are overridden, what data feeds the model, and whether certain groups systematically receive higher scrutiny. Determine whether the prioritization affects claim processing time even when it does not make the final decision. A nominal human review step may not neutralize model-induced harm.

Expand the case to include a model update delivered by the provider without customer notice. What was the agreed change-control mechanism? Which validation artifacts can be reproduced? Do system logs identify the version used for earlier recommendations? Are affected results reassessed when a known defect is discovered? The audit can fail on traceability and vendor governance even if current aggregate prediction metrics look acceptable.

Next, test a suspected incident involving inappropriate access to claim documents through a retrieval component. Distinguish data authorization, prompt-injection defenses, logging, contractual notification, and actual exposure evidence. Assign finding owners across the organization and vendor without confusing responsibility. A credible audit report explains which controls were tested, what evidence supports the findings, and what remains uncertain.

Prepare for AAIA with defensible audit workpapers

ISACA's published outline identifies a 90-question exam covering AI Governance and Risk, AI Operations, and AI Auditing Tools and Techniques. Use the official material to understand the domain weights and to verify current candidate requirements. The relationship to CISA is important: AAIA builds on professional audit competency while adding AI-specific subject matter. It is not interchangeable with AAISM, which focuses on AI security management rather than independent assurance.

Develop an illustrative audit file containing an audit objective, scope, criteria, selected AI use cases, data requests, test plan, sampling rationale, results, findings, and management responses. Stress-test the evidence. Could another reviewer repeat the test? Are controls specific enough to evaluate? Are conclusions narrower than the facts? Practicing this discipline is more useful than memorizing a long list of AI risks. The goal is to explain whether an organization can justify trusting a deployed AI system—and what evidence is missing when it cannot.