Amazon AWS Certified AI Practitioner AIF-C01 Practice Test 76

Question 750

A public-service assistant responds fluently but sometimes leaves citizen requests unfinished. Which description correctly defines a task-completion measure, rather than measuring wording or retrieval relevance?

  1. Groundedness checks correspondence between the answer and available evidence.
  2. Completion rate tracks successful end-to-end task outcomes.
  3. BERTScore compares contextual embeddings rather than only exact token overlap.
  4. Retrieval evaluation measures whether returned context is relevant to the query.

Explanation

Correct Answer

B:

For the proposed human-resources support use, the key operating principle is that completion rate tracks successful end-to-end task outcomes; that mechanism is what enables Task completion rate to measure whether real users achieve their goals with an assistant. The technical limitation—user satisfaction and response fluency are not identical to verified task success—needs a separate check.

Incorrect Answers

A:

In this human-resources support question about Task completion rate, the statement instead characterizes Groundedness testing: groundedness checks correspondence between the answer and available evidence. That can support a goal to verify that generated claims are supported by the retrieved sources, but does not explain the operation required for Task completion rate to measure whether real users achieve their goals with an assistant.

C:

In this human-resources support question about Task completion rate, the statement instead characterizes BERTScore: BERTScore compares contextual embeddings rather than only exact token overlap. That can support a goal to assess semantic similarity between generated responses and references, but does not explain the operation required for Task completion rate to measure whether real users achieve their goals with an assistant.

D:

In this human-resources support question about Task completion rate, the statement instead characterizes Retrieval relevance testing: retrieval evaluation measures whether returned context is relevant to the query. That can support a goal to check whether a RAG system retrieves passages that actually answer user questions, but does not explain the operation required for Task completion rate to measure whether real users achieve their goals with an assistant.

Question 751

Which explanation correctly describes Amazon Bedrock Model Evaluation when a media publishing project needs to compare supported foundation models on relevant quality and safety criteria?

  1. Trained reviewers assess dimensions that automatic text similarity metrics miss.
  2. Bedrock evaluation capabilities help structure model assessment and comparisons.
  3. BERTScore compares contextual embeddings rather than only exact token overlap.
  4. Task-level evaluation checks tool sequence, outcome and error handling.

Explanation

Correct Answer

B:

For the proposed media publishing use, the key operating principle is that Bedrock evaluation capabilities help structure model assessment and comparisons; that mechanism is what enables Amazon Bedrock Model Evaluation to compare supported foundation models on relevant quality and safety criteria. The technical limitation—one benchmark cannot determine fitness for every organization's tasks—needs a separate check.

Incorrect Answers

A:

In this media publishing question about Amazon Bedrock Model Evaluation, the statement instead characterizes Human evaluation: trained reviewers assess dimensions that automatic text similarity metrics miss. That can support a goal to judge whether generated advice is useful and safe in ambiguous business situations, but does not explain the operation required for Amazon Bedrock Model Evaluation to compare supported foundation models on relevant quality and safety criteria.

C:

In this media publishing question about Amazon Bedrock Model Evaluation, the statement instead characterizes BERTScore: BERTScore compares contextual embeddings rather than only exact token overlap. That can support a goal to assess semantic similarity between generated responses and references, but does not explain the operation required for Amazon Bedrock Model Evaluation to compare supported foundation models on relevant quality and safety criteria.

D:

In this media publishing question about Amazon Bedrock Model Evaluation, the statement instead characterizes Agent task success: task-level evaluation checks tool sequence, outcome and error handling. That can support a goal to evaluate whether an agent completes authorized workflows correctly, but does not explain the operation required for Amazon Bedrock Model Evaluation to compare supported foundation models on relevant quality and safety criteria.

Question 752

A manufacturing quality design uses ROUGE scoring in order to compare generated summaries against reference summaries using overlap-oriented measures. Which technical description is accurate?

  1. BERTScore compares contextual embeddings rather than only exact token overlap.
  2. Regression suites repeat representative risky and benign cases after changes.
  3. ROUGE emphasizes shared lexical units and recall-like overlap with reference text.
  4. Bedrock evaluation capabilities help structure model assessment and comparisons.
  5. A judge model scores outputs against defined quality criteria.

Explanation

Correct Answer

C:

For the proposed manufacturing quality use, the key operating principle is that ROUGE emphasizes shared lexical units and recall-like overlap with reference text; that mechanism is what enables ROUGE scoring to compare generated summaries against reference summaries using overlap-oriented measures. The technical limitation—high overlap does not guarantee factual correctness or semantic faithfulness—needs a separate check.

Incorrect Answers

A:

In this manufacturing quality question about ROUGE scoring, the statement instead characterizes BERTScore: BERTScore compares contextual embeddings rather than only exact token overlap. That can support a goal to assess semantic similarity between generated responses and references, but does not explain the operation required for ROUGE scoring to compare generated summaries against reference summaries using overlap-oriented measures.

B:

In this manufacturing quality question about ROUGE scoring, the statement instead characterizes Safety regression tests: regression suites repeat representative risky and benign cases after changes. That can support a goal to detect whether an updated prompt or model increases harmful or leaked outputs, but does not explain the operation required for ROUGE scoring to compare generated summaries against reference summaries using overlap-oriented measures.

D:

In this manufacturing quality question about ROUGE scoring, the statement instead characterizes Amazon Bedrock Model Evaluation: Bedrock evaluation capabilities help structure model assessment and comparisons. That can support a goal to compare supported foundation models on relevant quality and safety criteria, but does not explain the operation required for ROUGE scoring to compare generated summaries against reference summaries using overlap-oriented measures.

E:

In this manufacturing quality question about ROUGE scoring, the statement instead characterizes LLM-as-a-judge: a judge model scores outputs against defined quality criteria. That can support a goal to scale rubric-based evaluation of many candidate responses, but does not explain the operation required for ROUGE scoring to compare generated summaries against reference summaries using overlap-oriented measures.

Question 753

In a telecommunications review, BLEU scoring was proposed to evaluate machine translation by comparing generated text to reference translations. Which account of its underlying behavior is correct?

  1. BLEU uses n-gram precision with a brevity penalty across candidate translations.
  2. BERTScore compares contextual embeddings rather than only exact token overlap.
  3. ROUGE emphasizes shared lexical units and recall-like overlap with reference text.
  4. Retrieval evaluation measures whether returned context is relevant to the query.

Explanation

Correct Answer

A:

For the proposed telecommunications use, the key operating principle is that BLEU uses n-gram precision with a brevity penalty across candidate translations; that mechanism is what enables BLEU scoring to evaluate machine translation by comparing generated text to reference translations. The technical limitation—valid paraphrases can score poorly despite being acceptable to a human—needs a separate check.

Incorrect Answers

B:

In this telecommunications question about BLEU scoring, the statement instead characterizes BERTScore: BERTScore compares contextual embeddings rather than only exact token overlap. That can support a goal to assess semantic similarity between generated responses and references, but does not explain the operation required for BLEU scoring to evaluate machine translation by comparing generated text to reference translations.

C:

In this telecommunications question about BLEU scoring, the statement instead characterizes ROUGE scoring: ROUGE emphasizes shared lexical units and recall-like overlap with reference text. That can support a goal to compare generated summaries against reference summaries using overlap-oriented measures, but does not explain the operation required for BLEU scoring to evaluate machine translation by comparing generated text to reference translations.

D:

In this telecommunications question about BLEU scoring, the statement instead characterizes Retrieval relevance testing: retrieval evaluation measures whether returned context is relevant to the query. That can support a goal to check whether a RAG system retrieves passages that actually answer user questions, but does not explain the operation required for BLEU scoring to evaluate machine translation by comparing generated text to reference translations.

Question 754

A subscription services prototype calls for BERTScore to assess semantic similarity between generated responses and references. Which description would accurately document the method?

  1. BERTScore compares contextual embeddings rather than only exact token overlap.
  2. Retrieval evaluation measures whether returned context is relevant to the query.
  3. BLEU uses n-gram precision with a brevity penalty across candidate translations.
  4. Bedrock evaluation capabilities help structure model assessment and comparisons.

Explanation

Correct Answer

A:

For the proposed subscription services use, the key operating principle is that BERTScore compares contextual embeddings rather than only exact token overlap; that mechanism is what enables BERTScore to assess semantic similarity between generated responses and references. The technical limitation—semantic similarity alone is not sufficient to assess instruction following or safety—needs a separate check.

Incorrect Answers

B:

In this subscription services question about BERTScore, the statement instead characterizes Retrieval relevance testing: retrieval evaluation measures whether returned context is relevant to the query. That can support a goal to check whether a RAG system retrieves passages that actually answer user questions, but does not explain the operation required for BERTScore to assess semantic similarity between generated responses and references.

C:

In this subscription services question about BERTScore, the statement instead characterizes BLEU scoring: BLEU uses n-gram precision with a brevity penalty across candidate translations. That can support a goal to evaluate machine translation by comparing generated text to reference translations, but does not explain the operation required for BERTScore to assess semantic similarity between generated responses and references.

D:

In this subscription services question about BERTScore, the statement instead characterizes Amazon Bedrock Model Evaluation: Bedrock evaluation capabilities help structure model assessment and comparisons. That can support a goal to compare supported foundation models on relevant quality and safety criteria, but does not explain the operation required for BERTScore to assess semantic similarity between generated responses and references.

Question 755

To evaluate whether an agent completes authorized workflows correctly, a subscription services project considers Agent task success. Which statement correctly describes its role?

  1. BERTScore compares contextual embeddings rather than only exact token overlap.
  2. Trained reviewers assess dimensions that automatic text similarity metrics miss.
  3. Task-level evaluation checks tool sequence, outcome and error handling.
  4. Groundedness checks correspondence between the answer and available evidence.

Explanation

Correct Answer

C:

For the proposed subscription services use, the key operating principle is that task-level evaluation checks tool sequence, outcome and error handling; that mechanism is what enables Agent task success to evaluate whether an agent completes authorized workflows correctly. The technical limitation—good natural-language explanations do not prove an action was actually completed—needs a separate check.

Incorrect Answers

A:

In this subscription services question about Agent task success, the statement instead characterizes BERTScore: BERTScore compares contextual embeddings rather than only exact token overlap. That can support a goal to assess semantic similarity between generated responses and references, but does not explain the operation required for Agent task success to evaluate whether an agent completes authorized workflows correctly.

B:

In this subscription services question about Agent task success, the statement instead characterizes Human evaluation: trained reviewers assess dimensions that automatic text similarity metrics miss. That can support a goal to judge whether generated advice is useful and safe in ambiguous business situations, but does not explain the operation required for Agent task success to evaluate whether an agent completes authorized workflows correctly.

D:

In this subscription services question about Agent task success, the statement instead characterizes Groundedness testing: groundedness checks correspondence between the answer and available evidence. That can support a goal to verify that generated claims are supported by the retrieved sources, but does not explain the operation required for Agent task success to evaluate whether an agent completes authorized workflows correctly.

Question 756

The telecommunications team is reviewing how LLM-as-a-judge works in a plan to scale rubric-based evaluation of many candidate responses. Which statement gives the correct mechanism?

  1. Unit economics include token usage, retrieval cost and necessary human follow-up.
  2. A judge model scores outputs against defined quality criteria.
  3. Bedrock evaluation capabilities help structure model assessment and comparisons.
  4. Groundedness checks correspondence between the answer and available evidence.

Explanation

Correct Answer

B:

For the proposed telecommunications use, the key operating principle is that a judge model scores outputs against defined quality criteria; that mechanism is what enables LLM-as-a-judge to scale rubric-based evaluation of many candidate responses. The technical limitation—judges can be biased or inconsistent and benefit from calibration with human reviewers—needs a separate check.

Incorrect Answers

A:

In this telecommunications question about LLM-as-a-judge, the statement instead characterizes Production cost per interaction: unit economics include token usage, retrieval cost and necessary human follow-up. That can support a goal to compare model operating cost against each successfully handled request, but does not explain the operation required for LLM-as-a-judge to scale rubric-based evaluation of many candidate responses.

C:

In this telecommunications question about LLM-as-a-judge, the statement instead characterizes Amazon Bedrock Model Evaluation: Bedrock evaluation capabilities help structure model assessment and comparisons. That can support a goal to compare supported foundation models on relevant quality and safety criteria, but does not explain the operation required for LLM-as-a-judge to scale rubric-based evaluation of many candidate responses.

D:

In this telecommunications question about LLM-as-a-judge, the statement instead characterizes Groundedness testing: groundedness checks correspondence between the answer and available evidence. That can support a goal to verify that generated claims are supported by the retrieved sources, but does not explain the operation required for LLM-as-a-judge to scale rubric-based evaluation of many candidate responses.

Question 757

Which explanation correctly describes Retrieval relevance testing when a telecommunications project needs to check whether a RAG system retrieves passages that actually answer user questions?

  1. A judge model scores outputs against defined quality criteria.
  2. Regression suites repeat representative risky and benign cases after changes.
  3. Retrieval evaluation measures whether returned context is relevant to the query.
  4. BLEU uses n-gram precision with a brevity penalty across candidate translations.

Explanation

Correct Answer

C:

For the proposed telecommunications use, the key operating principle is that retrieval evaluation measures whether returned context is relevant to the query; that mechanism is what enables Retrieval relevance testing to check whether a RAG system retrieves passages that actually answer user questions. The technical limitation—a fluent final answer can hide retrieval misses and should be tested separately—needs a separate check.

Incorrect Answers

A:

In this telecommunications question about Retrieval relevance testing, the statement instead characterizes LLM-as-a-judge: a judge model scores outputs against defined quality criteria. That can support a goal to scale rubric-based evaluation of many candidate responses, but does not explain the operation required for Retrieval relevance testing to check whether a RAG system retrieves passages that actually answer user questions.

B:

In this telecommunications question about Retrieval relevance testing, the statement instead characterizes Safety regression tests: regression suites repeat representative risky and benign cases after changes. That can support a goal to detect whether an updated prompt or model increases harmful or leaked outputs, but does not explain the operation required for Retrieval relevance testing to check whether a RAG system retrieves passages that actually answer user questions.

D:

In this telecommunications question about Retrieval relevance testing, the statement instead characterizes BLEU scoring: BLEU uses n-gram precision with a brevity penalty across candidate translations. That can support a goal to evaluate machine translation by comparing generated text to reference translations, but does not explain the operation required for Retrieval relevance testing to check whether a RAG system retrieves passages that actually answer user questions.

Question 758

A human-resources support design uses Safety regression tests in order to detect whether an updated prompt or model increases harmful or leaked outputs. Which technical description is accurate?

  1. BERTScore compares contextual embeddings rather than only exact token overlap.
  2. Bedrock evaluation capabilities help structure model assessment and comparisons.
  3. A judge model scores outputs against defined quality criteria.
  4. Regression suites repeat representative risky and benign cases after changes.

Explanation

Correct Answer

D:

For the proposed human-resources support use, the key operating principle is that regression suites repeat representative risky and benign cases after changes; that mechanism is what enables Safety regression tests to detect whether an updated prompt or model increases harmful or leaked outputs. The technical limitation—an accuracy-only benchmark does not measure safety or privacy failures—needs a separate check.

Incorrect Answers

A:

In this human-resources support question about Safety regression tests, the statement instead characterizes BERTScore: BERTScore compares contextual embeddings rather than only exact token overlap. That can support a goal to assess semantic similarity between generated responses and references, but does not explain the operation required for Safety regression tests to detect whether an updated prompt or model increases harmful or leaked outputs.

B:

In this human-resources support question about Safety regression tests, the statement instead characterizes Amazon Bedrock Model Evaluation: Bedrock evaluation capabilities help structure model assessment and comparisons. That can support a goal to compare supported foundation models on relevant quality and safety criteria, but does not explain the operation required for Safety regression tests to detect whether an updated prompt or model increases harmful or leaked outputs.

C:

In this human-resources support question about Safety regression tests, the statement instead characterizes LLM-as-a-judge: a judge model scores outputs against defined quality criteria. That can support a goal to scale rubric-based evaluation of many candidate responses, but does not explain the operation required for Safety regression tests to detect whether an updated prompt or model increases harmful or leaked outputs.

Question 759

In an insurance operations review, Groundedness testing was proposed to verify that generated claims are supported by the retrieved sources. Which account of its underlying behavior is correct?

  1. Regression suites repeat representative risky and benign cases after changes.
  2. A judge model scores outputs against defined quality criteria.
  3. Groundedness checks correspondence between the answer and available evidence.
  4. Task-level evaluation checks tool sequence, outcome and error handling.

Explanation

Correct Answer

C:

For the proposed insurance operations use, the key operating principle is that groundedness checks correspondence between the answer and available evidence; that mechanism is what enables Groundedness testing to verify that generated claims are supported by the retrieved sources. The technical limitation—a source may itself be outdated or wrong, so grounded does not always mean true—needs a separate check.

Incorrect Answers

A:

In this insurance operations question about Groundedness testing, the statement instead characterizes Safety regression tests: regression suites repeat representative risky and benign cases after changes. That can support a goal to detect whether an updated prompt or model increases harmful or leaked outputs, but does not explain the operation required for Groundedness testing to verify that generated claims are supported by the retrieved sources.

B:

In this insurance operations question about Groundedness testing, the statement instead characterizes LLM-as-a-judge: a judge model scores outputs against defined quality criteria. That can support a goal to scale rubric-based evaluation of many candidate responses, but does not explain the operation required for Groundedness testing to verify that generated claims are supported by the retrieved sources.

D:

In this insurance operations question about Groundedness testing, the statement instead characterizes Agent task success: task-level evaluation checks tool sequence, outcome and error handling. That can support a goal to evaluate whether an agent completes authorized workflows correctly, but does not explain the operation required for Groundedness testing to verify that generated claims are supported by the retrieved sources.

Question 760

A professional-services prototype calls for Production cost per interaction to compare model operating cost against each successfully handled request. Which description would accurately document the method?

  1. Bedrock evaluation capabilities help structure model assessment and comparisons.
  2. A judge model scores outputs against defined quality criteria.
  3. BLEU uses n-gram precision with a brevity penalty across candidate translations.
  4. Unit economics include token usage, retrieval cost and necessary human follow-up.
  5. Completion rate tracks successful end-to-end task outcomes.

Explanation

Correct Answer

D:

For the proposed professional-services use, the key operating principle is that unit economics include token usage, retrieval cost and necessary human follow-up; that mechanism is what enables Production cost per interaction to compare model operating cost against each successfully handled request. The technical limitation—cheap requests may be expensive overall if retries or escalations rise—needs a separate check.

Incorrect Answers

A:

In this professional-services question about Production cost per interaction, the statement instead characterizes Amazon Bedrock Model Evaluation: Bedrock evaluation capabilities help structure model assessment and comparisons. That can support a goal to compare supported foundation models on relevant quality and safety criteria, but does not explain the operation required for Production cost per interaction to compare model operating cost against each successfully handled request.

B:

In this professional-services question about Production cost per interaction, the statement instead characterizes LLM-as-a-judge: a judge model scores outputs against defined quality criteria. That can support a goal to scale rubric-based evaluation of many candidate responses, but does not explain the operation required for Production cost per interaction to compare model operating cost against each successfully handled request.

C:

In this professional-services question about Production cost per interaction, the statement instead characterizes BLEU scoring: BLEU uses n-gram precision with a brevity penalty across candidate translations. That can support a goal to evaluate machine translation by comparing generated text to reference translations, but does not explain the operation required for Production cost per interaction to compare model operating cost against each successfully handled request.

E:

In this professional-services question about Production cost per interaction, the statement instead characterizes Task completion rate: completion rate tracks successful end-to-end task outcomes. That can support a goal to measure whether real users achieve their goals with an assistant, but does not explain the operation required for Production cost per interaction to compare model operating cost against each successfully handled request.

AWS Certified AI Practitioner AIF-C01 Practice Tests