A company built a Retrieval Augmented Generation (RAG) based AI assistant by using Amazon Bedrock Agents. Sometimes the AI assistant gives incomplete or inaccurate answers. The company wants to improve and evaluate the AI assistant's performance.
Which solution will meet these requirements?
A. Use automated metrics such as Bilingual Evaluation Understudy (BLEU) scores to measure response similarity to reference answers.
B. Monitor response latency and token generation speed to optimize performance.
C. Evaluate only the final answer quality against ground truth responses.
D. Confirm that the retrieved documents are relevant and that the final answers are accurate.
Show Answer
Correct Answer: D
Explanation: For a RAG-based AI assistant, evaluation should cover both stages of the pipeline: retrieval quality and generation quality. Incomplete or inaccurate answers can result from irrelevant retrieved documents, poor answer synthesis, or both. Verifying that retrieved documents are relevant and that the final answers are accurate provides a more complete evaluation than measuring latency, BLEU alone, or only final-answer accuracy.
Question 22
A company plans to use a dataset to train a new AI model. The company needs to determine whether a bias against a specific user group exists in the dataset before the company trains the model.
Which solution will meet these requirement?
A. Use Amazon SageMaker Clarify to compute pre-training bias metrics.
B. Use Amazon SageMaker Model Cards to compute pre-training bias metrics.
C. Use Amazon SageMaker Clarify to compute post-training bias metrics.
D. Use Amazon SageMaker Model Cards to compute post-training bias metrics.
Show Answer
Correct Answer: A
Explanation: Amazon SageMaker Clarify is designed to detect and measure bias in datasets before model training by computing pre-training bias metrics. Model Cards document model information and evaluation results rather than computing bias metrics. Post-training bias metrics require a trained model, which does not meet the requirement to assess the dataset before training.
Question 23
A software development company wants to increase its speed to market delivery. The company needs a solution that helps developers generate real-time code and unit tests.
Which AWS service meets these requirements?
A. Amazon Personalize
B. Amazon Q Developer
C. Amazon Comprehend
D. Amazon Kendra
Show Answer
Correct Answer: B
Explanation: Amazon Q Developer is the AWS AI coding assistant designed to provide real-time code suggestions, generate unit tests, and help developers build, debug, and optimize software, improving development speed. Amazon Personalize is for recommendation systems, Amazon Comprehend is for natural language processing, and Amazon Kendra is an enterprise search service.
Question 24
HOTSPOT
-
A company wants to use Amazon Bedrock to create a knowledge base from existing documents. The documents include text, tables, and images.
Select and order the steps from the following list to correctly describe how to convert data from the documents into a knowledge base by using an ingestion process in Amazon Bedrock. Select each step one time or not at all.
Show Answer
Correct Answer: Step 1: Parse the documents.
Step 2: Divide the data into chunks.
Step 3: Convert the chunks into vector embeddings.
Step 4: Write the vector embeddings to the vector store.
Explanation: Amazon Bedrock knowledge base ingestion parses source documents, chunks the content, generates embeddings for each chunk, and stores those embeddings in a vector store. RAG is used later during retrieval, not ingestion, and data lineage is unrelated.
Question 25
A company is developing an AI solution to help make hiring decisions.
Which strategy complies with AWS guidance for responsible AI?
A. Use the AI solution to make final hiring decisions without human review.
B. Train the AI solution exclusively on data from previous successful hires.
C. Test the AI solution to ensure that it does not discriminate against any protected groups.
D. Keep the AI decision-making process confidential to maintain a competitive advantage.
Show Answer
Correct Answer: C
Explanation: AWS Responsible AI guidance emphasizes fairness, bias detection and mitigation, and human oversight for high-impact decisions. Testing a hiring AI to ensure it does not discriminate against protected groups aligns with these principles. The other options conflict with responsible AI practices by removing human oversight, reinforcing historical bias, or reducing transparency.
Question 26
An AI practitioner wants to use Retrieval Augmented Generation (RAG) architecture. However, company leadership thinks RAG will make the architecture too complex.
Which response will support the AI practitioner's recommendation to use RAG?
A. RAG improves large language model (LLM) inference time.
B. RAG improves model training speed.
C. RAG decreases hallucinations.
D. RAG decreases development controls.
Show Answer
Correct Answer: C
Explanation: Retrieval-Augmented Generation (RAG) retrieves relevant external information to ground an LLM's responses, which reduces hallucinations and improves factual accuracy. It does not improve training speed, typically adds inference latency rather than reducing it, and does not decrease development controls.
Question 27
A company is preparing data to fine-tune a large language model (LLM). The company must follow the principles of responsible AI.
Which actions will meet this requirement? (Choose two.)
A. Include demographically diverse data sources to improve model fairness.
B. Apply consistent labeling standards across the training dataset.
C. Focus exclusively on high-volume data regardless of quality.
D. Use only synthetic data to train the model.
E. Use fewer labels to reduce ambiguity in data during model training.
Show Answer
Correct Answer: A, B
Explanation: Responsible AI data preparation includes promoting fairness through demographically diverse training data and maintaining high data quality with consistent labeling. Prioritizing volume over quality, using only synthetic data, or reducing labels to minimize ambiguity are not responsible AI best practices.
Question 28
A company has created an AI assistant to provide relevant information to employees about internal processes. The AI assistant uses a fine-tuned foundation model (FM). The company wants to use feedback from employees to further refine the model's responses.
Which solution will meet these requirements?
A. Implement model distillation.
B. Perform continued pre-training and incorporate feedback from employee ratings.
C. Use reinforcement learning from human feedback (RLHF).
D. Enable intelligent prompt routing.
Show Answer
Correct Answer: C
Explanation: The requirement is to improve an already fine-tuned foundation model using employee feedback on response quality. Reinforcement learning from human feedback (RLHF) is specifically designed to use human ratings or preferences to optimize model behavior. Model distillation compresses models rather than incorporating feedback, continued pre-training updates knowledge from unlabeled data rather than preference ratings, and intelligent prompt routing selects models or prompts without refining the model itself.
Question 29
A company has fine-tuned an Amazon Bedrock foundation model (FM) to produce short document summaries. The company wants an automated metric that compares each model-generated summary with its human-written reference summary.
Which metric will meet these requirements?
A. F1 score
B. Recall-Oriented Understudy for Gisting Evaluation (ROUGE)
C. Perplexity
D. Fréchet inception distance (FID)
Show Answer
Correct Answer: B
Explanation: ROUGE is the standard automated evaluation metric for text summarization. It compares machine-generated summaries with one or more human reference summaries using measures such as n-gram overlap and longest common subsequence. F1 is primarily for classification tasks, perplexity evaluates language model predictive performance rather than summary quality against a reference, and FID is used for image generation quality.
Question 30
A company is building a chatbot. The chatbot uses a large language model (LLM) and answers customer questions about products. The company wants the chatbot to answer only product questions. The company does not want the chatbot to answer questions about other topics.
Which solution will meet these requirements with the LEAST operational overhead?
A. Set guardrails on the LLM prompt template.
B. Write custom application logic to identify questions about other topics.
C. Reduce the information the LLM can access.
D. Set the temperature parameter value to a lower number.
Show Answer
Correct Answer: A
Explanation: Setting guardrails on the LLM prompt template is the lowest-overhead way to constrain the model to product-related topics and instruct it to refuse or redirect out-of-scope questions. Custom application logic requires ongoing maintenance, reducing accessible information does not reliably prevent off-topic responses, and lowering the temperature affects randomness rather than topic restriction.
$19
Get all 443 questions with detailed answers and explanations
Instant download HTML + PDF delivered the moment payment clears.
Secure Stripe checkout we never see or store your card details.
7-day refund if files are defective see our refund policy.
