AI-300 — Operationalizing Machine Learning and Generative AI Solutions Study Guide

Question 43

Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution. After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear on the review screen. You work in Microsoft Foundry with a prompt flow. You must manually evaluate prompts and compare results across prompt variants. You need to capture the inputs, outputs, token usage, and latencies for each flow run for the evaluation. Solution: Create prompt variants and compare their outputs in the Evaluation experience. Does the solution meet the goal?

Question 19

A team is validating a generative AI assistant for a company. The assistant generates responses by using internal knowledge sources. The company requires assurance that responses are accurate, supported by sources, and related to the user prompts before enabling production access. You need to implement quality metrics that confirm the assistant produces reliable and meaningful responses. Which two evaluation metrics should you use? Each correct answer presents part of the solution. NOTE: Each correct selection is worth one point.

Question 37

Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution. After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear on the review screen. You work in Microsoft Foundry with a prompt flow. You must manually evaluate prompts and compare results across prompt variants. You need to capture the inputs, outputs, token usage, and latencies for each flow run for the evaluation. Solution: In Microsoft Foundry, turn on Tracing for the prompt flow of the project and execute test runs to produce trace data. Does the solution meet the goal?

Question 3

You are preparing training data for a fine-tuning job in Microsoft Foundry. Real production conversations cannot be used due to compliance requirements. You need to generate synthetic interaction data that can be used for fine-tuning a generative model. What should you do?

Question 1

A team is building a Retrieval-Augmented Generation (RAG) system. The team observes that the retrieved documents are often irrelevant or incomplete. You need to improve retrieval accuracy. What should you adjust?

Question 7

A product team is building a customer support assistant that must respond consistently across multiple channels. Early testing shows that small wording changes in prompts cause large differences in tone and factual accuracy. The team needs prompts that are reliable, reusable, and adaptable across multiple use cases without retraining the underlying model. You need to design prompts that improve response quality while remaining flexible for future changes. Which two actions should you perform? Each correct answer presents part of the solution. (Choose two.) NOTE: Each correct selection is worth one point.

Question 25

You are authoring a notebook in Azure Machine Learning studio. You must install packages from the notebook into the currently running kernel. The installation must be limited to the currently running kernel only. You need to install the packages. Which magic function should you use?

Question 50

A team is developing a generative AI assistant. The team is experimenting with multiple prompt variants to improve the user experience. When comparing prompt variants, the team plans to assess whether the generated responses are grammatically correct. You need to evaluate the quality of the language from the generated responses. Which evaluator should you use?

Question 29

Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution. After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear in the review screen. You have an Azure Machine Learning workspace. You connect to a terminal session from the Notebooks page in Azure Machine Learning studio. You plan to add a new Jupyter kernel that will be accessible from the same terminal session. You need to perform the task that must be completed before you can add the new kernel. Solution: Delete the Python 3.6 - AzureML kernel. Does the solution meet the goal?

Question 45

You create a binary classification model. You use the Fairlearn package to assess model fairness. You must eliminate the need to retrain the model. You need to implement the Fairlearn package. Which algorithm should you use?

Question 22

You create an Azure Machine Learning workspace. You train an MLflow-formatted regression model by using tabular structured data. You must use a Responsible AI dashboard to assess the model. You need to use the Azure Machine Learning studio UI to generate the Responsible AI dashboard. What should you do first?

Question 46

You manage a Microsoft Foundry project. You build a multi-turn chatbot application. You plan to filter your traces to identify issues while observing how the application is responding. The solution must not use an external knowledge base. You need to select an evaluation metric. Which built-in evaluator should you use?

Question 17

You are training machine learning models in Azure Machine Learning. You use Hyperdrive to tune the hyperparameters. In previous model training and tuning runs, many models showed similar performance. You need to select an early termination policy that meets the following requirements: • accounts for the performance of all previous runs when evaluating the current run • avoids comparing the current run with only the best performing run to date Which two early termination policies should you use? Each correct answer presents part of the solution. NOTE: Each correct selection is worth one point.

Question 8

Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution. After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear in the review screen. You work in Microsoft Foundry with a prompt flow. You must manually evaluate prompts and compare results across prompt variants. You need to capture the inputs, outputs, token usage, and latencies for each flow run for the evaluation. Solution: Configure Azure Monitor to collect logs from the workspace. Use the logs to perform prompt evaluation. Does the solution meet the goal?

Question 21

DRAG DROP - A customer-facing web application uses a foundational model deployed through Microsoft Foundry. A new model version must be introduced and validated without disrupting production traffic. You need to deploy the new version by using a safe promotion strategy. Which three actions should you perform in sequence? To answer, move the appropriate actions from the list of actions to the answer area and arrange them in the correct order.