A company uses Amazon S3 and AWS Glue Data Catalog to manage a data lake that contains contact information for customers. The company uses PySpark and AWS Glue jobs with a DynamicFrame to run a workflow that processes data within the data lake.
A data engineer notices that the workflow is generating errors as a result of how customer postal codes are stored in the data lake. Some postal codes include unnecessary numbers or invalid characters.
The data engineer needs a solution to address the errors and correct the postal codes in the data lake.
A. Create a schema definition for PySpark that matches the format the processing workflow requires for postal codes. Pass the schema to the DynamicFrame during processing.
B. Use AWS Glue workflow properties to allow job state sharing. Configure the AWS Glue jobs to read values from the postal code column by using the properties from a previously successful run of the jobs.
C. Configure the column.push_down_predicate setting and the catalogPartitionPredicate settings for the postal code column in the DynamicFrame.
D. Set the DynamicFrame additional_options parameter ‘useS3ListImplementation’ to True.
Show Answer
Correct Answer: A
Explanation: The problem is caused by inconsistent or invalid postal code values in the data lake. Defining and applying an explicit schema for the postal code field ensures that AWS Glue and PySpark interpret the column in the expected format, enabling casting, validation, and correction during processing. This directly addresses data quality issues that lead to runtime errors. The other options relate to workflow state sharing, query optimization, or S3 listing behavior and do not correct invalid data values.
Question 21
A company is developing machine learning (ML) models. A data engineer needs to apply data quality rules to training data. The company stores the training data in an Amazon S3 bucket.
Which solution will meet these requirements with the LEAST operational overhead?
A. Create an AWS Lambda function to check data quality and to raise exceptions in the code. Run the function when data is added to the S3 bucket. Create an Amazon CloudWatch alarm for exceptions in the code.
B. Create an AWS Glue DataBrew project for the data in the S3 bucket. Create a ruleset for the data quality rules. Create a profile job to run the data quality rules. Use Amazon EventBridge to run the profile job when data is added to the S3 bucket.
C. Create an Amazon EMR provisioned cluster. Add a Python open source data quality package to the EMR cluster. Use the Python package to write code for data quality rules and to copy the data from the S3 bucket to the EMR cluster. Copy the data from the S3 bucket to the EMR cluster. Run the data quality rules.
D. Create AWS Lambda functions to evaluate data quality rules. Use AWS Step Functions to orchestrate a workflow that publishes notifications when the data fails to meet data quality rules.
Show Answer
Correct Answer: B
Explanation: AWS Glue DataBrew is a fully managed, low-code service designed for profiling, validating, and applying data quality rules on data stored in Amazon S3, including ML training data. Creating a ruleset and running a profile job provides built-in data quality checks without custom code or infrastructure management. Triggering the job with Amazon EventBridge when new data arrives further minimizes operational overhead compared to managing Lambda logic, Step Functions workflows, or EMR clusters.
Question 22
A data engineer needs to run a data transformation job whenever a user adds a file to an Amazon S3 bucket. The job will run for less than 1 minute. The job must send the output through an email message to the data engineer. The data engineer expects users to add one file every hour of the day.
Which solution will meet these requirements in the MOST operationally efficient way?
A. Create a small Amazon EC2 instance that polls the S3 bucket for new files. Run transformation code on a schedule to generate the output. Use operating system commands to send email messages.
B. Run an Amazon Elastic Container Service (Amazon ECS) task to poll the S3 bucket for new files. Run transformation code on a schedule to generate the output. Use operating system commands to send email messages.
C. Create an AWS Lambda function to transform the data. Use Amazon S3 Event Notifications to invoke the Lambda function when a new object is created. Publish the output to an Amazon Simple Notification Service (Amazon SNS) topic. Subscribe the data engineer’s email account to the topic.
D. Deploy an Amazon EMR cluster. Use EMR File System (EMRFS) to access the files in the S3 bucket. Run transformation code on a schedule to generate the output to a second S3 bucket. Create an Amazon Simple Notification Service (Amazon SNS) topic. Configure Amazon S3 Event Notifications to notify the topic when a new object is created.
Show Answer
Correct Answer: C
Explanation: The job is event-driven, short-running (<1 minute), and infrequent (about once per hour), which is ideal for a serverless approach. Amazon S3 Event Notifications can directly trigger an AWS Lambda function when a new object is added, eliminating polling and infrastructure management. Lambda easily handles the transformation within the time limit, and publishing the result to Amazon SNS allows simple email delivery by subscribing the engineer’s email address. This solution requires the least operational overhead compared with EC2, ECS, or EMR, which would be unnecessarily complex and costly.
Question 22
A company needs to implement a data mesh architecture in which domains for trading, risk, and compliance teams each have own their data. The teams need to share specific views with one another. The teams have over 1,000 tables across 50 databases in AWS Glue Data Catalog. All three teams use Amazon Athena to perform on-demand analysis. The teams use Amazon Redshift to generate complex reports. The compliance team must audit all data access. Access to personally identifiable information (PII) data must be restricted.
The company requires a scalable solution to meet the team requirements. The solution must provide the ability to perform analysis across team domains.
Which solution will meet these requirements?
A. Create views in Athena for on-demand analysis. Use the Athena views in Amazon Redshift to perform cross-domain analytics. Use AWS CloudTrail to audit data access. Use AWS Lake Formation to establish fine-grained access control.
B. Use AWS Glue Data Catalog views to perform analysis. Use AWS CloudTrail logs to audit data access. Use AWS Lake Formation to manage access permissions. Use security definer views to mask PII.
C. Use AWS Lake Formation to set up cross-domain access to tables. Set up fine-grained access controls.
D. Create materialized views and enable Amazon Redshift datashares for each domain. Configure cross-domain access policies.
Show Answer
Correct Answer: A
Explanation: Option A best meets all requirements. Athena views enable sharing curated, domain-owned views across teams in a data mesh. Amazon Redshift can query the same data and views through AWS Glue Data Catalog integration (Redshift Spectrum) for complex reporting. AWS Lake Formation provides scalable, fine-grained permissions to restrict PII and control cross-domain access, while AWS CloudTrail supports auditing of data access. Option C is incomplete because it does not address auditing, view-based sharing, or Redshift/Athena usage, and B relies on Glue Catalog views, which are not the primary or scalable mechanism for Athena/Redshift analytics.
Question 23
A data engineer is building a data pipeline. A large data file is uploaded to an Amazon S3 bucket once each day at unpredictable times. An AWS Glue workflow uses hundreds of workers to process the file and load the data into Amazon Redshift. The company wants to process the file as quickly as possible.
Which solution will meet these requirements?
A. Create an on-demand AWS Glue trigger to start the workflow. Create an AWS Lambda function that runs every 15 minutes to check the S3 bucket for the daily file. Configure the function to start the AWS Glue workflow if the file is present.
B. Create an event-based AWS Glue trigger to start the workflow. Configure Amazon S3 to log events to AWS CloudTrail. Create a rule in Amazon EventBridge to forward PutObject events to the AWS Glue trigger.
C. Create a scheduled AWS Glue trigger to start the workflow. Create a cron job that runs the AWS Glue job every 15 minutes. Set up the AWS Glue job to check the S3 bucket for the daily file. Configure the job to stop if the file is not present.
D. Create an on-demand AWS Glue trigger to start the workflow. Create an AWS Database Migration Service (AWS DMS) migration task. Set the DMS source as the S3 bucket. Set the target endpoint as the AWS Glue workflow.
Show Answer
Correct Answer: B
Explanation: The file arrives at unpredictable times and must be processed as soon as it lands. An event-driven approach is optimal. Amazon S3 can emit PutObject events (via CloudTrail) that are routed through Amazon EventBridge to an event-based AWS Glue trigger, which immediately starts the workflow. This avoids polling delays, scales automatically, and starts processing as soon as the upload completes. The other options rely on periodic polling or inappropriate services, which introduce latency or unnecessary complexity.
Question 23
A data engineer at a company is optimizing extract, transform, and load (ETL) workflows. The current architecture uses Amazon EMR and Apache Spark for large-scale transformations and AWS Glue for other ETL tasks. The workflows load processed data into an Amazon S3 based data lake.
The company wants to move to a fully managed serverless solution that can orchestrate multiple ETL jobs and automate execution. The new solution must continue to use Spark to process data. The company needs to orchestrate and automate the ETL workflows with minimal manual intervention.
Which solution will meet these requirements?
A. Migrate all ETL jobs to AWS Glue. Use AWS Glue workflows to orchestrate the pipeline.
B. Configure AWS Step Functions and Amazon EventBridge to orchestrate and invoke ETL workflows in AWS Glue and Amazon EMR.
C. Configure AWS Lambda functions to process Amazon S3 event notifications for data transformation tasks when new data is uploaded.
D. Use Amazon Managed Workflows for Apache Airflow automatic scheduling to orchestrate the Spark-based ETL jobs.
Show Answer
Correct Answer: A
Explanation: The requirement is a fully managed, serverless solution that continues to use Spark and can orchestrate and automate multiple ETL jobs. AWS Glue is serverless, runs Apache Spark natively, and provides Glue Workflows for built-in orchestration, triggers, and dependencies with minimal manual intervention. Other options either involve non-serverless components (EMR), add unnecessary orchestration complexity, or do not meet Spark orchestration needs.
Question 24
A company manages an Amazon Redshift data warehouse. The data warehouse is in a public subnet inside a custom VPC. A security group allows only traffic from within itself. An ACL is open to all traffic.
The company wants to generate several visualizations in Amazon QuickSight for an upcoming sales event. The company will run QuickSight Enterprise edition in a second AWS account inside a public subnet within a second custom VPC. The new public subnet has a security group that allows outbound traffic to the existing Redshift cluster.
A data engineer needs to establish connections between Amazon Redshift and QuickSight. QuickSight must refresh dashboards by querying the Redshift cluster.
Which solution will meet these requirements?
A. Configure the Redshift security group to allow inbound traffic on the Redshift port from the QuickSight security group.
B. Assign Elastic IP addresses to the QuickSight visualizations. Configure the QuickSight security group to allow inbound traffic on the Redshift port from the Elastic IP addresses.
C. Confirm that the CIDR ranges of the Redshift VPC and the QuickSight VPC are the same. If CIDR ranges are different, reconfigure one CIDR range to match the other. Establish network peering between the VPCs.
D. Create a QuickSight gateway endpoint in the Redshift VPC. Attach an endpoint policy to the gateway endpoint to ensure only specific QuickSight accounts can use the endpoint.
Show Answer
Correct Answer: A
Explanation: Amazon QuickSight needs network-level access to the Amazon Redshift cluster to issue queries for dashboard refreshes. Redshift is already publicly reachable, and the network ACL is open. The missing requirement is allowing inbound traffic to the Redshift cluster itself. By updating the Redshift security group to allow inbound traffic on the Redshift port (default 5439) from the QuickSight security group, QuickSight can securely connect. VPC peering is not required for public endpoints, Elastic IPs are not used by QuickSight in this way, and gateway endpoints are not applicable to Redshift–QuickSight connectivity.
Question 24
A data engineer is using an Apache Iceberg framework to build a data lake that contains 100 ТВ of data. The data engineer wants to run AWS Glue Apache Spark jobs that use the Iceberg framework.
What combination of steps will meet these requirements? (Choose two.)
A. Create a key named --conf for an AWS Glue job. Set Iceberg as a value for the --datalake-formats job parameter.
B. Specify the path to a specific version of Iceberg by using the -extra-jars job parameter. Set Iceberg as a value for the datalake-formats job parameter.
C. Set Iceberg as a value for the --datalake-formats job parameter.
D. Set the --enable-auto-scaling parameter to true.
E. Add the --job-bookmark-option: job-bookmark-enable parameter to an AWS Glue job.
Show Answer
Correct Answer: C, D
Explanation: To run AWS Glue Apache Spark jobs with Apache Iceberg, the job must explicitly enable Iceberg support by setting the --datalake-formats job parameter to Iceberg. This is the required configuration for Glue to load the Iceberg runtime. Given the large data volume (100 TB), enabling --enable-auto-scaling allows Glue to automatically scale executors to handle the workload efficiently. The other options are unnecessary or incorrect because Glue provides Iceberg support natively without manually supplying Iceberg JARs or unrelated parameters.
Question 25
A data engineer is optimizing query performance in Amazon Athena notebooks that use Apache Spark to analyze large datasets that are stored in Amazon S3. The data is partitioned.
An AWS Glue crawler updates the partitions.
The data engineer wants to minimize the amount of data that is scanned to improve efficiency of Athena queries.
Which solution will meet these requirements?
A. Apply partition filters in the queries.
B. Increase the frequency of AWS Glue crawler invocations to update the data catalog more often.
C. Organize the data that is in Amazon S3 by using a nested directory structure.
D. Configure Spark to use in-memory caching for frequently accessed data.
Show Answer
Correct Answer: A
Explanation: Using partition filters in Athena queries (for example, WHERE clauses on partition columns) enables partition pruning. Athena then reads only the relevant partitions instead of scanning all data in Amazon S3, which directly minimizes the amount of data scanned and improves query performance and cost efficiency. The other options do not directly reduce scanned data at query time.
Question 25
A data engineer is configuring an AWS Glue Apache Spark extract, transform, and load (ETL) job. The job contains a sort-merge join of two large and equally sized DataFrames.
The job is failing with the following error: No space left on device.
Which solution will resolve the error?
A. Use the AWS Glue Spark shuffle manager.
B. Deploy are Amazon Elastic Block Store (Amazon EBS) volume for the job to use.
C. Convert the sort-merge join in the job to be a broadcast join.
D. Convert the DataFrames to DynamicFrames, and perform a DynamicFrame join in the job.
Show Answer
Correct Answer: A
Explanation: A sort-merge join on two large, equally sized DataFrames generates a large shuffle that spills to local disk. The error indicates the executor’s local disk is exhausted. Enabling the AWS Glue Spark (S3-based external) shuffle manager writes shuffle files to Amazon S3 instead of local disk, eliminating the local storage bottleneck. A broadcast join is unsafe because neither DataFrame is small, Glue jobs do not support attaching EBS volumes, and DynamicFrame joins still rely on Spark shuffles.
$19
Get all 279 questions with detailed answers and explanations
Instant download HTML + PDF delivered the moment payment clears.
Secure Stripe checkout we never see or store your card details.
7-day refund if files are defective see our refund policy.