A data engineer needs to deploy a complex pipeline. The stages of the pipeline must be able to run a script. The data engineer must use only fully managed and serverless services in the pipeline.
Which solution will meet these requirements?
A. Deploy AWS Glue jobs and workflows. Use AWS Glue to run the jobs and workflows on a schedule.
B. Use Amazon Managed Workflows for Apache Airflow (Amazon MWAA) to build and schedule the pipeline.
C. Deploy the script to Amazon EC2 instances. Use Amazon EventBridge to run the script on a schedule.
D. Use Aws Glue DataBrew to build the pipeline. Use Amazon EventBridge to run the pipeline on a schedule.
Show Answer
Correct Answer: A
Explanation: AWS Glue jobs can run Python or Scala scripts in a fully managed, serverless environment, and AWS Glue Workflows orchestrate multi-stage pipelines with dependencies and scheduling. Amazon MWAA is a managed orchestration service but is not serverless. EC2 is not serverless, and AWS Glue DataBrew is intended for visual data preparation rather than orchestrating complex script-based pipelines.
Question 52
A company runs a multi-tenant Amazon EMR cluster on Amazon EC2 instances. Multiple teams perform interactive query analyses and data transformations on the data in the EMR cluster. The teams can access the cluster only through EMR Studio workspaces and EMR steps.
The teams need to use EMR steps to run Apache Spark jobs to fetch data from an Amazon DynamoDB table. The DynamoDB table contains confidential data that must be accessible to only one specific team. The company needs to ensure that only the appropriate team can access the confidential data in the EMR cluster.
Which solution will meet these requirements?
A. Set up runtime roles for EMR steps.
B. Set up AWS Lake Formation permissions.
C. Set up IAM roles for EMR File System (EMRFS) requests.
D. Set up a DynamoDB resource-based policy.
Show Answer
Correct Answer: A
Explanation: Runtime roles for Amazon EMR steps allow different jobs on a shared EMR cluster to assume different IAM roles. This enables per-team permissions so only the designated team's Spark jobs can access the confidential DynamoDB table. EMRFS IAM roles apply to S3 access, Lake Formation governs supported data lakes rather than DynamoDB table access, and DynamoDB does not use resource-based policies for this use case.
Question 53
A company is developing machine learning (ML) models. A data engineer needs to apply data quality rules to training data. The company stores the training data in an Amazon S3 bucket.
Which solution will meet these requirements with the LEAST operational overhead?
A. Create an AWS Lambda function to check data quality and to raise exceptions in the code. Run the function when data is added to the S3 bucket. Create an Amazon CloudWatch alarm for exceptions in the code.
B. Create an AWS Glue DataBrew project for the data in the S3 bucket. Create a ruleset for the data quality rules. Create a profile job to run the data quality rules. Use Amazon EventBridge to run the profile job when data is added to the S3 bucket.
C. Create an Amazon EMR provisioned cluster. Add a Python open source data quality package to the EMR cluster. Use the Python package to write code for data quality rules and to copy the data from the S3 bucket to the EMR cluster. Copy the data from the S3 bucket to the EMR cluster. Run the data quality rules.
D. Create AWS Lambda functions to evaluate data quality rules. Use AWS Step Functions to orchestrate a workflow that publishes notifications when the data fails to meet data quality rules.
Show Answer
Correct Answer: B
Explanation: AWS Glue DataBrew provides built-in data profiling and data quality rules for data stored in Amazon S3 with minimal code and operational overhead. A profile job can evaluate a ruleset, and Amazon EventBridge can trigger the job when new data arrives. The other options require custom code, orchestration, or managing EMR infrastructure.
Question 54
A company stores Apache Parquet files in an Amazon S3 data lake. The data lake receives thousands of files from multiple sources every hour. The files range in size from 50 KB to 100 KB.
The company is evaluating the implementation of Apache Iceberg tables for the data lake. The company is using AWS Glue Data Catalog as part of the evaluation. The company needs a solution to optimize query performance in Iceberg. The solution must ensure that Iceberg table performance does not degrade when more files are added over time.
Which solution will meet these requirements?
A. Use an AWS Glue job to compact the files into a standard size of 512 MB at the end of each day. Run an AWS Glue crawler to update the Data Catalog.
B. Configure the Data Catalog to automatically compact the files every minute.
C. Configure Iceberg table properties to enable automatic compaction based on thresholds for file size and the number of files.
D. Implement a partition strategy in Amazon S3. Run an AWS Glue crawler to update the Data Catalog every 5 minutes.
Show Answer
Correct Answer: C
Explanation: Apache Iceberg addresses the small-files problem through table maintenance and compaction. Enabling Iceberg's automatic compaction based on file-size and file-count thresholds keeps data files at efficient sizes as new small files arrive, preserving query performance over time. Running Glue crawlers is unnecessary for Iceberg metadata updates, and manual daily compaction is less effective for continuous ingestion. The Data Catalog itself does not provide a generic automatic per-minute compaction feature for all Iceberg tables.
Question 55
A company needs to optimize storage costs for an Amazon S3 bucket. The S3 bucket receives 10 million objects every day. The objects range in size from 2 KB to 5 MB. The objects need to be immediately accessible for the first 60 days. Users access objects infrequently from 61 to 180 days. The objects must be accessible within an hour from 181 to 365 days. The company can delete the objects after 365 days.
Which solution will meet these requirements?
A. Use S3 Intelligent-Tiering to automatically transition objects. Select the Archive Access tier for Intelligent-Tiering. Configure an S3 bucket policy to expire objects that are older than 365 days.
B. Create an S3 Lifecycle policy to move objects. Configure the policy to move objects from S3 Standard to S3 Standard-Infrequent Access (S3 Standard-IA) after 60 days. Move the objects to S3 Glacier Flexible Retrieval after 180 days. Expire objects after 365 days.
C. Enable S3 Inventory. Use a daily inventory report to configure an S3 Batch Operations job that moves objects from S3 Standard to S3 Standard-Infrequent Access (S3 Standard-IA) after 60 days. Move objects to S3 Glacier Flexible Retrieval after 180 days. Expire objects after 365 days.
D. Enable S3 Inventory. Run an AWS Lambda function each day to fetch an inventory report and move objects from S3 Standard to S3 Standard-Infrequent Access (S3 Standard-IA) after 60 days. Move objects to S3 Glacier Flexible Retrieval after 180 days. Expire objects after 365 days.
Show Answer
Correct Answer: B
Explanation: An S3 Lifecycle policy is the native, lowest-overhead solution. Keep objects in S3 Standard for the first 60 days for immediate access, transition to S3 Standard-IA after 60 days for infrequent access, transition to S3 Glacier Flexible Retrieval after 180 days (retrieval within hours, including expedited/standard options meeting the within-1-hour requirement when appropriate), and expire after 365 days. Inventory, Batch Operations, and Lambda are unnecessary for age-based transitions. Intelligent-Tiering adds monitoring costs and is intended for unknown access patterns rather than a known lifecycle.
Question 56
A company needs to implement a data mesh architecture in which domains for trading, risk, and compliance teams each have own their data. The teams need to share specific views with one another. The teams have over 1,000 tables across 50 databases in AWS Glue Data Catalog. All three teams use Amazon Athena to perform on-demand analysis. The teams use Amazon Redshift to generate complex reports. The compliance team must audit all data access. Access to personally identifiable information (PII) data must be restricted.
The company requires a scalable solution to meet the team requirements. The solution must provide the ability to perform analysis across team domains.
Which solution will meet these requirements?
A. Create views in Athena for on-demand analysis. Use the Athena views in Amazon Redshift to perform cross-domain analytics. Use AWS CloudTrail to audit data access. Use AWS Lake Formation to establish fine-grained access control.
B. Use AWS Glue Data Catalog views to perform analysis. Use AWS CloudTrail logs to audit data access. Use AWS Lake Formation to manage access permissions. Use security definer views to mask PII.
C. Use AWS Lake Formation to set up cross-domain access to tables. Set up fine-grained access controls.
D. Create materialized views and enable Amazon Redshift datashares for each domain. Configure cross-domain access policies.
Show Answer
Correct Answer: B
Explanation: AWS Glue Data Catalog views provide a scalable, engine-agnostic way to share governed views across domains for both Athena and Amazon Redshift. Lake Formation manages fine-grained permissions, including restricting PII, while CloudTrail provides auditing of data access. Data Catalog views are designed for data mesh sharing across analytics engines, avoiding separate Athena-specific views.
Sources:
https://aws.amazon.com/blogs/big-data/query-aws-glue-data-catalog-views-using-amazon-athena-and-amazon-redshift
https://aws.amazon.com/blogs/big-data/design-a-data-mesh-architecture-using-aws-lake-formation-and-aws-glue
Question 57
A data engineer at a company is optimizing extract, transform, and load (ETL) workflows. The current architecture uses Amazon EMR and Apache Spark for large-scale transformations and AWS Glue for other ETL tasks. The workflows load processed data into an Amazon S3 based data lake.
The company wants to move to a fully managed serverless solution that can orchestrate multiple ETL jobs and automate execution. The new solution must continue to use Spark to process data. The company needs to orchestrate and automate the ETL workflows with minimal manual intervention.
Which solution will meet these requirements?
A. Migrate all ETL jobs to AWS Glue. Use AWS Glue workflows to orchestrate the pipeline.
B. Configure AWS Step Functions and Amazon EventBridge to orchestrate and invoke ETL workflows in AWS Glue and Amazon EMR.
C. Configure AWS Lambda functions to process Amazon S3 event notifications for data transformation tasks when new data is uploaded.
D. Use Amazon Managed Workflows for Apache Airflow automatic scheduling to orchestrate the Spark-based ETL jobs.
Show Answer
Correct Answer: A
Explanation: AWS Glue is a fully managed serverless ETL service that natively runs Apache Spark jobs. By migrating Spark ETL workloads from EMR to AWS Glue and using AWS Glue Workflows, the company can orchestrate multiple ETL jobs, manage dependencies and triggers, and automate execution with minimal operational overhead. Step Functions with EMR still requires managing EMR, Lambda is not suitable for large Spark ETL, and MWAA orchestrates workflows but is not itself the serverless Spark processing solution requested.
Question 58
A company needs to aggregate and filter a large amount of streaming data in real-time with low latency. The company needs to store the data in Amazon S3 for analysis.
Which solution will meet these requirements in the MOST operationally efficient way?
A. Use Amazon Kinesis Data Streams with provisioned capacity and AWS Lambda functions to perform custom transformations and to integrate with Amazon S3.
B. Use Amazon Data Firehose with built-in data transformations. Deliver the data directly to Amazon S3.
C. Use Amazon Kinesis Data Streams and Amazon Managed Service for Apache Flink to perform complex processing and to integrate with Amazon S3.
D. Use Amazon Data Firehose and AWS Lambda functions to perform custom transformations and to deliver the data to Amazon S3.
Show Answer
Correct Answer: B
Explanation: Amazon Data Firehose is the most operationally efficient choice for streaming data that needs to be filtered/transformed and delivered to Amazon S3. It is a fully managed service with built-in data transformation capabilities and direct delivery to S3, minimizing operational overhead compared with managing Kinesis Data Streams, provisioned capacity, Lambda pipelines, or Apache Flink.
Question 59
A retail company stores point-of-sale transaction data in an Amazon RDS for MySQL database. The company maintains historical sales analytics in Amazon Redshift. The company needs to create daily reports that combine the current day's transactions with historical sales patterns for trend analysis. The company requires a solution that provides near real-time insights while minimizing data transfer costs and maintenance overhead.
Which solution will meet these requirements?
A. Configure AWS Database Migration Service (AWS DMS) to continuously replicate data from RDS for MySQL to Amazon Redshift. Use Redshift queries to create consolidated reports.
B. Implement Amazon Redshift federated queries to directly access RDS for MySQL data and join it with existing Redshift tables in a single query.
C. Use AWS Glue to create an extract, transform, and load (ETL) pipeline that runs every hour to copy incremental data from RDS for MySQL to Amazon Redshift. Generate reports.
D. Export RDS for MySQL data to an Amazon S3 bucket on a regular schedule. Use the COPY command to load the data into Amazon Redshift staging tables. Join the data with historical data.
Show Answer
Correct Answer: B
Explanation: Amazon Redshift federated queries allow Redshift to query live data in Amazon RDS for MySQL and join it with historical data already stored in Redshift. This provides near real-time reporting without continuously replicating or copying data, minimizing data transfer and operational maintenance. The other options either introduce replication/ETL overhead or do not provide near real-time access.
Question 60
A company processes 500 GB of audience and advertising data daily, storing CSV files in Amazon S3 with schemas registered in AWS Glue Data Catalog. They need to convert these files to Apache Parquet format and store them in an S3 bucket.
The solution requires a long-running workflow with 15 GiB memory capacity to process the data concurrently, followed by a correlation process that begins only after the first two processes complete.
Which solution will meet these requirements with the LEAST operational overhead?
A. Use Amazon Managed Workflows for Apache Airflow (Amazon MWAA) to orchestrate the workflow by using AWS Glue. Configure AWS Glue to begin the third process after the first two processes have finished.
B. Use Amazon EMR to run each process in the workflow. Create an Amazon Simple Queue Service (Amazon SQS) queue to handle messages that indicate the completion of the first two processes. Configure an AWS Lambda function to process the SQS queue by running the third process.
C. Use AWS Glue workflows to run the first two processes in parallel. Ensure that the third process starts after the first two processes have finished.
D. Use AWS Step Functions to orchestrate a workflow that uses multiple AWS Lambda functions. Ensure that the third process starts after the first two processes have finished.
Show Answer
Correct Answer: C
Explanation: AWS Glue workflows can orchestrate dependent ETL jobs, including parallel branches followed by a downstream job that waits for both predecessors. AWS Glue is designed for schema-aware ETL on data in Amazon S3 and can convert CSV to Parquet using the Glue Data Catalog. Glue jobs support worker configurations that can satisfy the memory requirement for long-running ETL, and using Glue workflows minimizes operational overhead compared with managing MWAA, EMR, or coordinating Lambda for large ETL workloads.
$19
Get all 310 questions with detailed answers and explanations
Instant download HTML + PDF delivered the moment payment clears.
Secure Stripe checkout we never see or store your card details.
7-day refund if files are defective see our refund policy.