Professional Data Engineer Free Practice Questions — Page 9
Question 63
You are part of a healthcare organization where data is organized and managed by respective data owners in various storage services. As a result of this decentralized ecosystem, discovering and managing data has become difficult. You need to quickly identify and implement a cost-optimized solution to assist your organization with the following:
• Data management and discovery
• Data lineage tracking
• Data quality validation
How should you build the solution?
A. Use BigLake to convert the current solution into a data lake architecture.
B. Build a new data discovery tool on Google Kubernetes Engine that helps with new source onboarding and data lineage tracking.
C. Use BigQuery to track data lineage, and use Dataprep to manage data and perform data quality validation.
D. Use Dataplex to manage data, track data lineage, and perform data quality validation.
Show Answer
Correct Answer: D
Explanation: Dataplex is Google Cloud's unified data fabric and data management service designed for decentralized data environments. It provides centralized data discovery and governance across storage systems, supports data lineage tracking, and includes built-in data quality capabilities, making it the most complete and cost-optimized managed solution. BigLake focuses on unified storage access rather than governance, building a custom GKE tool is unnecessary and costly, and BigQuery with Dataprep does not provide the integrated governance and lineage capabilities that Dataplex offers.
Question 64
You are administering shared BigQuery datasets that contain views used by multiple teams in your organization. The marketing team is concerned about the variability of their monthly BigQuery analytics spend using the on-demand billing model. You need to help the marketing team establish a consistent BigQuery analytics spend each month. What should you do?
A. Create a BigQuery Enterprise reservation with a baseline of 250 slots and autoscaling set to 500 for the marketing team, and bill them back accordingly.
B. Establish a BigQuery quota for the marketing team, and limit the maximum number of bytes scanned each day.
C. Create a BigQuery reservation with a baseline of 500 slots with no autoscaling for the marketing team, and bill them back accordingly.
D. Create a BigQuery Standard pay-as-you go reservation with a baseline of 0 slots and autoscaling set to 500 for the marketing team, and bill them back accordingly.
Show Answer
Correct Answer: C
Explanation: A fixed-capacity BigQuery reservation without autoscaling provides the most predictable monthly cost because you pay for a fixed number of reserved slots regardless of query volume. Option A introduces variable costs through autoscaling. Option B keeps on-demand pricing and merely caps usage, which can interrupt workloads rather than provide consistent spending. Option D also uses autoscaling, resulting in variable costs.
Question 65
You have thousands of Apache Spark jobs running in your on-premises Apache Hadoop cluster. You want to migrate the jobs to Google Cloud. You want to use managed services to run your jobs instead of maintaining a long-lived Hadoop cluster yourself. You have a tight timeline and want to keep code changes to a minimum. What should you do?
A. Move your data to BigQuery. Convert your Spark scripts to a SQL-based processing approach.
B. Rewrite your jobs in Apache Beam. Run your jobs in Dataflow.
C. Copy your data to Compute Engine disks. Manage and run your jobs directly on those instances.
D. Move your data to Cloud Storage. Run your jobs on Dataproc.
Show Answer
Correct Answer: D
Explanation: Dataproc is Google Cloud's managed service for Apache Spark and Hadoop workloads. Migrating data to Cloud Storage and running existing Spark jobs on Dataproc minimizes code changes while avoiding management of a long-lived Hadoop cluster, matching the requirements for a managed service and a tight migration timeline.
Question 66
You have a network of 1000 sensors. The sensors generate time series data: one metric per sensor per second, along with a timestamp. You already have 1 TB of data, and expect the data to grow by 1 GB every day. You need to access this data in two ways. The first access pattern requires retrieving the metric from one specific sensor stored at a specific timestamp, with a median single-digit millisecond latency. The second access pattern requires running complex analytic queries on the data, including joins, once a day. How should you store this data?
A. Store your data in BigQuery. Concatenate the sensor ID and timestamp, and use it as the primary key.
B. Store your data in Bigtable. Concatenate the sensor ID and timestamp and use it as the row key. Perform an export to BigQuery every day.
C. Store your data in Bigtable. Concatenate the sensor ID and metric, and use it as the row key. Perform an export to BigQuery every day.
D. Store your data in BigQuery. Use the metric as a primary key.
Show Answer
Correct Answer: B
Explanation: Bigtable is the appropriate choice for low-latency point lookups using a row key. A row key composed of sensor ID and timestamp efficiently supports retrieving a specific sensor's metric at an exact time with single-digit millisecond latency. BigQuery is designed for analytical workloads rather than low-latency key-based retrieval, so exporting the data daily from Bigtable to BigQuery satisfies the once-per-day complex analytics and join requirements.
Question 67
You work for a large ecommerce company. You store your customer's order data in Bigtable. You have a garbage collection policy set to delete the data after 30 days and the number of versions is set to 1. When the data analysts run a query to report total customer spending, the analysts sometimes see customer data that is older than 30 days. You need to ensure that the analysts do not see customer data older than 30 days while minimizing cost and overhead. What should you do?
A. Set the expiring values of the column families to 29 days and keep the number of versions to 1.
B. Use a timestamp range filter in the query to fetch the customer's data for a specific range.
C. Schedule a job daily to scan the data in the table and delete data older than 30 days.
D. Set the expiring values of the column families to 30 days and set the number of versions to 2.
Show Answer
Correct Answer: B
Explanation: Bigtable garbage collection is asynchronous and can take days after data expires before it is physically removed. Therefore, GC policies alone do not guarantee that reads exclude expired data. To ensure analysts never see data older than 30 days while minimizing cost and operational overhead, apply a timestamp range filter on read queries that matches the retention window. Changing the GC age or scheduling deletion jobs does not provide the same read-time guarantee.
Question 68
You are running a streaming pipeline with Dataflow and are using hopping windows to group the data as the data arrives. You noticed that some data is arriving late but is not being marked as late data, which is resulting in inaccurate aggregations downstream. You need to find a solution that allows you to capture the late data in the appropriate window. What should you do?
A. Use watermarks to define the expected data arrival window. Allow late data as it arrives.
B. Change your windowing function to tumbling windows to avoid overlapping window periods.
C. Change your windowing function to session windows to define your windows based on certain activity.
D. Expand your hopping window so that the late data has more time to arrive within the grouping.
Show Answer
Correct Answer: A
Explanation: Watermarks determine event-time progress and when windows are expected to be complete. If data arrives after the watermark, configuring allowed lateness enables those late elements to still be assigned to their correct event-time window and update aggregations. Changing the window type or simply enlarging hopping windows does not address late-event handling.
Question 69
You want to store your team’s shared tables in a single dataset to make data easily accessible to various analysts. You want to make this data readable but unmodifiable by analysts. At the same time, you want to provide the analysts with individual workspaces in the same project, where they can create and store tables for their own use, without the tables being accessible by other analysts. What should you do?
A. Give analysts the BigQuery Data Viewer role at the project level. Create one other dataset, and give the analysts the BigQuery Data Editor role on that dataset.
B. Give analysts the BigQuery Data Viewer role at the project level. Create a dataset for each analyst, and give each analyst the BigQuery Data Editor role at the project level.
C. Give analysts the BigQuery Data Viewer role on the shared dataset. Create a dataset for each analyst, and give each analyst the BigQuery Data Editor role at the dataset level for their assigned dataset.
D. Give analysts the BigQuery Data Viewer role on the shared dataset. Create one other dataset and give the analysts the BigQuery Data Editor role on that dataset.
Show Answer
Correct Answer: C
Explanation: Grant read-only access only to the shared dataset using the BigQuery Data Viewer role. Create a separate dataset for each analyst and grant each analyst the BigQuery Data Editor role only on their own dataset. This provides private workspaces while preventing modification of the shared dataset and preventing access to other analysts' personal datasets.
Question 70
Your car factory is pushing machine measurements as messages into a Pub/Sub topic in your Google Cloud project. A Dataflow streaming job, that you wrote with the Apache Beam SDK, reads these messages, sends acknowledgment to Pub/Sub, applies some custom business logic in a DoFn instance, and writes the result to BigQuery. You want to ensure that if your business logic fails on a message, the message will be sent to a Pub/Sub topic that you want to monitor for alerting purposes. What should you do?
A. Enable retaining of acknowledged messages in your Pub/Sub pull subscription. Use Cloud Monitoring to monitor the subscription/num_retained_acked_messages metric on this subscription.
B. Use an exception handling block in your Dataflow’s DoFn code to push the messages that failed to be transformed through a side output and to a new Pub/Sub topic. Use Cloud Monitoring to monitor the topic/num_unacked_messages_by_region metric on this new topic.
C. Enable dead lettering in your Pub/Sub pull subscription, and specify a new Pub/Sub topic as the dead letter topic. Use Cloud Monitoring to monitor the subscription/dead_letter_message_count metric on your pull subscription.
D. Create a snapshot of your Pub/Sub pull subscription. Use Cloud Monitoring to monitor the snapshot/num_messages metric on this snapshot.
Show Answer
Correct Answer: B
Explanation: Dataflow acknowledges Pub/Sub messages independently of your downstream business logic. If a DoFn throws after the message has been acknowledged, Pub/Sub dead-lettering will not capture it because dead-letter topics only apply to messages that are repeatedly not acknowledged. The recommended pattern is to catch exceptions in the DoFn, emit failed records via a side output, write them to a dedicated Pub/Sub topic, and monitor that topic for alerting.
Question 71
You are designing a real-time system for a ride hailing app that identifies areas with high demand for rides to effectively reroute available drivers to meet the demand. The system ingests data from multiple sources to Pub/Sub, processes the data, and stores the results for visualization and analysis in real-time dashboards. The data sources include driver location updates every 5 seconds and app-based booking events from riders. The data processing involves real-time aggregation of supply and demand data for the last 30 seconds, every 2 seconds, and storing the results in a low-latency system for visualization. What should you do?
A. Group the data by using a tumbling window in a Dataflow pipeline, and write the aggregated data to Memorystore.
B. Group the data by using a hopping window in a Dataflow pipeline, and write the aggregated data to Memorystore.
C. Group the data by using a session window in a Dataflow pipeline, and write the aggregated data to BigQuery.
D. Group the data by using a hopping window in a Dataflow pipeline, and write the aggregated data to BigQuery.
Show Answer
Correct Answer: B
Explanation: The requirement is to compute aggregates over the last 30 seconds every 2 seconds, which requires overlapping windows (a hopping/sliding window), not tumbling or session windows. For storing results used by real-time dashboards with low-latency access, Memorystore is more appropriate than BigQuery, which is an analytical data warehouse rather than a low-latency serving layer.
Question 72
You are designing a Dataflow pipeline for a batch processing job. You want to mitigate multiple zonal failures at job submission time. What should you do?
A. Submit duplicate pipelines in two different zones by using the --zone flag.
B. Set the pipeline staging location as a regional Cloud Storage bucket.
C. Specify a worker region by using the --region flag.
D. Create an Eventarc trigger to resubmit the job in case of zonal failure when submitting the job.
Show Answer
Correct Answer: C
Explanation: To mitigate multiple zonal failures at job submission time, submit the Dataflow job using a worker region rather than pinning it to a single zone. Using the --region flag allows Dataflow to place workers across available zones within the region and improves resilience to zonal failures. Using --zone ties execution to one zone, a regional Cloud Storage bucket only affects staging storage, and Eventarc resubmission is not the recommended mitigation for submission-time zonal failures.
$19
Get all 332 questions with detailed answers and explanations
Instant download HTML + PDF delivered the moment payment clears.
Secure Stripe checkout we never see or store your card details.
7-day refund if files are defective see our refund policy.