Professional Data Engineer Free Practice Questions — Page 16
Question 150
You need to give new website users a globally unique identifier (GUID) using a service that takes in data points and returns a GUID. This data is sourced from both internal and external systems via HTTP calls that you will make via microservices within your pipeline. There will be tens of thousands of messages per second and that can be multi-threaded. and you worry about the backpressure on the system. How should you design your pipeline to minimize that backpressure?
A. Call out to the service via HTTP.
B. Create the pipeline statically in the class definition.
C. Create a new object in the startBundle method of DoFn.
D. Batch the job into ten-second increments.
Show Answer
Correct Answer: D
Explanation: The key requirement is minimizing backpressure when processing tens of thousands of messages per second while making external HTTP calls. Per-element HTTP requests can overwhelm the downstream service and stall the pipeline. Batching requests reduces the number of outbound calls and amortizes latency, which is the recommended pattern for high-throughput enrichment workloads. Options B and C address client initialization efficiency, not backpressure, and A would maximize backpressure by issuing one HTTP request per element.
Question 151
You have Cloud Functions written in Node.js that pull messages from Cloud Pub/Sub and send the data to BigQuery. You observe that the message processing rate on the Pub/Sub topic is orders of magnitude higher than anticipated, but there is no error logged in Cloud Logging. What are the two most likely causes of this problem? (Choose two.)
A. Publisher throughput quota is too small.
B. Total outstanding messages exceed the 10-MB maximum.
C. Error handling in the subscriber code is not handling run-time errors properly.
D. The subscriber code cannot keep up with the messages.
E. The subscriber code does not acknowledge the messages that it pulls.
Show Answer
Correct Answer: D, E
Explanation: A subscriber that cannot keep up with incoming messages will build a backlog, and failure to acknowledge messages causes Pub/Sub to redeliver them repeatedly, inflating the observed processing rate without necessarily producing application errors in Cloud Logging. Publisher quota and the 10 MB outstanding-message limit do not fit the described symptoms, and missing runtime error handling alone does not directly explain a much higher processing rate.
Question 152
You work for a shipping company that uses handheld scanners to read shipping labels. Your company has strict data privacy standards that require scanners to only transmit tracking numbers when events are sent to Kafka topics. A recent software update caused the scanners to accidentally transmit recipients' personally identifiable information (PII) to analytics systems, which violates user privacy rules. You want to quickly build a scalable solution using cloud-native managed services to prevent exposure of PII to the analytics systems. What should you do?
A. Create an authorized view in BigQuery to restrict access to tables with sensitive data.
B. Install a third-party data validation tool on Compute Engine virtual machines to check the incoming data for sensitive information.
C. Use Cloud Logging to analyze the data passed through the total pipeline to identify transactions that may contain sensitive information.
D. Build a Cloud Function that reads the topics and makes a call to the Cloud Data Loss Prevention (Cloud DLP) API. Use the tagging and confidence levels to either pass or quarantine the data in a bucket for review.
Show Answer
Correct Answer: D
Explanation: Cloud DLP is the managed Google Cloud service designed to inspect, classify, and de-identify sensitive data such as PII. A Cloud Function consuming Kafka topic data (or the ingestion stream) can invoke the Cloud DLP API, using inspection results and confidence scores to route clean records onward while quarantining records containing PII. The other options either restrict downstream access (BigQuery views), only detect issues after the fact (Cloud Logging), or rely on unmanaged third-party infrastructure rather than a scalable cloud-native managed solution.
Question 153
You are managing a Cloud Dataproc cluster. You need to make a job run faster while minimizing costs, without losing work in progress on your clusters. What should you do?
A. Increase the cluster size with more non-preemptible workers.
B. Increase the cluster size with preemptible worker nodes, and configure them to forcefully decommission.
C. Increase the cluster size with preemptible worker nodes, and use Cloud Stackdriver to trigger a script to preserve work.
D. Increase the cluster size with preemptible worker nodes, and configure them to use graceful decommissioning.
Show Answer
Correct Answer: D
Explanation: To speed up Dataproc jobs while minimizing costs, add preemptible workers because they are significantly cheaper than regular workers. To avoid losing work in progress when workers are removed or the cluster is scaled down, use graceful decommissioning so running tasks can complete before nodes are decommissioned. Forceful decommissioning risks losing work, and Stackdriver-triggered scripts are unnecessary for this built-in capability.
Question 154
You have a data stored in BigQuery. The data in the BigQuery dataset must be highly available. You need to define a storage, backup, and recovery strategy of this data that minimizes cost. How should you configure the BigQuery table that have a recovery point objective (RPO) of 30 days?
A. Set the BigQuery dataset to be regional. In the event of an emergency, use a point-in-time snapshot to recover the data.
B. Set the BigQuery dataset to be regional. Create a scheduled query to make copies of the data to tables suffixed with the time of the backup. In the event of an emergency, use the backup copy of the table.
C. Set the BigQuery dataset to be multi-regional. In the event of an emergency, use a point-in-time snapshot to recover the data.
D. Set the BigQuery dataset to be multi-regional. Create a scheduled query to make copies of the data to tables suffixed with the time of the backup. In the event of an emergency, use the backup copy of the table.
Show Answer
Correct Answer: B
Explanation: A 30-day recovery objective cannot be met with BigQuery time travel/point-in-time recovery alone because its retention window is limited (historically up to 7 days by default in exam context). To retain recoverable copies for 30 days, schedule recurring copies/backups. To minimize cost, use a regional dataset rather than a multi-regional dataset, since the question does not require cross-region disaster recovery.
Question 155
You're using Bigtable for a real-time application, and you have a heavy load that is a mix of read and writes. You've recently identified an additional use case and need to perform hourly an analytical job to calculate certain statistics across the whole database. You need to ensure both the reliability of your production application as well as the analytical workload.
What should you do?
A. Export Bigtable dump to GCS and run your analytical job on top of the exported files.
B. Add a second cluster to an existing instance with a multi-cluster routing, use live-traffic app profile for your regular workload and batch-analytics profile for the analytics workload.
C. Add a second cluster to an existing instance with a single-cluster routing, use live-traffic app profile for your regular workload and batch-analytics profile for the analytics workload.
D. Increase the size of your existing cluster twice and execute your analytics workload on your new resized cluster.
Show Answer
Correct Answer: C
Explanation: Use replication with a second cluster and separate app profiles using single-cluster routing so the production application's live traffic is pinned to one cluster while the batch analytics workload is pinned to the other. This isolates the heavy analytical reads from the latency-sensitive mixed read/write workload. Multi-cluster routing is intended for automatic failover/load routing rather than workload isolation between dedicated clusters.
Question 156
Your team is responsible for developing and maintaining ETLs in your company. One of your Dataflow jobs is failing because of some errors in the input data, and you need to improve reliability of the pipeline (incl. being able to reprocess all failing data).
What should you do?
A. Add a filtering step to skip these types of errors in the future, extract erroneous rows from logs.
B. Add a tryג€¦ catch block to your DoFn that transforms the data, extract erroneous rows from logs.
C. Add a tryג€¦ catch block to your DoFn that transforms the data, write erroneous rows to Pub/Sub PubSub directly from the DoFn.
D. Add a tryג€¦ catch block to your DoFn that transforms the data, use a sideOutput to create a PCollection that can be stored to Pub/Sub later.
Show Answer
Correct Answer: D
Explanation: The recommended Apache Beam/Dataflow pattern is to catch transformation errors in a DoFn and emit failed records to a side output (dead-letter PCollection). That PCollection can then be written using the appropriate IO connector (such as PubSubIO), preserving pipeline reliability and enabling later reprocessing. Writing directly to Pub/Sub from within a DoFn is discouraged because IO should generally be handled by Beam IO transforms rather than embedded in user code. Filtering out bad records loses recoverability, and extracting rows from logs is not a robust reprocessing strategy.
Question 157
An organization maintains a Google BigQuery dataset that contains tables with user-level data. They want to expose aggregates of this data to other Google
Cloud projects, while still controlling access to the user-level data. Additionally, they need to minimize their overall storage cost and ensure the analysis cost for other projects is assigned to those projects. What should they do?
A. Create and share an authorized view that provides the aggregate results.
B. Create and share a new dataset and view that provides the aggregate results.
C. Create and share a new dataset and table that contains the aggregate results.
D. Create dataViewer Identity and Access Management (IAM) roles on the dataset to enable sharing.
Show Answer
Correct Answer: A
Explanation: An authorized view is designed to expose only a filtered or aggregated result while protecting access to the underlying user-level tables. It avoids duplicating data, minimizing storage costs. Query processing charges are incurred by the project that runs the query against the view, satisfying the requirement that analysis costs be assigned to the consuming projects. Granting dataset-level Data Viewer access would expose the base data, and creating a separate aggregate table would increase storage costs. While Google recommends placing authorized views in a separate dataset, that is a best practice rather than an inherent requirement of using authorized views.
Question 158
You are deploying a new storage system for your mobile application, which is a media streaming service. You decide the best fit is Google Cloud Datastore. You have entities with multiple properties, some of which can take on multiple values. For example, in the entity 'Movie' the property 'actors' and the property
'tags' have multiple values but the property 'date released' does not. A typical query would ask for all movies with actor=<actorname> ordered by date_released or all movies with tag=Comedy ordered by date_released. How should you avoid a combinatorial explosion in the number of indexes?
A. Manually configure the index in your index config as follows:
B. Manually configure the index in your index config as follows:
C. Set the following in your entity options: exclude_from_indexes = 'actors, tags'
D. Set the following in your entity options: exclude_from_indexes = 'date_published'
Show Answer
Correct Answer: A
Explanation: The correct approach is to manually define separate composite indexes for each multivalued property with the single-valued sort property (for example, actors + date_released and tags + date_released). This avoids creating a composite index containing multiple multivalued properties, which causes combinatorial index explosion. Excluding actors/tags would prevent the required queries from being indexed, and excluding the sort property would prevent the required ordered queries.
Question 159
You are migrating a table to BigQuery and are deciding on the data model. Your table stores information related to purchases made across several store locations and includes information like the time of the transaction, items purchased, the store ID, and the city and state in which the store is located. You frequently query this table to see how many of each item were sold over the past 30 days and to look at purchasing trends by state, city, and individual store. How would you model this table for the best query performance?
A. Partition by transaction time; cluster by state first, then city, then store ID.
B. Partition by transaction time; cluster by store ID first, then city, then state.
C. Top-level cluster by state first, then city, then store ID.
D. Top-level cluster by store ID first, then city, then state.
Show Answer
Correct Answer: A
Explanation: Partition the table by transaction time so queries for the last 30 days prune partitions efficiently. Then cluster by the most common geographic hierarchy used in filters: state, then city, then store ID. BigQuery clustering is most effective when queries filter on the leftmost clustered columns, so ordering the hierarchy from broader to narrower supports queries by state, by state+city, and by state+city+store. Top-level clustering alone would not optimize the frequent time-range queries.
$19
Get all 332 questions with detailed answers and explanations
Instant download HTML + PDF delivered the moment payment clears.
Secure Stripe checkout we never see or store your card details.
7-day refund if files are defective see our refund policy.