Professional Data Engineer Free Practice Questions — Page 15
Question 123
You are designing a cloud-native historical data processing system to meet the following conditions:
✑ The data being analyzed is in CSV, Avro, and PDF formats and will be accessed by multiple analysis tools including Dataproc, BigQuery, and Compute
Engine.
✑ A batch pipeline moves daily data.
✑ Performance is not a factor in the solution.
✑ The solution design should maximize availability.
How should you design data storage for this solution?
A. Create a Dataproc cluster with high availability. Store the data in HDFS, and perform analysis as needed.
B. Store the data in BigQuery. Access the data using the BigQuery Connector on Dataproc and Compute Engine.
C. Store the data in a regional Cloud Storage bucket. Access the bucket directly using Dataproc, BigQuery, and Compute Engine.
D. Store the data in a multi-regional Cloud Storage bucket. Access the data directly using Dataproc, BigQuery, and Compute Engine.
Show Answer
Correct Answer: D
Explanation: The data must support multiple file formats (CSV, Avro, PDF) and be accessible by Dataproc, BigQuery, and Compute Engine, which favors object storage over HDFS or a database-only solution. Performance is not a concern, but availability should be maximized. Cloud Storage provides native integration with all required services and supports all file types. A multi-regional bucket offers higher availability and durability than a regional bucket, making it the best fit. HDFS reduces availability and portability, and BigQuery alone is unsuitable for storing PDFs.
Question 124
An online retailer has built their current application on Google App Engine. A new initiative at the company mandates that they extend their application to allow their customers to transact directly via the application. They need to manage their shopping transactions and analyze combined data from multiple datasets using a business intelligence (BI) tool. They want to use only a single database for this purpose. Which Google Cloud database should they choose?
A. BigQuery
B. Cloud SQL
C. Cloud BigTable
D. Cloud Datastore
Show Answer
Correct Answer: B
Explanation: The retailer needs a single database that can both handle transactional workloads (shopping transactions with ACID guarantees) and support analysis via a BI tool. Cloud SQL is a fully managed relational database designed for OLTP transactions and can also be queried directly by BI tools for reporting and analysis. BigQuery excels at analytics but is not suitable as the primary transactional database for an application. Cloud Bigtable and Cloud Datastore lack strong relational transactions and are not ideal for BI-style analytics. Therefore, Cloud SQL best satisfies all requirements.
Question 125
Your company is selecting a system to centralize data ingestion and delivery. You are considering messaging and data integration systems to address the requirements. The key requirements are:
✑ The ability to seek to a particular offset in a topic, possibly back to the start of all data ever captured
✑ Support for publish/subscribe semantics on hundreds of topics
Retain per-key ordering -
Which system should you choose?
A. Apache Kafka
B. Cloud Storage
C. Dataflow
D. Firebase Cloud Messaging
Show Answer
Correct Answer: A
Explanation: Apache Kafka satisfies all the stated requirements: it allows consumers to seek to any offset in a topic (including replaying from the beginning), supports publish/subscribe semantics across hundreds or thousands of topics, and guarantees per-key ordering by routing records with the same key to the same partition. None of the other options provide offset-based replay with strong per-key ordering and large-scale pub/sub semantics.
Question 126
You need to create a new transaction table in Cloud Spanner that stores product sales data. You are deciding what to use as a primary key. From a performance perspective, which strategy should you choose?
A. The current epoch time
B. A concatenation of the product name and the current epoch time
C. A random universally unique identifier number (version 4 UUID)
D. The original order identification number from the sales system, which is a monotonically increasing integer
Show Answer
Correct Answer: C
Explanation: From a performance perspective in Cloud Spanner, the primary goal is to avoid write hotspots by ensuring even key distribution across splits. Monotonically increasing keys such as epoch time (A) or an increasing order ID (D) concentrate writes on a small key range and hurt scalability. A concatenation with product name and time (B) can still skew distribution if some products are much more popular. A random version 4 UUID distributes writes uniformly across key ranges, which aligns with Cloud Spanner schema design best practices and delivers the best write performance.
Question 127
A shipping company has live package-tracking data that is sent to an Apache Kafka stream in real time. This is then loaded into BigQuery. Analysts in your company want to query the tracking data in BigQuery to analyze geospatial trends in the lifecycle of a package. The table was originally created with ingest-date partitioning. Over time, the query processing time has increased. You need to implement a change that would improve query performance in BigQuery. What should you do?
A. Implement clustering in BigQuery on the ingest date column.
B. Implement clustering in BigQuery on the package-tracking ID column.
C. Tier older data onto Cloud Storage files and create a BigQuery table using Cloud Storage as an external data source.
D. Re-create the table using data partitioning on the package delivery date.
Show Answer
Correct Answer: B
Explanation: Query performance has degraded as data volume grows while the table is already partitioned by ingest date. Analysts analyzing a package’s lifecycle and geospatial path typically filter or group by package-tracking ID across many ingest-date partitions. Adding clustering on the package-tracking ID allows BigQuery to colocate rows for the same package within each partition and skip irrelevant blocks, significantly reducing scanned data. Repartitioning by delivery date is less effective because lifecycle events span multiple days and delivery date may be NULL or only known at the end.
Question 128
You work for a large financial institution that is planning to use Dialogflow to create a chatbot for the company's mobile app. You have reviewed old chat logs and tagged each conversation for intent based on each customer's stated intention for contacting customer service. About 70% of customer requests are simple requests that are solved within 10 intents. The remaining 30% of inquiries require much longer, more complicated requests. Which intents should you automate first?
A. Automate the 10 intents that cover 70% of the requests so that live agents can handle more complicated requests.
B. Automate the more complicated requests first because those require more of the agents' time.
C. Automate a blend of the shortest and longest intents to be representative of all intents.
D. Automate intents in places where common words such as 'payment' appear only once so the software isn't confused.
Show Answer
Correct Answer: A
Explanation: Best practice is to automate the most frequent, simple intents first. Automating the 10 intents that cover 70% of requests delivers the greatest immediate impact, reduces agent load, and lets human agents focus on the more complex 30% of cases.
Question 129
You need to give new website users a globally unique identifier (GUID) using a service that takes in data points and returns a GUID. This data is sourced from both internal and external systems via HTTP calls that you will make via microservices within your pipeline. There will be tens of thousands of messages per second and that can be multi-threaded. and you worry about the backpressure on the system. How should you design your pipeline to minimize that backpressure?
A. Call out to the service via HTTP.
B. Create the pipeline statically in the class definition.
C. Create a new object in the startBundle method of DoFn.
D. Batch the job into ten-second increments.
Show Answer
Correct Answer: D
Explanation: The main source of backpressure is making a slow external HTTP call per element at very high throughput. With tens of thousands of messages per second, even modest call latency would overwhelm the pipeline. Batching elements into time-based windows (for example, 10-second increments) reduces the number of outbound service calls, smooths load on the external GUID service, and allows the pipeline to process data more efficiently. Options B and C address object initialization efficiency, not backpressure, and A worsens the problem by calling the service per element.
Question 130
You have Cloud Functions written in Node.js that pull messages from Cloud Pub/Sub and send the data to BigQuery. You observe that the message processing rate on the Pub/Sub topic is orders of magnitude higher than anticipated, but there is no error logged in Cloud Logging. What are the two most likely causes of this problem? (Choose two.)
A. Publisher throughput quota is too small.
B. Total outstanding messages exceed the 10-MB maximum.
C. Error handling in the subscriber code is not handling run-time errors properly.
D. The subscriber code cannot keep up with the messages.
E. The subscriber code does not acknowledge the messages that it pulls.
Show Answer
Correct Answer: C, E
Explanation: A much higher-than-expected processing rate without errors usually indicates message redelivery rather than true throughput. If the subscriber code does not acknowledge messages (E), Pub/Sub will repeatedly redeliver them, inflating the observed processing rate. This commonly happens when acknowledgments are never sent or are skipped on failure paths. In addition, improper error handling in the Cloud Function (C) can cause runtime failures or early exits where messages are neither successfully processed nor acknowledged, yet no errors are logged explicitly, leading again to repeated redelivery and an apparently excessive processing rate.
Question 131
You work for a shipping company that uses handheld scanners to read shipping labels. Your company has strict data privacy standards that require scanners to only transmit tracking numbers when events are sent to Kafka topics. A recent software update caused the scanners to accidentally transmit recipients' personally identifiable information (PII) to analytics systems, which violates user privacy rules. You want to quickly build a scalable solution using cloud-native managed services to prevent exposure of PII to the analytics systems. What should you do?
A. Create an authorized view in BigQuery to restrict access to tables with sensitive data.
B. Install a third-party data validation tool on Compute Engine virtual machines to check the incoming data for sensitive information.
C. Use Cloud Logging to analyze the data passed through the total pipeline to identify transactions that may contain sensitive information.
D. Build a Cloud Function that reads the topics and makes a call to the Cloud Data Loss Prevention (Cloud DLP) API. Use the tagging and confidence levels to either pass or quarantine the data in a bucket for review.
Show Answer
Correct Answer: D
Explanation: The requirement is to quickly prevent PII from reaching analytics systems using scalable, managed cloud-native services. Using a Cloud Function subscribed to the Kafka (or bridged) topics to invoke Cloud Data Loss Prevention (DLP) directly inspects message payloads for PII. Based on DLP findings and confidence levels, the function can forward only compliant tracking numbers and quarantine or divert sensitive records for review. This directly enforces privacy at ingestion, scales automatically, and avoids exposing PII downstream. Other options either act too late (BigQuery views, Logging) or require unmanaged infrastructure (third-party tools on VMs).
Question 132
You are managing a Cloud Dataproc cluster. You need to make a job run faster while minimizing costs, without losing work in progress on your clusters. What should you do?
A. Increase the cluster size with more non-preemptible workers.
B. Increase the cluster size with preemptible worker nodes, and configure them to forcefully decommission.
C. Increase the cluster size with preemptible worker nodes, and use Cloud Stackdriver to trigger a script to preserve work.
D. Increase the cluster size with preemptible worker nodes, and configure them to use graceful decommissioning.
Show Answer
Correct Answer: D
Explanation: To speed up jobs while minimizing cost, you should scale out using preemptible workers, which are cheaper than non-preemptible nodes. To avoid losing work in progress when nodes are removed or preempted, configure the cluster to use graceful decommissioning so running tasks can finish and data is safely migrated. This balances performance, cost efficiency, and workload safety.
$19
Get all 328 questions with detailed answers and explanations
Instant download HTML + PDF delivered the moment payment clears.
Secure Stripe checkout we never see or store your card details.
7-day refund if files are defective see our refund policy.