Professional Data Engineer Free Practice Questions — Page 17
Question 160
You are designing a basket abandonment system for an ecommerce company. The system will send a message to a user based on these rules:
✑ No interaction by the user on the site for 1 hour
Has added more than $30 worth of products to the basket
✑ Has not completed a transaction
You use Google Cloud Dataflow to process the data and decide if a message should be sent. How should you design the pipeline?
A. Use a fixed-time window with a duration of 60 minutes.
B. Use a sliding time window with a duration of 60 minutes.
C. Use a session window with a gap time duration of 60 minutes.
D. Use a global window with a time based trigger with a delay of 60 minutes.
Show Answer
Correct Answer: C
Explanation: A session window with a 60-minute gap groups events by user activity and closes after 60 minutes of inactivity. That directly matches the requirement to detect basket abandonment after one hour of no interaction while evaluating the accumulated basket value and absence of a completed transaction.
Question 161
You work for an advertising company, and you've developed a Spark ML model to predict click-through rates at advertisement blocks. You've been developing everything at your on-premises data center, and now your company is migrating to Google Cloud. Your data center will be closing soon, so a rapid lift-and-shift migration is necessary. However, the data you've been using will be migrated to migrated to BigQuery. You periodically retrain your Spark ML models, so you need to migrate existing training pipelines to Google Cloud. What should you do?
A. Use Vertex AI for training existing Spark ML models
B. Rewrite your models on TensorFlow, and start using Vertex AI
C. Use Dataproc for training existing Spark ML models, but start reading data directly from BigQuery
D. Spin up a Spark cluster on Compute Engine, and train Spark ML models on the data exported from BigQuery
Show Answer
Correct Answer: C
Explanation: For a rapid lift-and-shift migration, keep the existing Spark ML training pipelines and run them on Dataproc, which is the managed Spark service on Google Cloud. Since the data is migrating to BigQuery, use the Spark-BigQuery connector to read training data directly from BigQuery. This minimizes code changes while using managed infrastructure. Rewriting to TensorFlow is unnecessary, Vertex AI is not the most direct lift-and-shift path for existing Spark ML pipelines, and managing your own Spark cluster on Compute Engine adds operational overhead.
Question 162
Your company is using WILDCARD tables to query data across multiple tables with similar names. The SQL statement is currently failing with the following error:
Which table name will make the SQL statement work correctly?
A. 'bigquery-public-data.noaa_gsod.gsod'
B. bigquery-public-data.noaa_gsod.gsod*
C. 'bigquery-public-data.noaa_gsod.gsod'*
D. 'bigquery-public-data.noaa_gsod.gsod*`
Show Answer
Correct Answer: D
Explanation: BigQuery wildcard table references must be enclosed in backticks because the wildcard character (*) is part of the table identifier. The options appear to contain a typography error using single quotes instead of backticks. Interpreting D as using backticks (`bigquery-public-data.noaa_gsod.gsod*`) makes it the correct syntax. Option B without backticks is not valid for this wildcard table reference.
Question 163
You are operating a streaming Cloud Dataflow pipeline. Your engineers have a new version of the pipeline with a different windowing algorithm and triggering strategy. You want to update the running pipeline with the new version. You want to ensure that no data is lost during the update. What should you do?
A. Update the Cloud Dataflow pipeline inflight by passing the --update option with the --jobName set to the existing job name
B. Update the Cloud Dataflow pipeline inflight by passing the --update option with the --jobName set to a new unique job name
C. Stop the Cloud Dataflow pipeline with the Cancel option. Create a new Cloud Dataflow job with the updated code
D. Stop the Cloud Dataflow pipeline with the Drain option. Create a new Cloud Dataflow job with the updated code
Show Answer
Correct Answer: D
Explanation: Changing the windowing algorithm and triggering strategy is a major semantic change. Dataflow in-place updates are intended for compatible changes and major windowing/trigger changes are not recommended because state compatibility and output semantics can be unpredictable. To avoid losing in-flight data, gracefully drain the existing streaming job so buffered and in-flight elements are processed, then start a new job with the updated pipeline. Cancel would discard in-flight work, and updating in place is not appropriate for this kind of change.
Question 164
You create an important report for your large team in Google Data Studio 360. The report uses Google BigQuery as its data source. You notice that visualizations are not showing data that is less than 1 hour old. What should you do?
A. Disable caching by editing the report settings.
B. Disable caching in BigQuery by editing table details.
C. Refresh your browser tab showing the visualizations.
D. Clear your browser history for the past hour then reload the tab showing the virtualizations.
Show Answer
Correct Answer: A
Explanation: Looker Studio (formerly Data Studio) caches query results, including for BigQuery, to improve performance. If reports are not showing data newer than about an hour, the relevant fix is to disable or adjust report caching rather than changing BigQuery table settings or refreshing/clearing the browser cache. Browser refresh does not bypass the report's data cache.
Question 165
You are building a new data pipeline to share data between two different types of applications: jobs generators and job runners. Your solution must scale to accommodate increases in usage and must accommodate the addition of new applications without negatively affecting the performance of existing ones. What should you do?
A. Create an API using App Engine to receive and send messages to the applications
B. Use a Cloud Pub/Sub topic to publish jobs, and use subscriptions to execute them
C. Create a table on Cloud SQL, and insert and delete rows with the job information
D. Create a table on Cloud Spanner, and insert and delete rows with the job information
Show Answer
Correct Answer: B
Explanation: Cloud Pub/Sub is designed for decoupled, scalable, asynchronous messaging. Job generators publish messages to a topic, and job runners consume them through subscriptions. This architecture scales automatically and allows new consumer applications to be added via additional subscriptions without impacting existing publishers or subscribers.
Question 166
You have several Spark jobs that run on a Cloud Dataproc cluster on a schedule. Some of the jobs run in sequence, and some of the jobs run concurrently. You need to automate this process. What should you do?
A. Create a Cloud Dataproc Workflow Template
B. Create an initialization action to execute the jobs
C. Create a Directed Acyclic Graph in Cloud Composer
D. Create a Bash script that uses the Cloud SDK to create a cluster, execute jobs, and then tear down the cluster
Show Answer
Correct Answer: C
Explanation: Cloud Composer (managed Apache Airflow) is designed to orchestrate scheduled workflows with both sequential and parallel task execution using DAGs. Dataproc Workflow Templates can define Dataproc job workflows but do not provide native scheduling, so for scheduled orchestration with dependencies and concurrent execution, Composer is the best fit.
Question 167
You have a data pipeline with a Dataflow job that aggregates and writes time series metrics to Bigtable. You notice that data is slow to update in Bigtable. This data feeds a dashboard used by thousands of users across the organization. You need to support additional concurrent users and reduce the amount of time required to write the data. Which two actions should you take? (Choose two.)
A. Configure your Dataflow pipeline to use local execution
B. Increase the maximum number of Dataflow workers by setting maxNumWorkers in PipelineOptions
C. Increase the number of nodes in the Bigtable cluster
D. Modify your Dataflow pipeline to use the Flatten transform before writing to Bigtable
E. Modify your Dataflow pipeline to use the CoGroupByKey transform before writing to Bigtable
Show Answer
Correct Answer: B, C
Explanation: Increasing the maximum number of Dataflow workers allows the pipeline to scale out processing and parallelize writes. Increasing the number of Bigtable nodes increases write throughput and supports more concurrent read/write traffic. Local execution is for development, not production scaling. Flatten and CoGroupByKey are data transforms and do not inherently improve Bigtable write throughput for this scenario.
Question 168
Your company is in the process of migrating its on-premises data warehousing solutions to BigQuery. The existing data warehouse uses trigger-based change data capture (CDC) to apply updates from multiple transactional database sources on a daily basis. With BigQuery, your company hopes to improve its handling of
CDC so that changes to the source systems are available to query in BigQuery in near-real time using log-based CDC streams, while also optimizing for the performance of applying changes to the data warehouse. Which two steps should they take to ensure that changes are available in the BigQuery reporting table with minimal latency while reducing compute overhead? (Choose two.)
A. Perform a DML INSERT, UPDATE, or DELETE to replicate each individual CDC record in real time directly on the reporting table.
B. Insert each new CDC record and corresponding operation type to a staging table in real time.
C. Periodically DELETE outdated records from the reporting table.
D. Periodically use a DML MERGE to perform several DML INSERT, UPDATE, and DELETE operations at the same time on the reporting table.
E. Insert each new CDC record and corresponding operation type in real time to the reporting table, and use a materialized view to expose only the newest version of each unique record.
Show Answer
Correct Answer: B, D
Explanation: To minimize compute overhead in BigQuery, stream CDC events into a staging (delta) table rather than issuing per-row DML against the reporting table. Then periodically apply accumulated changes with a single MERGE statement, which batches INSERT, UPDATE, and DELETE operations efficiently. This is the recommended BigQuery CDC pattern for balancing near-real-time ingestion with efficient application of changes.
Question 169
Your platform on your on-premises environment generates 100 GB of data daily, composed of millions of structured JSON text files. Your on-premises environment cannot be accessed from the public internet. You want to use Google Cloud products to query and explore the platform data. What should you do?
A. Use Cloud Scheduler to copy data daily from your on-premises environment to Cloud Storage. Use the BigQuery Data Transfer Service to import data into BigQuery.
B. Use a Transfer Appliance to copy data from your on-premises environment to Cloud Storage. Use the BigQuery Data Transfer Service to import data into BigQuery.
C. Use Transfer Service for on-premises data to copy data from your on-premises environment to Cloud Storage. Use the BigQuery Data Transfer Service to import data into BigQuery.
D. Use the BigQuery Data Transfer Service dataset copy to transfer all data into BigQuery.
Show Answer
Correct Answer: C
Explanation: The best choice is to use Storage Transfer Service for on-premises data to continuously and securely transfer the daily on-premises JSON files into Cloud Storage, then load them into BigQuery. Transfer Appliance is intended for one-time or bulk migrations rather than ongoing daily ingestion. Cloud Scheduler does not perform file transfers, and BigQuery Data Transfer Service cannot directly ingest data from an on-premises filesystem.
$19
Get all 332 questions with detailed answers and explanations
Instant download HTML + PDF delivered the moment payment clears.
Secure Stripe checkout we never see or store your card details.
7-day refund if files are defective see our refund policy.