Professional Data Engineer Free Practice Questions — Page 7
Question 43
You are administering shared BigQuery datasets that contain views used by multiple teams in your organization. The marketing team is concerned about the variability of their monthly BigQuery analytics spend using the on-demand billing model. You need to help the marketing team establish a consistent BigQuery analytics spend each month. What should you do?
A. Create a BigQuery Enterprise reservation with a baseline of 250 slots and autoscaling set to 500 for the marketing team, and bill them back accordingly.
B. Establish a BigQuery quota for the marketing team, and limit the maximum number of bytes scanned each day.
C. Create a BigQuery reservation with a baseline of 500 slots with no autoscaling for the marketing team, and bill them back accordingly.
D. Create a BigQuery Standard pay-as-you go reservation with a baseline of 0 slots and autoscaling set to 500 for the marketing team, and bill them back accordingly.
Show Answer
Correct Answer: C
Explanation: To achieve a consistent and predictable monthly BigQuery spend, the team should move from on-demand (bytes scanned) pricing to capacity-based pricing with a fixed reservation. A reservation with a baseline of 500 slots and no autoscaling provides a fixed monthly cost regardless of query volume. Autoscaling (A or D) would reintroduce variability, and quotas on bytes scanned (B) can block queries without guaranteeing predictable costs. Therefore, a fixed-size reservation without autoscaling best meets the requirement.
Question 44
You have thousands of Apache Spark jobs running in your on-premises Apache Hadoop cluster. You want to migrate the jobs to Google Cloud. You want to use managed services to run your jobs instead of maintaining a long-lived Hadoop cluster yourself. You have a tight timeline and want to keep code changes to a minimum. What should you do?
A. Move your data to BigQuery. Convert your Spark scripts to a SQL-based processing approach.
B. Rewrite your jobs in Apache Beam. Run your jobs in Dataflow.
C. Copy your data to Compute Engine disks. Manage and run your jobs directly on those instances.
D. Move your data to Cloud Storage. Run your jobs on Dataproc.
Show Answer
Correct Answer: D
Explanation: Dataproc is a fully managed service for running Apache Spark and Hadoop with minimal code changes. Moving data to Cloud Storage and running existing Spark jobs on Dataproc avoids maintaining clusters, fits a tight migration timeline, and preserves current job logic, unlike rewriting for BigQuery or Dataflow or self-managing Compute Engine.
Question 45
You have a network of 1000 sensors. The sensors generate time series data: one metric per sensor per second, along with a timestamp. You already have 1 TB of data, and expect the data to grow by 1 GB every day. You need to access this data in two ways. The first access pattern requires retrieving the metric from one specific sensor stored at a specific timestamp, with a median single-digit millisecond latency. The second access pattern requires running complex analytic queries on the data, including joins, once a day. How should you store this data?
A. Store your data in BigQuery. Concatenate the sensor ID and timestamp, and use it as the primary key.
B. Store your data in Bigtable. Concatenate the sensor ID and timestamp and use it as the row key. Perform an export to BigQuery every day.
C. Store your data in Bigtable. Concatenate the sensor ID and metric, and use it as the row key. Perform an export to BigQuery every day.
D. Store your data in BigQuery. Use the metric as a primary key.
Show Answer
Correct Answer: B
Explanation: The workload has two distinct access patterns: (1) very low-latency point lookups by sensor and timestamp, and (2) daily complex analytical queries with joins. Bigtable is optimized for massive time-series data and single-row lookups with single-digit millisecond latency when using a well-designed row key such as sensor ID + timestamp. BigQuery, while excellent for analytics, is not suitable for low-latency point reads. Exporting the data daily from Bigtable to BigQuery cleanly supports the analytical workload. The other options either misuse BigQuery for low-latency lookups or use an inefficient row-key design.
Question 46
You work for a large ecommerce company. You store your customer's order data in Bigtable. You have a garbage collection policy set to delete the data after 30 days and the number of versions is set to 1. When the data analysts run a query to report total customer spending, the analysts sometimes see customer data that is older than 30 days. You need to ensure that the analysts do not see customer data older than 30 days while minimizing cost and overhead. What should you do?
A. Set the expiring values of the column families to 29 days and keep the number of versions to 1.
B. Use a timestamp range filter in the query to fetch the customer's data for a specific range.
C. Schedule a job daily to scan the data in the table and delete data older than 30 days.
D. Set the expiring values of the column families to 30 days and set the number of versions to 2.
Show Answer
Correct Answer: B
Explanation: In Bigtable, garbage collection is asynchronous and can take up to a week to physically remove expired data, so queries may still return data older than 30 days even with a GC policy in place. To ensure analysts never see data older than 30 days while minimizing cost and operational overhead, you must apply a read-time filter. Using a timestamp range filter in queries explicitly excludes cells older than 30 days, which is the recommended and documented best practice. Changing GC settings or running deletion jobs does not guarantee read-time correctness and adds unnecessary overhead.
Question 47
You are running a streaming pipeline with Dataflow and are using hopping windows to group the data as the data arrives. You noticed that some data is arriving late but is not being marked as late data, which is resulting in inaccurate aggregations downstream. You need to find a solution that allows you to capture the late data in the appropriate window. What should you do?
A. Use watermarks to define the expected data arrival window. Allow late data as it arrives.
B. Change your windowing function to tumbling windows to avoid overlapping window periods.
C. Change your windowing function to session windows to define your windows based on certain activity.
D. Expand your hopping window so that the late data has more time to arrive within the grouping.
Show Answer
Correct Answer: A
Explanation: In Dataflow/Beam streaming, whether data is considered late is determined by watermarks. If events arrive after the watermark without late-data handling configured, they may be dropped or mis-aggregated. By using watermarks correctly and explicitly allowing late data, late-arriving events are still assigned to their correct hopping windows and included in aggregations. Changing window types or sizes does not address the core issue of event-time vs watermark handling.
Question 48
You want to store your team’s shared tables in a single dataset to make data easily accessible to various analysts. You want to make this data readable but unmodifiable by analysts. At the same time, you want to provide the analysts with individual workspaces in the same project, where they can create and store tables for their own use, without the tables being accessible by other analysts. What should you do?
A. Give analysts the BigQuery Data Viewer role at the project level. Create one other dataset, and give the analysts the BigQuery Data Editor role on that dataset.
B. Give analysts the BigQuery Data Viewer role at the project level. Create a dataset for each analyst, and give each analyst the BigQuery Data Editor role at the project level.
C. Give analysts the BigQuery Data Viewer role on the shared dataset. Create a dataset for each analyst, and give each analyst the BigQuery Data Editor role at the dataset level for their assigned dataset.
D. Give analysts the BigQuery Data Viewer role on the shared dataset. Create one other dataset and give the analysts the BigQuery Data Editor role on that dataset.
Show Answer
Correct Answer: C
Explanation: The requirement is read-only access to a shared dataset and private, writable workspaces for each analyst. Granting BigQuery Data Viewer on the shared dataset ensures analysts can read but not modify shared tables. Creating a separate dataset per analyst and granting each analyst BigQuery Data Editor at the dataset level allows them to create and manage their own tables while preventing access to other analysts’ datasets. This follows least-privilege best practices.
Question 49
Your car factory is pushing machine measurements as messages into a Pub/Sub topic in your Google Cloud project. A Dataflow streaming job, that you wrote with the Apache Beam SDK, reads these messages, sends acknowledgment to Pub/Sub, applies some custom business logic in a DoFn instance, and writes the result to BigQuery. You want to ensure that if your business logic fails on a message, the message will be sent to a Pub/Sub topic that you want to monitor for alerting purposes. What should you do?
A. Enable retaining of acknowledged messages in your Pub/Sub pull subscription. Use Cloud Monitoring to monitor the subscription/num_retained_acked_messages metric on this subscription.
B. Use an exception handling block in your Dataflow’s DoFn code to push the messages that failed to be transformed through a side output and to a new Pub/Sub topic. Use Cloud Monitoring to monitor the topic/num_unacked_messages_by_region metric on this new topic.
C. Enable dead lettering in your Pub/Sub pull subscription, and specify a new Pub/Sub topic as the dead letter topic. Use Cloud Monitoring to monitor the subscription/dead_letter_message_count metric on your pull subscription.
D. Create a snapshot of your Pub/Sub pull subscription. Use Cloud Monitoring to monitor the snapshot/num_messages metric on this snapshot.
Show Answer
Correct Answer: B
Explanation: In Dataflow, messages are acknowledged to Pub/Sub before user-defined processing completes. If business logic in a DoFn fails after acknowledgment, Pub/Sub dead-lettering cannot capture those failures. The correct pattern is to handle exceptions inside the DoFn, route failed elements to a side output, and publish them to a dedicated Pub/Sub topic for monitoring and alerting. This is the recommended approach for error handling with Dataflow streaming pipelines.
Question 50
You are designing a real-time system for a ride hailing app that identifies areas with high demand for rides to effectively reroute available drivers to meet the demand. The system ingests data from multiple sources to Pub/Sub, processes the data, and stores the results for visualization and analysis in real-time dashboards. The data sources include driver location updates every 5 seconds and app-based booking events from riders. The data processing involves real-time aggregation of supply and demand data for the last 30 seconds, every 2 seconds, and storing the results in a low-latency system for visualization. What should you do?
A. Group the data by using a tumbling window in a Dataflow pipeline, and write the aggregated data to Memorystore.
B. Group the data by using a hopping window in a Dataflow pipeline, and write the aggregated data to Memorystore.
C. Group the data by using a session window in a Dataflow pipeline, and write the aggregated data to BigQuery.
D. Group the data by using a hopping window in a Dataflow pipeline, and write the aggregated data to BigQuery.
Show Answer
Correct Answer: B
Explanation: The requirement is to compute aggregates over the last 30 seconds, updated every 2 seconds, which implies overlapping windows. Hopping (sliding) windows in Dataflow are designed for this pattern, unlike tumbling windows which do not overlap. The results must be stored in a low-latency system for real-time dashboards; Memorystore provides in-memory, sub-millisecond access, whereas BigQuery is optimized for analytical queries with higher latency. Therefore, a hopping window with output to Memorystore is the correct choice.
Question 51
You are designing a Dataflow pipeline for a batch processing job. You want to mitigate multiple zonal failures at job submission time. What should you do?
A. Submit duplicate pipelines in two different zones by using the --zone flag.
B. Set the pipeline staging location as a regional Cloud Storage bucket.
C. Specify a worker region by using the --region flag.
D. Create an Eventarc trigger to resubmit the job in case of zonal failure when submitting the job.
Show Answer
Correct Answer: C
Explanation: To mitigate multiple zonal failures at Dataflow job submission time, you should run the job at the regional level. Specifying a worker region with the --region flag allows Dataflow to place and manage resources across multiple zones within that region, providing resilience against zonal outages. Using a specific zone, duplicating pipelines, or adding external resubmission logic does not address the root issue as effectively as regional execution.
Question 52
You have a BigQuery dataset named “customers”. All tables will be tagged by using a Data Catalog tag template named “gdpr”. The template contains one mandatory field, “has_sensitive_data”, with a boolean value. All employees must be able to do a simple search and find tables in the dataset that have either true or false in the “has_sensitive_data’ field. However, only the Human Resources (HR) group should be able to see the data inside the tables for which “has_sensitive data” is true. You give the all employees group the bigquery.metadataViewer and bigquery.connectionUser roles on the dataset. You want to minimize configuration overhead. What should you do next?
A. Create the “gdpr” tag template with private visibility. Assign the bigquery.dataViewer role to the HR group on the tables that contain sensitive data.
B. Create the “gdpr” tag template with private visibility. Assign the datacatalog.tagTemplateViewer role on this tag to the all employees group, and assign the bigquery.dataViewer role to the HR group on the tables that contain sensitive data.
C. Create the “gdpr” tag template with public visibility. Assign the bigquery.dataViewer role to the HR group on the tables that contain sensitive data.
D. Create the “gdpr” tag template with public visibility. Assign the datacatalog.tagTemplateViewer role on this tag to the all employees group, and assign the bigquery.dataViewer role to the HR group on the tables that contain sensitive data.
Show Answer
Correct Answer: C
Explanation: All employees must be able to search tables by the tag field value with minimal overhead, while only HR can view sensitive data. Using a Data Catalog tag template with public visibility allows anyone who can view the BigQuery tables (via bigquery.metadataViewer) to see and search on the tag values without granting extra Data Catalog roles. Data access is controlled separately by granting bigquery.dataViewer only to HR on tables with sensitive data. This meets the requirements with the least additional IAM configuration.
$19
Get all 328 questions with detailed answers and explanations
Instant download HTML + PDF delivered the moment payment clears.
Secure Stripe checkout we never see or store your card details.
7-day refund if files are defective see our refund policy.