Professional Data Engineer Free Practice Questions — Page 9
Question 63
You are deploying an Apache Airflow directed acyclic graph (DAG) in a Cloud Composer 2 instance. You have incoming files in a Cloud Storage bucket that the DAG processes, one file at a time. The Cloud Composer instance is deployed in a subnetwork with no Internet access. Instead of running the DAG based on a schedule, you want to run the DAG in a reactive way every time a new file is received. What should you do?
A. 1. Enable Private Google Access in the subnetwork, and set up Cloud Storage notifications to a Pub/Sub topic. 2. Create a push subscription that points to the web server URL.
B. 1. Enable the Cloud Composer API, and set up Cloud Storage notifications to trigger a Cloud Function. 2. Write a Cloud Function instance to call the DAG by using the Cloud Composer API and the web server URL. 3. Use VPC Serverless Access to reach the web server URL.
C. 1. Enable the Airflow REST API, and set up Cloud Storage notifications to trigger a Cloud Function instance. 2. Create a Private Service Connect (PSC) endpoint. 3. Write a Cloud Function that connects to the Cloud Composer cluster through the PSC endpoint.
D. 1. Enable the Airflow REST API, and set up Cloud Storage notifications to trigger a Cloud Function instance. 2. Write a Cloud Function instance to call the DAG by using the Airflow REST API and the web server URL. 3. Use VPC Serverless Access to reach the web server URL.
Show Answer
Correct Answer: A
Explanation: In a Cloud Composer 2 environment with no internet access, you cannot reliably call the Airflow web server or REST API from Cloud Functions using the web server URL, PSC, or Serverless VPC Access. Google’s supported pattern for reactive triggering in private environments is Pub/Sub-based triggering: enable Private Google Access, configure Cloud Storage notifications to Pub/Sub, and have Airflow consume Pub/Sub messages (for example with a sensor or pull-based DAG) to trigger processing. This avoids any direct inbound connectivity to the Airflow web server and is the documented, supported approach.
Question 64
You have an Oracle database deployed in a VM as part of a Virtual Private Cloud (VPC) network. You want to replicate and continuously synchronize 50 tables to BigQuery. You want to minimize the need to manage infrastructure. What should you do?
A. Deploy Apache Kafka in the same VPC network, use Kafka Connect Oracle Change Data Capture (CDC), and Dataflow to stream the Kafka topic to BigQuery.
B. Create a Pub/Sub subscription to write to BigQuery directly. Deploy the Debezium Oracle connector to capture changes in the Oracle database, and sink to the Pub/Sub topic.
C. Deploy Apache Kafka in the same VPC network, use Kafka Connect Oracle change data capture (CDC), and the Kafka Connect Google BigQuery Sink Connector.
D. Create a Datastream service from Oracle to BigQuery, use a private connectivity configuration to the same VPC network, and a connection profile to BigQuery.
Show Answer
Correct Answer: D
Explanation: The requirement is to continuously replicate Oracle tables to BigQuery while minimizing infrastructure management. Datastream is a fully managed, serverless CDC service that natively supports Oracle-to-BigQuery replication with private VPC connectivity. The other options require deploying and operating Kafka, Debezium, or Dataflow components, which adds significant operational overhead.
Question 65
You are running a Dataflow streaming pipeline, with Streaming Engine and Horizontal Autoscaling enabled. You have set the maximum number of workers to 1000. The input of your pipeline is Pub/Sub messages with notifications from Cloud Storage. One of the pipeline transforms reads CSV files and emits an element for every CSV line. The job performance is low, the pipeline is using only 10 workers, and you notice that the autoscaler is not spinning up additional workers. What should you do to improve performance?
A. Enable Vertical Autoscaling to let the pipeline use larger workers.
B. Change the pipeline code, and introduce a Reshuffle step to prevent fusion.
C. Update the job to increase the maximum number of workers.
D. Use Dataflow Prime, and enable Right Fitting to increase the worker resources.
Show Answer
Correct Answer: B
Explanation: The pipeline is limited by fusion: reading Pub/Sub notifications and then reading CSV files can be fused into a single stage, restricting parallelism and preventing the autoscaler from adding workers. Adding a Reshuffle breaks fusion, redistributes elements, increases parallelism, and allows horizontal autoscaling to scale out. Increasing max workers is already done, and vertical autoscaling or right fitting won’t address the lack of parallelism.
Question 66
You are deploying a batch pipeline in Dataflow. This pipeline reads data from Cloud Storage, transforms the data, and then writes the data into BigQuery. The security team has enabled an organizational constraint in Google Cloud, requiring all Compute Engine instances to use only internal IP addresses and no external IP addresses. What should you do?
A. Ensure that your workers have network tags to access Cloud Storage and BigQuery. Use Dataflow with only internal IP addresses.
B. Ensure that the firewall rules allow access to Cloud Storage and BigQuery. Use Dataflow with only internal IPs.
C. Create a VPC Service Controls perimeter that contains the VPC network and add Dataflow, Cloud Storage, and BigQuery as allowed services in the perimeter. Use Dataflow with only internal IP addresses.
D. Ensure that Private Google Access is enabled in the subnetwork. Use Dataflow with only internal IP addresses.
Show Answer
Correct Answer: D
Explanation: Dataflow workers are Compute Engine VMs. When an organization enforces a constraint that disallows external IP addresses, the correct way to let those workers still access Google-managed services such as Cloud Storage and BigQuery is to enable Private Google Access on the subnetwork and run Dataflow with only internal IPs. Private Google Access allows VMs without external IPs to reach Google APIs and services over Google’s network. VPC Service Controls are used to define security perimeters around APIs and are not required to satisfy the internal-IP-only requirement in this scenario.
Question 67
You are designing a data warehouse in BigQuery to analyze sales data for a telecommunication service provider. You need to create a data model for customers, products, and subscriptions. All customers, products, and subscriptions can be updated monthly, but you must maintain a historical record of all data. You plan to use the visualization layer for current and historical reporting. You need to ensure that the data model is simple, easy-to-use, and cost-effective. What should you do?
A. Create a normalized model with tables for each entity. Use snapshots before updates to track historical data.
B. Create a normalized model with tables for each entity. Keep all input files in a Cloud Storage bucket to track historical data.
C. Create a denormalized model with nested and repeated fields. Update the table and use snapshots to track historical data.
D. Create a denormalized, append-only model with nested and repeated fields. Use the ingestion timestamp to track historical data.
Show Answer
Correct Answer: D
Explanation: An append-only, denormalized model in BigQuery is simple and cost-effective for analytics. Using nested and repeated fields reduces joins and improves query performance for BI tools. Appending new records instead of updating preserves full history without managing snapshots, and an ingestion timestamp enables easy point-in-time and current-state reporting in the visualization layer.
Question 68
You have important legal hold documents in a Cloud Storage bucket. You need to ensure that these documents are not deleted or modified. What should you do?
A. Set a retention policy. Lock the retention policy.
B. Set a retention policy. Set the default storage class to Archive for long-term digital preservation.
C. Enable the Object Versioning feature. Add a lifecycle rule.
D. Enable the Object Versioning feature. Create a copy in a bucket in a different region.
Show Answer
Correct Answer: A
Explanation: A bucket-level retention policy prevents objects from being deleted or overwritten for the specified retention period. Locking the retention policy makes it immutable, ensuring the period cannot be shortened or removed, which is required for legal hold and compliance scenarios.
Question 69
Your company's data platform ingests CSV file dumps of booking and user profile data from upstream sources into Cloud Storage. The data analyst team wants to join these datasets on the email field available in both the datasets to perform analysis. However, personally identifiable information (PII) should not be accessible to the analysts. You need to de-identify the email field in both the datasets before loading them into BigQuery for analysts. What should you do?
A. 1. Create a pipeline to de-identify the email field by using recordTransformations in Cloud Data Loss Prevention (Cloud DLP) with masking as the de-identification transformations type. 2. Load the booking and user profile data into a BigQuery table.
B. 1. Create a pipeline to de-identify the email field by using recordTransformations in Cloud DLP with format-preserving encryption with FFX as the de-identification transformation type. 2. Load the booking and user profile data into a BigQuery table.
C. 1. Load the CSV files from Cloud Storage into a BigQuery table, and enable dynamic data masking. 2. Create a policy tag with the email mask as the data masking rule. 3. Assign the policy to the email field in both tables. A 4. Assign the Identity and Access Management bigquerydatapolicy.maskedReader role for the BigQuery tables to the analysts.
D. 1. Load the CSV files from Cloud Storage into a BigQuery table, and enable dynamic data masking. 2. Create a policy tag with the default masking value as the data masking rule. 3. Assign the policy to the email field in both tables. 4. Assign the Identity and Access Management bigquerydatapolicy.maskedReader role for the BigQuery tables to the analysts
Show Answer
Correct Answer: B
Explanation: The analysts must be able to join on the email field while never accessing the real PII, and the de‑identification must occur before loading into BigQuery. Cloud DLP format‑preserving encryption (FPE) with FFX produces a deterministic, consistent pseudonym for the same email across datasets, enabling reliable joins while hiding the original value. Simple masking would break joinability, and BigQuery dynamic data masking applies after load and does not meet the requirement to de‑identify data prior to ingestion.
Question 70
dataset.inventory_vm sample records:
You have an inventory of VM data stored in the BigQuery table. You want to prepare the data for regular reporting in the most cost-effective way. You need to exclude VM rows with fewer than 8 vCPU in your report. What should you do?
A. Create a view with a filter to drop rows with fewer than 8 vCPU, and use the UNNEST operator.
B. Create a materialized view with a filter to drop rows with fewer than 8 vCPU, and use the WITH common table expression.
C. Create a view with a filter to drop rows with fewer than 8 vCPU, and use the WITH common table expression.
D. Use Dataflow to batch process and write the result to another BigQuery table.
Show Answer
Correct Answer: A
Explanation: The goal is to prepare data for regular reporting in the most cost-effective way while filtering out VMs with fewer than 8 vCPU. A standard view is the cheapest option because it stores no data and incurs cost only at query time, unlike materialized views or Dataflow pipelines, which add storage and processing costs. The vCPU information is stored in a nested field, so UNNEST is required to access and filter it correctly. Therefore, creating a simple view with a filter and UNNEST satisfies both the technical requirement and the cost constraint.
Question 71
Your company operates in three domains: airlines, hotels, and ride-hailing services. Each domain has two teams: analytics and data science, which create data assets in BigQuery with the help of a central data platform team. However, as each domain is evolving rapidly, the central data platform team is becoming a bottleneck. This is causing delays in deriving insights from data, and resulting in stale data when pipelines are not kept up to date. You need to design a data mesh architecture by using Dataplex to eliminate the bottleneck. What should you do?
A. 1. Create one lake for each team. Inside each lake, create one zone for each domain. 2. Attach each of the BigQuery datasets created by the individual teams as assets to the respective zone. 3. Have the central data platform team manage all zones’ data assets.
B. 1. Create one lake for each team. Inside each lake, create one zone for each domain. 2. Attach each of the BigQuery datasets created by the individual teams as assets to the respective zone. 3. Direct each domain to manage their own zone’s data assets.
C. 1. Create one lake for each domain. Inside each lake, create one zone for each team. 2. Attach each of the BigQuery datasets created by the individual teams as assets to the respective zone. 3. Direct each domain to manage their own lake’s data assets.
D. 1. Create one lake for each domain. Inside each lake, create one zone for each team. 2. Attach each of the BigQuery datasets created by the individual teams as assets to the respective zone. 3. Have the central data platform team manage all lakes’ data assets.
Show Answer
Correct Answer: C
Explanation: A data mesh with Dataplex should be domain-oriented and decentralized. Creating one lake per domain establishes clear domain ownership, while zones per team organize assets within the domain. Letting each domain manage its own lake removes the central platform bottleneck and aligns with Dataplex’s domain-centric data mesh model.
Question 72
You have one BigQuery dataset which includes customers’ street addresses. You want to retrieve all occurrences of street addresses from the dataset. What should you do?
A. Write a SQL query in BigQuery by using REGEXP_CONTAINS on all tables in your dataset to find rows where the word “street” appears.
B. Create a deep inspection job on each table in your dataset with Cloud Data Loss Prevention and create an inspection template that includes the STREET_ADDRESS infoType.
C. Create a discovery scan configuration on your organization with Cloud Data Loss Prevention and create an inspection template that includes the STREET_ADDRESS infoType.
D. Create a de-identification job in Cloud Data Loss Prevention and use the masking transformation.
Show Answer
Correct Answer: B
Explanation: To retrieve all occurrences of street addresses, you need accurate detection of address patterns rather than simple keyword matching. Cloud Data Loss Prevention (DLP) provides a predefined STREET_ADDRESS infoType that detects full street addresses in structured data like BigQuery tables. Creating a deep inspection job on the dataset’s tables allows you to scan the specific dataset and identify all rows containing street addresses. SQL regex (A) is unreliable, discovery scans (C) are broader and unnecessary for a single dataset, and de-identification (D) focuses on masking rather than retrieval.
$19
Get all 328 questions with detailed answers and explanations
Instant download HTML + PDF delivered the moment payment clears.
Secure Stripe checkout we never see or store your card details.
7-day refund if files are defective see our refund policy.