Professional Data Engineer Free Practice Questions — Page 4
Question 16
You are creating the CI/CD cycle for the code of the directed acyclic graphs (DAGs) running in Cloud Composer. Your team has two Cloud Composer instances: one instance for development and another instance for production. Your team is using a Git repository to maintain and develop the code of the DAGs. You want to deploy the DAGs automatically to Cloud Composer when a certain tag is pushed to the Git repository. What should you do?
A. 1. Use Cloud Build to copy the code of the DAG to the Cloud Storage bucket of the development instance for DAG testing. 2. If the tests pass, use Cloud Build to copy the code to the bucket of the production instance.
B. 1. Use Cloud Build to build a container with the code of the DAG and the KubernetesPodOperator to deploy the code to the Google Kubernetes Engine (GKE) cluster of the development instance for testing. 2. If the tests pass, use the KubernetesPodOperator to deploy the container to the GKE cluster of the production instance.
C. 1. Use Cloud Build to build a container and the KubernetesPodOperator to deploy the code of the DAG to the Google Kubernetes Engine (GKE) cluster of the development instance for testing. 2. If the tests pass, copy the code to the Cloud Storage bucket of the production instance.
D. 1. Use Cloud Build to copy the code of the DAG to the Cloud Storage bucket of the development instance for DAG testing. 2. If the tests pass, use Cloud Build to build a container with the code of the DAG and the KubernetesPodOperator to deploy the container to the Google Kubernetes Engine (GKE) cluster of the production instance.
Show Answer
Correct Answer: A
Explanation: Cloud Composer DAGs are deployed by placing files in the environment’s Cloud Storage DAGs bucket. A CI/CD pipeline triggered by a Git tag can use Cloud Build to copy DAG code to the development Composer bucket for validation, and upon success, copy the same code to the production Composer bucket. Using GKE or container-based deployment is unnecessary and not the standard approach for managing Composer DAGs.
Question 16
You need to orchestrate a pipeline with several Google Cloud services: a batch Dataflow job, then a BigQuery query job followed by a Vertex AI batch prediction. The logic is sequential. You want a lightweight, serverless orchestration solution with minimal operational overhead. What service should you use?
A. Select Cloud Composer.
B. Select Compute Engine with cron.
C. Select Dataproc with Apache Oozie.
D. Select Cloud Workflows.
Show Answer
Correct Answer: D
Explanation: The requirement is for a sequential orchestration across managed GCP services with minimal operational overhead. Cloud Workflows is fully serverless, lightweight, and designed to orchestrate Google Cloud APIs such as Dataflow, BigQuery, and Vertex AI in a step-by-step manner. Cloud Composer (Airflow) is heavier and requires infrastructure management, while Compute Engine with cron and Dataproc with Oozie add unnecessary operational complexity.
Question 17
You have 100 GB of data stored in a BigQuery table. This data is outdated and will only be accessed one or two times a year for analytics with SQL. For backup purposes, you want to store this data to be immutable for 3 years. You want to minimize storage costs. What should you do?
A. 1. Create a BigQuery table clone. 2. Query the clone when you need to perform analytics.
B. 1. Create a BigQuery table snapshot. 2. Restore the snapshot when you need to perform analytics.
C. 1. Perform a BigQuery export to a Cloud Storage bucket with archive storage class. 2. Enable versioning on the bucket. 3. Create a BigQuery external table on the exported files.
D. 1. Perform a BigQuery export to a Cloud Storage bucket with archive storage class. 2. Set a locked retention policy on the bucket. 3. Create a BigQuery external table on the exported files.
Show Answer
Correct Answer: D
Explanation: The goal is to minimize storage cost while keeping data immutable for 3 years and still query it occasionally. Exporting the BigQuery table to Cloud Storage with the Archive storage class provides the lowest-cost storage for rarely accessed data. Setting a locked retention policy enforces immutability for the required period, which versioning alone does not guarantee. Creating an external BigQuery table allows querying the data with SQL when needed without restoring it back into BigQuery storage. BigQuery clones or snapshots are more expensive and unnecessary for long-term archival.
Question 17
Your company uses Looker Studio connected to BigQuery for reporting. Users are experiencing slow dashboard load times due to complex queries on a large table. The queries involve aggregations and filtering on several columns. You need to optimize query performance to decrease the dashboard load times. What should you do?
A. Configure Looker Studio to use a shorter data refresh interval to ensure fresh data is always displayed.
B. Create a materialized view in BigQuery that pre-calculates the aggregations and filters used in the Looker Studio dashboards.
C. Implement row-level security in BigQuery to restrict data access and reduce the amount of data processed by the queries.
D. Use BigQuery BI Engine to accelerate query performance by caching frequently accessed data.
Show Answer
Correct Answer: B, D
Explanation: Slow Looker Studio dashboards caused by complex aggregations on large BigQuery tables are best addressed by reducing query work and accelerating execution. Creating a materialized view pre-computes common aggregations and filters, significantly reducing processing at query time. Using BigQuery BI Engine further improves performance by caching data in memory and optimizing queries from BI tools like Looker Studio. Shorter refresh intervals and row-level security do not directly address query performance.
Question 18
You have created an external table for Apache Hive partitioned data that resides in a Cloud Storage bucket, which contains a large number of files. You notice that queries against this table are slow. You want to improve the performance of these queries. What should you do?
A. Change the storage class of the Hive partitioned data objects from Coldline to Standard.
B. Create an individual external table for each Hive partition by using a common table name prefix. Use wildcard table queries to reference the partitioned data.
C. Upgrade the external table to a BigLake table. Enable metadata caching for the table.
D. Migrate the Hive partitioned data objects to a multi-region Cloud Storage bucket.
Show Answer
Correct Answer: C
Explanation: Queries on external Hive-partitioned data with many files are often slowed by metadata listing and processing. Upgrading the external table to a BigLake table and enabling metadata caching reduces metadata overhead by caching file and partition metadata, significantly improving query performance. The other options do not address the core metadata bottleneck.
Question 18
Your data science team needs to perform interactive SQL queries on large datasets stored in Apache Parquet format within a Cloud Storage bucket. The team is familiar with Apache Hive and wants to leverage existing HiveQL queries. You need to provide an environment for the team to run their interactive HiveQL queries directly against the data in Cloud Storage. You want to keep operational overhead to a minimum. What should you do?
A. Install and configure an Apache Hadoop and Hive cluster manually on a group of Compute Engine instances.
B. Load the Parquet data into a BigQuery native table and use the BigQuery Connector for Hive to run the queries.
C. Configure BigQuery with an external table definition pointing to the Parquet files.
D. Deploy a Dataproc cluster with Hive services enabled.
Show Answer
Correct Answer: D
Explanation: The team wants to run interactive HiveQL queries directly on Parquet files stored in Cloud Storage, with minimal operational overhead. Dataproc is a managed service that provides Apache Hive out of the box, integrates natively with Cloud Storage, and supports querying Parquet data without data movement. It avoids the heavy operational burden of manually managing Hadoop/Hive clusters while preserving full HiveQL compatibility, making it the best fit.
Question 19
You are monitoring your organization’s data lake hosted on BigQuery. The ingestion pipelines read data from Pub/Sub and write the data into tables on BigQuery. After a new version of the ingestion pipelines is deployed, the daily stored data increased by 50%. The volumes of data in Pub/Sub remained the same and only some tables had their daily partition data size doubled. You need to investigate and fix the cause of the data increase. What should you do?
A. 1. Check for duplicate rows in the BigQuery tables that have the daily partition data size doubled. 2. Schedule daily SQL jobs to deduplicate the affected tables. 3. Share the deduplication script with the other operational teams to reuse if this occurs to other tables.
B. 1. Check for code errors in the deployed pipelines. 2. Check for multiple writing to pipeline BigQuery sink. 3. Check for errors in Cloud Logging during the day of the release of the new pipelines. 4. If no errors, restore the BigQuery tables to their content before the last release by using time travel.
C. 1. Check for duplicate rows in the BigQuery tables that have the daily partition data size doubled. 2. Check the BigQuery Audit logs to find job IDs. 3. Use Cloud Monitoring to determine when the identified Dataflow jobs started and the pipeline code version. 4. When more than one pipeline ingests data into a table, stop all versions except the latest one.
D. 1. Roll back the last deployment. 2. Restore the BigQuery tables to their content before the last release by using time travel. 3. Restart the Dataflow jobs and replay the messages by seeking the subscription to the timestamp of the release.
Show Answer
Correct Answer: C
Explanation: The data volume increase happened after a new pipeline deployment, while Pub/Sub input volume stayed the same and only some BigQuery partitions doubled. This strongly suggests duplicate ingestion caused by multiple pipeline versions or jobs writing to the same tables. Option C focuses on investigation and root-cause resolution: verifying duplicate rows, using BigQuery Audit Logs to identify which jobs wrote the data, correlating job execution with pipeline versions via Cloud Monitoring, and stopping overlapping or duplicate pipeline versions. This directly identifies and fixes the cause of the data increase rather than masking it or only restoring data. Other options either assume remediation without full investigation (B, D) or rely on ongoing deduplication instead of fixing the ingestion issue (A).
Question 20
Your organization's data assets are stored in BigQuery, Pub/Sub, and a PostgreSQL instance running on Compute Engine. Because there are multiple domains and diverse teams using the data, teams in your organization are unable to discover existing data assets. You need to design a solution to improve data discoverability while keeping development and configuration efforts to a minimum. What should you do?
A. Use Data Catalog to automatically catalog BigQuery datasets. Use Data Catalog APIs to manually catalog Pub/Sub topics and PostgreSQL tables.
B. Use Data Catalog to automatically catalog BigQuery datasets and Pub/Sub topics. Use Data Catalog APIs to manually catalog PostgreSQL tables.
C. Use Data Catalog to automatically catalog BigQuery datasets and Pub/Sub topics. Use custom connectors to manually catalog PostgreSQL tables.
D. Use customer connectors to manually catalog BigQuery datasets, Pub/Sub topics, and PostgreSQL tables.
Show Answer
Correct Answer: B
Explanation: The requirement is to improve data discoverability with minimal development and configuration effort. Data Catalog natively and automatically catalogs BigQuery datasets and Pub/Sub topics, requiring no extra work. PostgreSQL running on Compute Engine is an external data source, and while community connectors exist, using them still requires deployment and configuration. The simplest, lowest-effort approach is to create custom entries via the Data Catalog APIs for PostgreSQL metadata. Therefore, option B best satisfies the requirements.
Question 21
You are designing a fault-tolerant architecture to store data in a regional BigQuery dataset. You need to ensure that your application is able to recover from a corruption event in your tables that occurred within the past seven days. You want to adopt managed services with the lowest RPO and most cost-effective solution. What should you do?
A. Access historical data by using time travel in BigQuery.
B. Export the data from BigQuery into a new table that excludes the corrupted data
C. Create a BigQuery table snapshot on a daily basis.
D. Migrate your data to multi-region BigQuery buckets.
Show Answer
Correct Answer: A
Explanation: BigQuery time travel allows point-in-time access to table data for up to seven days by default, enabling recovery from recent corruption with the lowest possible RPO. It is fully managed, requires no manual backups, and is the most cost-effective option since it is built in without additional storage or operational overhead.
Question 22
You are running your BigQuery project in the on-demand billing model and are executing a change data capture (CDC) process that ingests data. The CDC process loads 1 GB of data every 10 minutes into a temporary table, and then performs a merge into a 10 TB target table. This process is very scan intensive and you want to explore options to enable a predictable cost model. You need to create a BigQuery reservation based on utilization information gathered from BigQuery Monitoring and apply the reservation to the CDC process. What should you do?
A. Create a BigQuery reservation for the dataset.
B. Create a BigQuery reservation for the job.
C. Create a BigQuery reservation for the service account running the job.
D. Create a BigQuery reservation for the project.
Show Answer
Correct Answer: D
Explanation: BigQuery reservations (slot commitments) can only be assigned at the organization, folder, or project level. They cannot be applied directly to datasets, jobs, or service accounts. To move from on‑demand to a predictable cost model for a scan‑intensive CDC workload, you must purchase slot capacity and assign the reservation to the project that runs the CDC process. Jobs in that project will then use the reserved slots instead of on‑demand billing.
$19
Get all 328 questions with detailed answers and explanations
Instant download HTML + PDF delivered the moment payment clears.
Secure Stripe checkout we never see or store your card details.
7-day refund if files are defective see our refund policy.