Search
START TYPING AND PRESS ENTER TO SEARCH

    Have a Question About This Course?





    Image

    Orchestrate BigQuery Workloads with Dataform (OBWD)

    Google Cloud
    Dataform is a service for data analysts to develop, test, version control, and schedule complex SQL workflows for data transformation in BigQuery. In this course you will explore the components of Dataform core, learn how to define tables and dependencies in SQLX, document BigQuery tables and views, understand BigQuery security settings and how to manage these with Dataform, write assertions, execute SQL workflows, and explore additional advanced use cases.

    Orchestrate BigQuery Workloads with Dataform (OBWD) Objectives

    • Understand the components of Dataform core.
    • Create tables and views in BigQuery using Dataform.
    • Document BigQuery tables and views.
    • Understand BigQuery security settings using Dataform.
    • Use assertions to validate data in Dataform workflows.
    • Execute Dataform SQL workflows in an automated fashion.

    Need Assistance Finding the Right Training Solution

    Our Consultants are here to assist you
    Arrange a consultation

    Key Point of Training Programs

    • Orchestrate BigQuery Workloads with Dataform (OBWD) Prerequisites

      Knowledge of SQL data analysis and BigQuery as discussed in BigQuery for Data Analysis.

    • Orchestrate BigQuery Workloads with Dataform (OBWD) Course Format

      Live Virtual Course

    • Orchestrate BigQuery Workloads with Dataform (OBWD) Outline

      Module 1 - Dataform Core Components
      Topics:

      SQL workflow
      Repositories and workspaces
      Default files and folders
      Compiled graphs
      Objectives:

      Understand the components of Dataflow core.
      Module 2 - Table Definitions and Dependencies
      Topics:

      Declare a data source.
      Create a table.
      Create an incremental table.
      Set partitioning and clustering options.
      Create an empty table.
      Create an external BigLake table.
      Create views and materialized views.
      Define dependencies.
      Objectives:

      Create tables and views in BigQuery using Dataform
      Module 3 - Document BigQuery Tables and Views
      Topics:

      Use column descriptions.
      Use globally defined JavaScript constants.
      Add labels.
      Objectives:

      Document BigQuery tables and views.
      Activities:

      Lab: Build SQL Workflows with Dependencies in Dataform
      Module 4 - BigQuery Security Settings
      Topics:

      IAM dataset and table/view access
      Column-level security
      Row-level security
      Objectives:

      Understand BigQuery security settings using Dataform
      Module 5 - Assertions
      Topics:

      Use built-in assertions.
      Create manual assertions.
      Objectives:

      Use assertions to validate data in Dataform workflows.
      Activities:

      Lab: Work with Assertions and BigQuery Security Settings in Dataform.
      Module 6 - SQL Workflow Executions
      Topics:

      Dataform code lifecycle.
      What happens during compilation.
      Customize and schedule compilation results.
      Execute workflows (UI, Cloud Scheduler, Cloud Composer).
      Logging and monitoring.
      Objectives:

      Execute Dataform SQL workflows in an automated fashion.
      Activities:

      Lab: Automate and Monitor SQL Workflow Executions in Dataform
      Module 7 - Advanced Use Cases
      Topics:

      Create a BigLake table after file upload using Cloud Run functions.
      Build a Machine Learning pipeline with BigQuery ML.
      Work with Slowly Changing Dimensions Type 2.
      Objectives:

      Explore additional use cases for Dataform.
      Activities:

      Lab: Create a BigLake Table with Dataform Using Cloud Run Functions.