Published views

Learn how to turn your branches into data warehouse environments.

Overview

With Virtual Data Builds, you can manage your data environment by turning branches into virtual environments. This functionality offers flexibility and control over which branches to publish, and it allows for seamless integration with datasets and schemas in Bigquery and Snowflake.

Turning branches into data warehouse environments

Selecting branches to publish

By default, the main branch is always published.

You can easily decide which additional branches to publish by following these steps:

Click on your profile picture.
Navigate to Space settings > Features.
Add branches to Published views.

Naming Convention for branches and assets

A new dataset (for Bigquery) or schema (for Snowflake) will be created using the following naming convention: y42__{{space_slug}}__{{branch_name}}.

The naming convention for the two types of assets materialized in the dataset/schema is as follows:

Model: {{model_name}}
Source: source__{{source_name}}__{{table_name}}

Creating views based on the published branch

The views will be created based on the state of the branch that you publish. These views target the correct physical UUID tables, leveraging the internals of Virtual Data Builds.

Comparison between Y42 and BigQuery, illustrating the link between the asset in Y42, and the corresponding UUID in BigQuery.

Referencing assets

Y42 creates a view for every asset that has been built successfully, referencing the correct underlying UUID table of the DWH within the dataset/schema of the respective published branch.

Managing and controlling published views

View Reference to the Latest Successful Build

The views always reference the latest successful build of the respective branch (excluding builds that have failed tests).
Within the Y42 UI, you can find a redirection link straight to the view as part of the last successful build. This functionality only works if the branch is published as a virtual environment.

Unpublishing Assets

If certain assets shouldn't be published to the virtual branch environment of the respective branch, you can simply do so in the UI by hovering over the link and clicking on “Unpublish.”

Under the hood, this modifies a property called publish_view: boolean, which can also be edited directly in code mode.

my_first_dbt_model.yml

models:
  - name: my_first_dbt_model
    description: "A starter dbt model"
    config:
      y42_table:
        publish_view: false 
    columns:
      - name: id
        description: "The primary key for this table"
        tests:
          - unique
          - not_null

This feature is particularly useful if certain sources or models, e.g., staging models or raw sources, shouldn't be published for downstream consumption.