Packages

Understand how to modularize and extend your models with packages.

Overview

Packages represent libraries made available by other users for public or private usage.

Install and use a package

Locate the packages.yml file

In Code Editor mode, open the packages.yml file in the project's root directory. Create one if it does not yet exist. Note that it must be placed at the top-level of your Y42 project, alongside dbt_project.yml.

y42

alerts
models
orchestrations
...
dbt_project.yml
packages.yml

Add the package

To install packages in Y42, specify the package's name and version in the packages.yml file. dbt maintains a hub of open-source packages (opens in a new tab) that can be utilized in dbt models.

For example, to install the dbt_expectations (opens in a new tab) package:

packages.yml

packages:
  - package: calogica/dbt_expectations
    version: [">=0.8.0", "<0.9.0"]

You packages will be automatically installed once you commit your changes.

Show packages in the UI

To show packages in the Asset List, Code, and Data Lineage use CMD + K > Install package dependencies (if you haven't installed any package dependencies yet) or Refresh package dependencies (if you have installed new packages and want to refresh the packages displayed in the UI).

Use the package

We can now reference any function from the dbt_expectation package in our models. For example, to check if a model's row count falls within a range of values, we can use the expect_table_row_count_to_be_between test from the package:

models/orders.yml

version: 2
models:
  - name: orders
    tests: 
      - dbt_expectations.expect_table_row_count_to_be_between:
          min_value: 100
          max_value: 1000
    columns:
      - name: order_id
        tests: # column-level tests
          - unique
          - not_null

Package specific configuration

While many packages in Y42 function immediately upon installation, there are some that require additional steps for configuration. Certain packages, for example, require particular variables to be configured in the dbt_project.yml file for them to operate correctly.

Example: Configuring the Salesforce package

Consider the Salesforce package (opens in a new tab) as an example. It requires the salesforce_database and salesforce_schema variables to be configured within the dbt_project.yml file.

By default, this package will run using your target database and the salesforce schema. If your Salesforce data is not located there, you need to add the following configuration to your root dbt_project.yml file:

dbt_project.yml

# ... Other Configurations ...
vars:
  salesforce_database: 'your_database_name' # replace with your actual database name
  salesforce_schema: 'your_schema_name'     # replace with your actual schema name
# ... Other Configurations ...

Always remember to check the documentation of the specific package you are installing. Each package may have its own unique configuration needs that are best detailed in their individual documentation, and configuring them correctly is crucial to ensure they function properly within Y42.

FAQ

Are all dbt packages cross-compatible with Y42?

Macros Tests