Projects
Manage your data pipelines in a unified directory.
Y42 project configuration.Overview
A Y42 project is a collection of SQL and YAML code files. The SQL files define data transformations, while the YAML files specify configurations. These files can be further organized into subdirectories, according to your desired structure.
Project structure
When you create a space, Y42 initializes a Git repository with the default folder structure shown below. All top-level folders must be present in order for the project to be successfully compiled. However, you can customize the internal folder structure of each top-level folder to best suit your organizational needs.
Project config: Define your project's metadata and table settings.
Sources: Load and extract your raw data from external sources.
Models: Transform your data, so that it is ready for analysis.
Tests: Run tests to check whether your tables are correct.
Orchestrations: Build your data pipelines in an automatic workflow.
Alerts: Send notifications when tests fail during orchestrations.
Selectors: Specify builds of the data pipeline that are more advanced.
Snapshots: Capture states of data over time.
Analysis: Explore your data with ad-hoc transformations.
Macros: Write reusable code for your SQL statements.
Packages: Import prewritten code from others to your project.
Gitignore: Choose files or directories that shouldn't be tracked with git.
And there we have it, our data pipelines defined in a unified directory.
Project Configuration
The dbt_project.yml file serves as the project-level configuration file.
It is the only file that needs to be defined, at minimum, and must be placed in the project's root directory.
Note that you can access this file, along with all other .yml configuration files, exclusively in Code Editor mode.
Key components of the project configuration file:
- Project name
- Version
- Directory structure
- Asset configurations