Sources

Get an overview of Sources in Y42.

Sources viewed in Data Catalog mode.

Overview

In Y42, sources have a broader application compared to dbt. Apart from declaring raw data in the data warehouse that you do not control, sources in Y42 can also be used to perform ingestion tasks using Airbyte. This adds an additional layer of functionality and control for managing ELT data pipelines.

There are three types of source assets available:

1. Native Reference source, which is used to reference data in your data warehouse.
2. Airbyte sources, which allow you to ingest data from third-party services using Y42-managed Airbyte connectors.
3. Fivetran, which allows you to trigger and manage Fivetran connector syncs.

Native Reference source

Native reference sources create views of other datasets, in your Y42 space's dataset (BigQuery) or schema (Snowflake). Referencing data involves pointing to the location of the data in its origin source rather than replicating it. This method is particularly useful when regulations or data volume make it impractical to copy the data into your Y42 space's dataset/schema.

Airbyte sources

Y42 provides a managed service for running data ingestion jobs. To make this possible, Y42 leverages Airbyte to securely extract and load data from external sources to Y42.

Integrate with Fivetran

If you utilize Fivetran for your data ingestion process, you can seamlessly integrate with Fivetran to trigger and manage syncs.