Ingest data using Airbyte

Extract and load data into a Y42 space's dataset or schema.

Add an Airbyte source asset.

Overview

Y42 leverages Airbyte (opens in a new tab) to integrate data from external sources. For each data ingestion job, Y42 creates an instance of the Airbyte open-source connector (opens in a new tab). To pull data in Y42 you will need two define two objects:

• Source: For each Airbyte connector you will need to create a source object. Sources provide metadata information about external data sources, such as table or column-level information.
• Secret: Secrets hold information about authentication. When you set up a source, you don't need to worry about creating additional accounts, you can use your existing account with the data provider (e.g., username and password for database sources or an API token)

Every airbyte connectors allows various levels of configurations and features: incremental imports, CDC replication, and others. For a deeper dive into these features, please refer to the individual documentation pages for each source.

Add an Airbyte source

FAQ