Transform data in Data Editor mode
Overview
Let's use the Code Editor mode and use VSCode in the browser to add a stg_payments
model on our dev
branch.
Instructions
Add a model asset in Code editor
mode
Switch to the Code editor
mode in the top left corner by clicking on the Data catalog
dropdown
Create a SQL and YML file
-
Open the
models
folder -
Create new file:
stg_payments.sql
-
Insert the following SQL query:
_21with source as (_21_21select * from {{ source('raw_jaffle_shop_data', 'raw_payments') }}_21_21),_21_21renamed as (_21_21select_21"id" as payment_id,_21"order_id" as order_id,_21"payment_method" as payment_method,_21_21-- `amount` is currently stored in cents, so we convert it to dollars_21TO_NUMBER(REPLACE("amount", ',', '')) / 100 as amount_21_21from source_21_21)_21_21select * from renamed -
Create new file:
stg_payments.yml
-
Insert the following YML:
_17version: 2_17_17models:_17- name: stg_payments_17columns:_17- name: payment_id_17description: ""_17data_type: STRING_17tests: []_17- name: payment_method_17description: ""_17data_type: STRING_17tests: []_17- name: order_id_17data_type: STRING_17- name: amount_17data_type: NUMERICMake sure you save both files.
-
Preview the model by clicking on
Query Data
on the right side of the bottom drawer or by using the hotkey CTRL + RETURN while having your cursor in the SQL code. Verify that you can see the renamedorder_id
and convertedamount
column.
Commit the SQL and YML file
- Click on the
Commit & Push
button at the top, you should again see two files:stg_payments.sql
stg_payments.yml
- Name your Commit:
adding stg_payments in code editor mode
- Commit your changes
- Wait for the pre-configured checks
Build the table
- Click on the
stg_payments.sql
file in the left file explorer - Open the bottom drawer
- Navigate to
Build
- You should see a pre-defined build command, if you don’t see a build command, enter
y42 build -s stg_payments
- Click on
Build now
- Observe the build job and wait for the job to be
Ready
Go back to the Data catalog
mode, you should automatically see your newly created stg_payments
asset’s query and columns metadata defined in the stg_payments.yml
View materialized tables in your warehouse
- Navigate to your warehouse
- Navigate to the Y42 database / project
- Open the
y42__{space_name}__dev
schema / dataset Verify that you can see your newly materializeddev
table:stg_payments
Merge the changes into main
- Go back to Y42
- Click on the three dots
⋯
next to the refresh button in the top bar - Click on
Merge branch
- In the opened pop up, you should see
dev
being merged intomain
and the following message:Branch can be automatically merged (1 commit behind)
- Click on
Merge
You will automatically be redirected to yourmain
branch
Up next
Staging layer done! Let's take a step back and take a look at what you've built using the Lineage
mode.