At the end of our development flow, there's just one thing left to do: merge our changes to the
main branch. With this step, we make the pipeline generally available and indicate that it is production-ready.
This section will explain GitOps and virtual data builds before merging our changes to the
Remember when we created a working branch back in section 3? At that point, we split off from the
main branch to start developing. In the GitOps paradigm, this is how we iterate on our pipelines.
Working in a feature branch means we leave the production version untouched. We only apply the changes to production once we are sure they do what we want. And if there's an unforeseen mistake, we can revert to a working state with a single command.
What's special about Y42 is that it doesn't just apply this paradigm to code but also to data builds. This means that when we merge a branch into
main, its data builds will also immediately be available on
main. And if we overlooked a mistake, we can instantaneously roll back to another data build.
We call this approach Virtual Data Builds. A build is virtual until you decide to merge it in, at which point it becomes the production build.
At this point, we can be confident that our pipeline works as we expect it to. So let's merge our changes to
- At the top center of the screen, click on the name of your branch (e.g.,
Merge branchfrom the dropdown menu
- Verify that the Merge points from
- Wait for the merge to complete
main, you should now see the entire pipeline configuration as we created it in the previous sections
- Verify from the data preview that all assets contain the right data
- In your data warehouse, look up the tables generated by Y42 and check if they contain the same data
- At the top center of the screen, press
Open Git historyto see all commits on the branch
- If something went wrong, you can right-click the commit and select
Revert committo roll back the changes
Outstanding! You have taken your first steps on Y42 and set up ingestion, a pipeline, a scheduler, and alerting. And with the merge to
main, you have brought everything live to production.
If you would like to dive further into the Y42's paradigms, take a look at the following blog posts: