Virtual data builds
Y42 adopts a novel build execution strategy, Virtual Builds, to manage data in the data warehouse. In a nutshell, Virtual Builds provide the following benefits:
- Reduced storage and compute costs in the data warehouse
- Warp speed branching and zero-overhead environment setup (in fact, little to no need for environment management at all).
- Instant deployment and roll-backs - simply merge a branch to deploy data or undo a commit to roll back changes.
- Zero setup required - virtual builds just work, you don’t need to write a single line of YAML configuration to reap the same benefits that zero-copy clones provide.
Virtual Builds separate virtual tables, as defined in SQL and YAML files, from physical tables - the actual data materialized in the data warehouse. This allows for efficient updates to the physical tables and keeps the data synchronized with Git branches.