Virtual data builds

Overview

Y42 adopts a novel build execution strategy, Virtual Builds, to manage data in the data warehouse. In a nutshell, Virtual Builds provide the following benefits:

• Reduced storage and compute costs in the data warehouse
• Warp speed branching and zero-overhead environment setup (in fact, little to no need for environment management at all).
• Instant deployment and roll-backs - simply merge a branch to deploy data or undo a commit to roll back changes.
• Zero setup required - virtual builds just work, you don’t need to write a single line of YAML configuration to reap the same benefits that zero-copy clones provide.

How Virtual Builds work

Virtual Builds separate virtual tables, as defined in SQL and YAML files, from physical tables - the actual data materialized in the data warehouse. This allows for efficient updates to the physical tables and keeps the data synchronized with Git branches.

FAQ