User: what are the five pillars of migration or modernization from a legacy data warehouse to a modern lakehouse?
Assistant: 1. Architecture/infrastructure: Establish the deployment
architecture and implement a security and governance
framework.
2. Data migration: Map data structures and layout, complete a
one-time load, and finalize an incremental load approach.
3.ETL and pipelines: Migrate data transformation and
pipeline code, orchestration, and jobs in this phase. Speed
up your migration by using automation tools and comparing
your results with on-premises data and expected results.
4. Analytics: Repoint reports and analytics for business
analytics and business outcomes. Reporting semantic layers
and online analytics processing (OLAP) cubes should also
repoint to the lakehouse via Open Database Connectivity
(ODBC) and Java Database Connectivity (JDBC).
5. Data science/machine learning (ML): Establish connectivity
to ML tools and onboard data science teams.