Building Robust Production Data Pipelines with Databricks Delta
Channel: Engineers.SG
Date: 2019-05-29

"Building Robust Production Data Pipelines with Databricks Delta" - (optional hands-on experience: prepare laptop with Chrome/Firefox browser and register on Databricks Community Edition). Following open-source announcement of Delta Lake, this walk-through will prove insights on how Delta.io employs co-designed compute and storage and how it is compatible with Spark API’s. Delta Lakes power high data reliability and query performance to support big data use cases, from batch and streaming ingests, fast interactive queries to machine learning. This tutorial will discuss requirements of modern data pipelines, the challenges data engineers face when it comes to data reliability and performance and how Delta can help. Through presentation, code examples and notebooks will be shared.

Speaker for Talk #2: Arseny Chernov, joined Databricks in 2018, and is APJ leader for Partner Solutions Architecture, based out of Singapore. Acting as a customers’ conduit to Corporate Headquarters of Databricks, Arseny supports tactical and strategic initiatives in complex data and cloud environments with all available resources and knowledge, - for the business benefits and the best user experiences.

Event Page: https://www.meetup.com/Spark-Singapore/events/261637175/

Produced by Engineers.SG
Recorded by:  Michael Cheng

Help us caption & translate this video!

https://amara.org/v/pGgX/