The power of type classes in big data ETL – Zhenhao Li
Channel: Typelevel
Date: 08/07/2017

In this talk, we will explore a type driven approach of big data ETL in Spark. Through code snippets, we will see how to express data processing logic with type classes and singleton types using Shapeless, and how to build a higher level DSL over Spark to make the logic easy to read from the code.

* Material *

Slides: https://cdn.rawgit.com/Zhen-hao/typelevel2017/ae9d8c18/index.html

* Event * 

Typelevel Summit Copenhagen, June 2017: http://typelevel.org/event/2017-06-summit-copenhagen/

* About Speaker *

Zhenhao Li is a data engineer and data scientist at Connecterra, a data science and IoT startup. He is responsible for making scalable data processing jobs and pipelines and making sure data science insights are generated and delivered in real time. Before joining Connecterra, Zhenhao worked for Accenture in the area of big data and IoT technology consulting, helping major clients to adapt new technologies such as Kafka and Flink, the immutable data paradigm, and functional programming to gain business value faster. He holds a bachelor degree in software engineering and a master degree in logic. He was doing a PhD in mathematical logic at the University of Amsterdam when decided to change his career path to big data and data science. He loves functional programming, and Scala is his primary language for engineering work.

* Licensing *

This recording is licensed under CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/).