Beam Summit 2022 - Migration Spark to Apache Beam/Dataflow and hexagonal architecture + DDD
Channel: Apache Beam
Date: 03/08/2022

Session presented by Mazlum Tosun at Beam Summit 2022.

In his previous customer, Mazlum, Tech lead GCP and Data, did a code migration from Spark/Dataproc and Apache Beam/Dataflow. I proposed a hexagonal architecture + Domain-driven design with Apache Beam, in order to isolate business code (bounded context/domain) and technical code (infrastructure). This architecture is used with code decoupling and dependency injection. He used Dagger2 and he explains why.

The purpose is to show a Beam project with this architecture and explain why it’s interesting. One example with Beam Java and another with Kotlin will be shown. The Kotlin version uses data classes and extensions to have a more concise and expressive code. Mazlum also uses this architecture in his actual customer in production.

Link to slides here 👉 https://2022.beamsummit.org/slides/MigrationSparkBeam.pdf