From HelloWorld to Configurable and Reusable Apache Spark Applications In Scala
Channel: Oliver T
Date: 2019-04-20

We can think of a Spark application as the unit of work in complex data workflows. Building a reusable and configurable Spark application comes with its own challenges, especially for developers that are just starting in the domain. 
The presentation will show to the new Scala developers in Apache Spark how the spark-utils framework enables production-readiness and reusability of their application.
Spark applications are seen as the units of work in complex data workflows. Reusability, parametrization and configuration of application code presents challenges. Solving these will allow the developer to focus on value adding work instead of mundane tasks such as dealing with configuration parameters, initializing the SparkContext or even kinking-off a new project.
The presentation will describe the journey from the first steps into Apache Spark all the way to a simple framework that can help kick-off an Apache Spark project very easy, with a minimal amount of code. The resulting framework is already available and ready to use, as an open source project.

References
spark-utils    https://github.com/tupol/spark-utils
spark-utils-demos   https://github.com/tupol/spark-utils-demos
scala-utils    https://github.com/tupol/scala-utils
spark-apps.seed.g8   https://github.com/tupol/spark-apps.seed.g8
spark-tools    https://github.com/tupol/spark-tools
Lightbend Config   https://github.com/lightbend/config
Giter8    http://www.foundweekends.org/giter8/
Apache Spark    http://spark.apache.org/
ScalaZ     https://github.com/scalaz/scalaz
Scala     https://scala-lang.org