Robert Metzger - Tiny Flink — Minimizing the memory footprint of Apache Flink
Channel: Plain Schwarz
Date: 2023-06-21

Apache Flink has been designed for, and is mostly used with large-scale real-time data processing use-cases. Companies report about TBs of data being processed per second, or TBs of state in huge clusters.

But what if you need to process low-throughput streams? Running a full, distributed Flink cluster might be an overkill, as there’s quite a bit of overhead for distributed coordination.

In this talk, we’ll explore options to reduce your resource footprint. We’ll dive deeper into Flink’s MiniCluster, allowing you to run Flink in-JVM for integration tests, as a micro service or just a small processing your data in Kubernetes. We will also discuss lessons learned from running MiniCluster in production for a service offering Flink SQL in the cloud.

Attend this talk if you want to learn about Apache Flink and its various options to deploy and configure it.

Speaker: Robert Metzger

More: https://2023.berlinbuzzwords.de/sessions/?id=BWNJZN

Web: https://2023.berlinbuzzwords.de/
Fediverse: https://floss.social/@berlinbuzzwords
Linkedin: https://www.linkedin.com/showcase/13978964/
Twitter: https://twitter.com/berlinbuzzwords