Event Streaming and the Data Communication Layer
Channel: The ASF
Date: 22/10/2020

Event Streaming and the Data Communication Layer
Adam Bellemare

A presentation from ApacheCon @Home 2020
https://apachecon.com/acah2020/

Streaming technologies unlock decoupled, near real-time services at scale. The most important part of any streaming platform is the event-broker (eg. Apache Kafka or Pulsar) as it plays the role of the Data Communication Layer (DCL). Many organizations fail to grasp the importance of the DCL and often relegate it to the role of a simple asynchronous message queue, leaving their key business domain events locked away in monolithic data stores. This is one of the many pitfalls that will be covered in this presentation, along with strategies and tipss for avoiding them. A well constructed DCL decouples both the ownership and production of data from the downstream services that require access to it. Access to clean, reliable, structured, and sorted data streams enables extremely powerful event-driven patterns. Data becomes much easier to access and no longer relies upon the producer's implementation to serve disparate business requirements. Teams and products can organize much more clearly along business bounded contexts, and modular, disposable, and compositional services become extremely easy to build and test. This presentation covers the best practices, responsibilities of the various actors, recommendations about specific technological implementations, and both the organizational changes required and those that will occur as a result of a reliable DCL implementation.

Adam Bellemare is the author of Building Event-Driven Microservices (O'Reilly, 2020). He has been working on event-driven architectures since 2010. His major accomplishments in this time include building an event-driven processing platform at BlackBerry, driving the migration to event-driven microservices at Flipp, and most recently, starting a new role to improve event-driven architectures at Shopify. He has contributed to both Apache Avro and Apache Kafka and is a keen supporter of the open-source community.