Kafka Streams and Protobuf stream processing at trivago
Channel: FOSDEM
Date: 06/03/2018

by Clemens Valiente

At: FOSDEM 2017

A lot of components in the Kafka and hadoop ecosystem assume you are workingwith avro messages. In this talk I will highlight some of the advantages andchallenges of stream processing with protobuf encoded messages, and why itmight be worth using them.

A powerful feature of protobuf messages is its handling of unknown fields, andthe ability to partially parse messages with different schema versions or noschema at all. This helps in stream processing when you don't actually need toknow all message fields in order to process and forward them.


Room: H.2213
Scheduled start: 2017-02-04 16:30:00