Disintegrated Telemetry: The Pains of Monitoring Asynchronous Workflows - Johannes Tax, Grafana Labs
Channel: CNCF [Cloud Native Computing Foundation]
Date: 2024-03-21

Don't miss out! Join us at our next Flagship Conference: KubeCon + CloudNativeCon North America in Salt Lake City from November 12 - 15, 2024. Connect with our current graduated, incubating, and sandbox projects as the community gathers to further the education and advancement of cloud native computing. Learn more at https://kubecon.io

Disintegrated Telemetry: The Pains of Monitoring Asynchronous Workflows - Johannes Tax, Grafana Labs

Different components involved in asynchronous workflows typically communicate via messages or events. Instrumenting and observing such workflows often results in disintegrated telemetry pieces (metrics and traces), from which it is hard to deduce the lifetime and impact of a message or event. Distributed Tracing offers two different solution approaches to this problem: strong correlation via parent/child relationships, and weak correlation via links. I show how each solution approach is broken in its own way, and provide insights that help you to choose the least broken solution for your scenario. Finally, to show some light at the end of the tunnel, I give an overview of standardization efforts in this space, including W3C context propagation drafts for messaging protocols, and the messaging semantic conventions created by the OpenTelemetry messaging workgroup, which I'm leading.