Why, How to, and Issues: Tail-Based Sampling in the OpenTelemetry Collector - Reese Lee, New Relic
Channel: CNCF [Cloud Native Computing Foundation]
Date: 01/06/2022

Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Why, How to, and Issues: Tail-Based Sampling in the OpenTelemetry Collector - Reese Lee, New Relic

When you are running OpenTelemetry in production and your services are producing a firehouse of spans, the traditional and default head-based sampling approach won’t cut it. This is because traces are sampled at initiation, which can be useful for some environments, but for larger systems, it can mean you miss out on key trace data. This is where configuring the Collector to sample your traces after they have fully completed–tail-based sampling–becomes a great option. In this talk, you’ll learn about head- and tail-based sampling, and why the latter approach is useful for obtaining the highest level of granularity in troubleshooting. You’ll learn how to configure your OpenTelemetry Collector to do this, and see the implementation in a suite of microservices, with traces exported to Jaeger. You’ll also learn of the current issues with implementing tail-based sampling in the OpenTelemetry Collector in production so you can take the challenges into account for your own deployments.