How Much Overhead How to Evaluate Observability Agent Performance   Braydon Kains, Google Cloud
Channel: CNCF [Cloud Native Computing Foundation]
Date: 2023-11-08

"How Much Overhead?": How to Evaluate Observability Agent Performance - Braydon Kains, Google Cloud

Observability agents, such as the OpenTelemetry Collector, Fluent Bit, Vector, and more, are highly robust and do a lot of different kinds of work to achieve their amazing outcomes. When adding such a crucial component to your infrastructure, you have to ask: how will this impact my workload?
There is no straightforward answer to this seemingly simple question. In this talk, I will go into the common types of work these agents do, and how they all factor into an agent’s resource usage and performance:

What really is overhead?

Ingestion: reading from files, scraping system information, ingestion over network calls

Processing: parsing, modifying, and transforming data

Exporting: Sending the data to some kind of backend

Important performance variables: Batching, buffering, resource limiting, backpressure

After attending this talk you should feel empowered to better understand what an observability agent’s overhead and performance really means and how to evaluate it.