How we converted our Cortex data to TSDB blocks - Peter Štibraný, Grafana Labs
Channel: CNCF [Cloud Native Computing Foundation]
Date: 2021-05-14

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

How we converted our Cortex data to TSDB blocks - Peter Štibraný, Grafana Labs

Cortex has recently gained support for storing data in Prometheus TSDB format (using so-called "blocks engine", also reusing some parts of Thanos). In Grafana Labs, we had hundreds of terabytes of data in previous format, and we wrote scalable tooling to convert it all to TSDB blocks. In the talk, I would like to present the tooling, show some details from Prometheus TSDB internals, and show how simple it is to generate TSDB blocks using Go code and what pitfalls to avoid.