From .Net Metrics to Graphql Sub.
Tags: dotnet, monitoring, observability
Published: 2022-11-02 07:10:59

Metrics scenario:

- You are collecting metrics using `.Net Metrics API`
- Then process, aggregate and export metrics using  `Opentelemetry SDK` 


![.Net metrics telemetry](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pct9vcy1v2hxf3snzk0v.png)

Under the hood, `Opentelemetry SDK` connects to the `.Net Metrics API` by listening to registered meters.

A combination of both is recommended!

If you are creating an internal library, you probably do not want to equip all your `class libraries` with opentelemetry packages.


## .Net Metric != Opentelemetry Metric.

It is important to point out that things work a little differently on each side.

![Net vs Opentelemetry](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d26bztribjc32uup6414.png)

</br>

**.Net Metrics API**
- Provides just abstraction and events for telemetry/metrics. 
- No value store
- No aggregation
- No ready-made solutions
  
**Opentelemetry SDK**
- Provides extended implementation
- Aggregation
- Value store.
- Less code control (cant manipulate with the store)

</br>

## Counter as example

Let's take a look at how the `.Net` counter is handled from start to finish...

1) We define `Meter`. 
2) We register concrete counter under `Meter`.
3) We trigger a counter from code based on some event/action.

```c#
    // Meter instance
    var _meter = new Meter("MeterName","v0.1");

    // Counter creation
    var counter = _meter.CreateCounter<long>("InstrumentName", "Unit", "Description");

    // Triger on Event
    counter.Add(1)
```

It is important to understand that the **.Net counter has no value** and **has no idea about the previous state** 

You cannot query the current value, reset it, or interact with it.

![Net Counter Example](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x1l3eztzbkhj7bz3jehd.png)

### What happens if you call `counter.Add(Value)` multiple times?

- Registered listener callbacks are called with the current Add(..) value.
- No previous value or state is passed to the listener.
- No value is present between the calls

### Then why is it called a counter?

It`s just an abstraction with information about the type and how much to increase/decrease the value. (+1 ,+10, -5) by concrete listeners.

### How to get real counter behaviour?

**A: Manual - > Write code yourself**

- You need to implement a custom metric listener
- Based on the counter event callback, you need to update a variable that holds the counter reading for you.

Simple example:

![Internal state implementation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4r7atf6hmfy955gvxric.png)

>Advantage is that you have full code control.
  
**B: ReadyMade -> Use Opentelemetry SDK**
- Use Opentelemetry SDK to do this automaticaly for you and access values by custom readers/exporters.

>Advantage is that someboady else wrote this for you 


`Opentelemetry SDK` contains `AgregateStore`. For each metric, a cumulative, a sum, or a histogram value is automatically stored, depending on the type.

For this reason, readers/exporters can access counter value.

![Opentelemetry Agregation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/74pxpd7whoc0066mcg7d.png)

**All aggregates are stored in memory**! When the process is restarted, all values are lost 

## Value representation

All metric values are treated internally as `double` or `long`.

Depending on the source type, `Opentelemetry SDK` treats them as `double/long` and assigns the value to a structure. This ensures the same memory allocation for all types.

So if you create a `counter`, its value will end up as `long` in the aggregate memory.

From sources:

![Opentelemetry Metric Point Value](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tu4pxim5zodkk1ur1fbd.png)

This has simplified many SDK internal processes.

Forget the idea that a counter of type `int` saves some memory :)

## Unusal custom reader+exporter

- Readers are responsible for manipulating data and passing metrics to the exporter.
- Exporters receive a batch of metrics, and each metric may contain 1 or more metric points that must be exported.

Why do we need a custom?

In a project I was working on, I was looking for the easiest way to pass some real-time metrics to GraphQL subscriptions.

Lets have a look on example. The `servers` are IoT brokers for specific communication protocol.

![Opentelemetry to GraphQL](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/daloejjpnq14ib9mhdrx.png)

- There are several independent parts in the code and it was not possible to create a worker service for each to propagate values.
- In the future, there will be more "servers" and API normalization was necessary.

I end up creating a custom reader + exporter that reads metrics and propagate those values to Graphql subscriptions.

- Each part simply uses the abstract `.Net Metric API` + a base class to hold the metric in the required form.
- Reader and Exporter collect and push changes to Graphql Sub.
- All exported values are wrapped by a custom unified object with a dynamic value type property

Clients and Portal users can now easily access specific real-time metrics data. All other metrics/traces are exported using the default built-in exporters.

The metric name is converted to a subscription topic and allows filtering based on server UID or metric name.

You can find me on [GitHub](https://github.com/damikun).and feel free to ask questions. For this reason, readers/exporters cant access counter value.