Measurements with MetricKit
Tags: swift, ios, metrics
Published: 2024-04-27 21:47:09

# Intro
Sometimes you need to measure something in the app. It can be different things like app launch time or just custom time spent between certain points of execution. One of the most convenient tools in this regard is MetricKit framework. With its help, you can measure different performance and diagnostic metrics, monitor regressions and identify problems in your app. The article describes the basic setup, output data format and possible options to process it. Also, it describes custom measurements and events reporting with MetricKit.

# Getting started
Let’s start with a simple example: how to measure your app’s launch time?
## Setup
First, we need to define an object responsible for handling info from the framework and implement `MXMetricManagerSubscriber`. This protocol contains two optional methods:
`optional func didReceive(_ payloads: [MXMetricPayload])`
`optional func didReceive(_ payloads: [MXDiagnosticPayload])`
In this article we will focus only on the first one that provides metrics.
According to [docs](https://developer.apple.com/documentation/metrickit/mxmetricmanagersubscriber/didreceive(_:)-3zq5g):

> The system calls this method at most once per day. It’s safe to process the payload on a separate thread.

In real life you can open an app every day but the callback will be called once in three days, it only guarantees to call the method no more than once a day. in my experience, it is called on average once every 2-3 days.
Let’s create a separate class responsible for metrics reporting:
```

import MetricKit

final class MetricReporter: MXMetricManagerSubscriber {
	func didReceive(_ payloads: [MXMetricPayload]) {
		// TODO: Report metrics
	}
}
```
Those methods are called in the background so they don’t affect app performance. Later I will explain how to parse data from those metrics. Now we need to have this object during the whole app lifecycle so let’s add it to AppDelegate.swift:

```
@main
class AppDelegate: UIResponder, UIApplicationDelegate {
    private lazy var metricReporter = MetricReporter()
    func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
        MXMetricManager.shared.add(metricReporter)
        return true
    }
}
```
So now we are ready to try payload example.

## Payload overview

To get a sample for MetricKit we can use Xcode `Debug -> Simulate MetricKit Payloads`, but it is important to use a real device, for a simulator this option is disabled.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/argbbokpv1br9px9y4rt.png)

Every `MXMetricPayload` contains different instances of `MXMetric` subclasses such as `MXAppLaunchMetric`, `MXAppExitMetric`, `MXMemoryMetric` and others. All of them in turn contain different sets of following classes and structures:

`Measurement` structure represents value including information about units. This is how it looks like in the example of peak memory usage:

```
"peakMemoryUsage" : "200000 kB"
```

`MXHistogram` class represents the number of times the measured value falls into a specific range of possible values. This is how it looks in json form in the example of the launch time of the app:

```
"histogrammedTimeToFirstDrawKey" : {
      "histogramNumBuckets" : 3,
      "histogramValue" : {
        "0" : {
          "bucketEnd" : "1010 ms",
          "bucketCount" : 50,
          "bucketStart" : "1000 ms"
        },
        "1" : {
          "bucketEnd" : "2010 ms",
          "bucketCount" : 60,
          "bucketStart" : "2000 ms"
        },
        "2" : {
          "bucketEnd" : "3010 ms",
          "bucketCount" : 30,
          "bucketStart" : "3000 ms"
        }
      }
    }
```
`MXAverage` class represents average value including information about sample count and standard deviation. This is how it looks like in the example of average pixel luminance:

```
"averagePixelLuminance" : {
    "averageValue" : "50 apl",
    "standardDeviation" : 0,
    "sampleCount" : 500
  }
```

# Processing
There are two different approaches for metrics data processing: preprocess data on a device and send it or send data as it is. Both have disadvantages and advantages, let’s take a look at the details.

## Processing on device
Process data on mobile and send result values to your analytics solution. In general, you need to transform metrics-specific data structures into simpler ones, for example extract average value from MXHistogram data structure. A great example of this approach can be found [here](https://www.avanderlee.com/swift/metrickit-launch-time/). Just a few important things to remember:
- Remember to check `includesMultipleApplicationVersions` field to filter out reports containing data from different versions
- Remember to filter out abnormal values with `isNan` , `isNormal` or `isFinite` checks. In the previously mentioned example author used `isNan` but you can also utilise `isFinite` or `isNormal`, but remember that zero is not a normal number

In this case, you select on mobile the exact data you want to send for analyzing. Unfortunately, if you make any mistake on this step it would be quite hard to catch since you don’t have access to raw data.

## Offload processing computation off mobile devices
Offload processing computation off mobile devices and send data as it is for processing. In [this](https://nshipster.com/metrickit/) great example the author uses a web service to process metric payloads. The main advantages of this method are:

1. All raw data is available so if later you discover any issues with your processing you can fix it without any consequences
2. At any point in time you can access data from the past

The main disadvantage here is the higher complexity.

# Custom metrics
Apart from predefined measurements MetricKit also supports custom measurements and event tracking with `mxSignpost`
## Custom measurements
This is how you can do it on the example of applying some heavy operation:

```
func apply() {
    // create log handler
    let handle = MXMetricManager.makeLogHandle(category: "ApplyCategory")
    mxSignpost(.begin, log: handle, name: "ApplyTrace")
    // critical code section begins
    // ...
    // critical code section ends
    // end measuring
    mxSignpost(.end, log: handle, name: "ApplyTrace")
}

func cancel() {
    let handle = MXMetricManager.makeLogHandle(category: "CancelCategory")
    mxSignpost(.begin, log: handle, name: "CancelTrace")
    // ...
    mxSignpost(.end, log: handle, name: "CancelTrace")
}
```
Though it looks pretty simple there are several important things to highlight:

- It is important to specify the same name in mxSignpost calls, otherwise, you will not get results in a report
- There is special [note](https://developer.apple.com/documentation/metrickit/mxsignpostmetric) about this API usage:

> The system limits the number of custom signpost metrics saved to the log in order to reduce on-device memory overhead. Limit the use of custom metrics to critical sections of code.

As a result, you get something similar to the following piece in result payloads:

```
{
  "signpostMetrics" : [
    {
      "signpostIntervalData" : {
        "histogrammedSignpostDurations" : {
          "histogramNumBuckets" : 2,
          "histogramValue" : {
            "0" : {
              "bucketCount" : 1,
              "bucketStart" : "0 ms",
              "bucketEnd" : "99 ms"
            },
            "1" : {
              "bucketCount" : 1,
              "bucketStart" : "100 ms",
              "bucketEnd" : "199 ms"
            }
          }
        },
        "signpostCumulativeCPUTime" : "262 ms",
        "signpostAverageMemory" : "64433 kB",
        "signpostCumulativeLogicalWrites" : "748 kB"
      },
      "signpostCategory" : "ApplyCategory",
      "signpostName" : "ApplyTrace",
      "totalSignpostCount" : 2
    },
    {
      "signpostIntervalData" : {
        "histogrammedSignpostDurations" : {
          "histogramNumBuckets" : 1,
          "histogramValue" : {
            "0" : {
              "bucketCount" : 81,
              "bucketStart" : "0 ms",
              "bucketEnd" : "99 ms"
            }
          }
        },
        "signpostCumulativeCPUTime" : "295 ms",
        "signpostAverageMemory" : "211037 kB",
        "signpostCumulativeLogicalWrites" : "168 kB"
      },
      "signpostCategory" : "CancelCategory",
      "signpostName" : "CancelTrace",
      "totalSignpostCount" : 81
    }
  ]
}
```
As you can see in the attached payload there is histogram data that allows you to collect information about the execution time for specific code sections.  

## Events tracking 
Apart from `begin` and `end`, you can use `event` `OSSignpostType` and in this case in a payload you will receive a number of times this event occurred. Here is an example:

```
let handle = MXMetricManager.makeLogHandle(category: "TestViewController")

override func viewDidLoad() {
	super.viewDidLoad()
	mxSignpost(.event, log: handle, name: "viewDidLoad")
}

override func viewWillAppear() {
	super.viewWillAppear()
	mxSignpost(.event, log: handle, name: "viewWillAppear")
}
```
With this code we can get the following payload part:

```
"signpostMetrics" : [
    {
      "signpostCategory" : "TestViewController",
      "totalSignpostCount" : 5,
      "signpostName" : "viewDidLoad"
    },
    {
      "signpostCategory" : "TestViewController",
      "totalSignpostCount" : 5,
      "signpostName" : "viewWillAppear"
    }
  ]
```
As you can see this can be utilised as an event-collecting tool to build various types of event funnels

# Conclusion

MetricKit provides wide options of reporting tools to monitor:
1. Predefined metrics like application launch time, network usage, memory usage, etc.
2. Custom metrics recorded with `mxSignpost` for critical code sections
3. Events tracking with `mxSignpost` for critical events

These options can be utilised in various scenarios such as performance related improvements or event funnels.