How to collect metrics from node.js applications in PM2 with exporting to Prometheus
Tags: webdev, javascript, programming, node
Published: 2024-02-12 21:33:58

It’s no secret that for stable and reliable working of node.js applications, it’s necessary to monitor their performance and get useful insights from their metrics. It means being able to collect information about app state before issues appear to preventing failures.

In this article, I’d like to present a method for collecting metrics node.js applications from running in PM2 and exporting this data to Prometheus.

When you simply start a node.js application using the `node` command or through PM2 with the `pm2 start app.js` command the new application is launched as a single instance. Collecting and exporting any metrics in this case would not be difficult. To do this, you can install the [prom-client](https://www.npmjs.com/package/prom-client) package and add the metrics you need, for example, the number of requests to our application.

```javascript
import client from 'prom-client';
import { createServer } from 'http';

const registry = new client.Registry();
const PREFIX = `nodejs_app_`;

export const metricRequestsTotal = new client.Counter({
    name: `${PREFIX}request_counter`,
    help: 'Show total request count',
    registers: [registry],
});

// Start your application
// ...
nodejsapp.get('/*', async (req, res) => {
    metricRequestsTotal.inc();
});

// Start Prom server for export metrics
const promServer = createServer(async (req, res) => {
    res.setHeader('Content-Type', registry.contentType);
    res.end(await registry.metrics());
    return;
});

promServer.listen(9100, () =>
    console.log('Prom server started')
);
```

In this example, when your application is launched, it also stars metric exporting on an additional port `9100`. Of course you can use a separate URL with the main application, but it shouldn’t be public for users.

This approach will work as long as you don’t need to run multiple instances of the application in cluster mode. For collecting metrics in cluster mode, prom-client provides [a good example](https://github.com/siimon/prom-client/blob/master/example/cluster.js) with metric aggregation. But what if you’re running the application through the process manager PM2, where you don’t have access to the running cluster?

In one of [my previous articles](https://vexell.medium.com/pm2-module-to-monitoring-node-js-application-with-export-to-prometheus-and-grafana-43d4b958c563), I talked about the [pm2-prom-module](https://www.npmjs.com/package/pm2-prom-module), which allows collecting overall statistics for PM2 applications but couldn’t collect internal statistics from each application. However, starting from version 2.0, such limitation is no longer present, and all statistics from PM2 and within the application are provided through a single module.


o communicate between node.js application and [pm2-prom-module](https://www.npmjs.com/package/pm2-prom-module), you need to install the npm package [pm2-prom-module-client](https://www.npmjs.com/package/pm2-prom-module-client) in your application. As a result, the example above will look like this:

```javascript
import client from 'prom-client';
import { initMetrics } from 'pm2-prom-module-client';

const registry = new client.Registry();
const PREFIX = `nodejs_app_`;

const metricRequestCounter = new client.Counter({
    name: `${PREFIX}request_counter`,
    help: 'Show total request count',
    registers: [registry],
});

// Register your `registry` to send data to pm2-prom-module
initMetrics(registry);

// ...
nodejsapp.get('/*', async (req, res) => {
    // ...
    metricRequestCounter?.inc();
    // ...
});
```

Now pm2-prom-module will provide PM2 statistics and also internal statistics from your applications.

```bash
/// Common metrics from PM2

# HELP pm2_free_memory Show available host free memory
# TYPE pm2_free_memory gauge
pm2_free_memory{serviceName="my-app"} 377147392

# HELP pm2_cpu_count Show available CPUs count
# TYPE pm2_cpu_count gauge
pm2_cpu_count{serviceName="my-app"} 4

# HELP pm2_available_apps Show available apps to monitor
# TYPE pm2_available_apps gauge
pm2_available_apps{serviceName="my-app"} 2

/// Apps metrics
# HELP nodejs_app_request_counter Show total request count
# TYPE nodejs_app_request_counter counter
nodejs_app_request_counter{app="app1",serviceName="my-app"} 10
nodejs_app_request_counter{app="app2",serviceName="my-app"} 17
```

By default, all statistics are aggregated across all running instances in every app, but if you want detailed statistics for each instance, you can enable this through the module’s configuration parameter:

```bash
pm2 set pm2-prom-module:aggregate_app_metrics false
```

This module, unlike some others, allows collecting metrics separately for each application, and they do not intersect.

Overall, the statistics provided by PM2 are enough for basic application monitoring. However, in our case, for example, we wanted more detailed information about page rendering time or loading of certain data blocks. For such purposes, we used Histogram metric type.

```javascript
// ...
export const metricRequestTime = new client.Histogram({
    name: 'nodejs_app_page_execute',
    help: 'Time to processing request',
    registers: [registry],
    buckets: [0.1, 0.2, 0.3, 0.5, 0.7, 1, 2, 3, 5],
    labelNames: ['code', 'page'],
});

// ...
const endMetricRequestTime = metricRequestTime.startTimer();
// ...
endMetricRequestTime({ code: 200, page: 'Homepage });
```

In example above, we use 2 additional parameters: `code` (HTTP response code) and `page` (page identifier), which provide detailed statistics. For example, you can build such graphs in [Grafana](https://grafana.com/):

- Most frequently pages `sum(rate(nodejs_app_page_execute_count{serviceName=”$serviceName”,code=”200"}[5m])) by (page)`
- The number of HTTP (response codes) success or errors. `sum(rate(nodejs_app_page_execute_count{serviceName=”$serviceName”}[5m])) by (code)`
- The page load time specifying, for example, the 99th percentile. `histogram_quantile(0.99, sum(rate(nodejs_app_page_execute_bucket{serviceName=”$serviceName”, code=”200"}[1m])) by (page, le))`
- And many other graphs for each individual page.

By the way, I don’t recommend using the URL of the page as the value for `page` — it will significantly increase memory usage and the size of the response in Prometheus, and mess in the graphs. It’s better to use page identifier.

So, to summarize, we have a single point for collecting and serving statistics for all applications running in PM2, and a monitoring system built on Prometheus with functional dashboards in Grafana.

![pm2-prom-module in Grafana](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wraq0se9sl42qctbawxj.png)

If you like **pm2-prom-module**, I would appreciate if you could [star it on Github](https://github.com/VeXell/pm2-prom-module). Thanks for reading.