Instrumenting Node.js internals
Tags: node, instrumentation, javascript
Published: 2020-09-30 15:31:17

---
title: Instrumenting Node.js internals
published: true
description: Introduction of a tool to get insights on how your Node.js application is using the network or the filesystem, among others
tags: nodejs, instrumentation, javascript
cover_image: https://dev-to-uploads.s3.amazonaws.com/i/117xly9nsh2rli4oy74e.jpg
---
Back with one of my clients, I remember a specific process that every engineer was too afraid to change because it was prone to failures and interacted with a few different parts of the infrastructure. It was a queue worker and **I was told that whenever the worker just froze someone would had to restart it manually**. I initially thought it made no sense but after a few failed attempts trying to reproduce the scenario I started to think different. I didn't had too much time to spend on fixing it, I was hired to focus on a newer architecture but it always resonated with me the lack of tooling to quickly get an overview of what a process is doing.

Fast-forward to today, I still think there's a lot of room to cover in the developer tooling space. With this in mind I've worked on an experiment called [instrument](https://github.com/a0viedo/instrument) that can collect information from your Node.js process and aggregate it in different ways.


## Choosing your instrumentation spot
There are a few places where I can imagine you could intercept internal calls and collect data:

![drawing of different layers that could be instrumented](https://dev-to-uploads.s3.amazonaws.com/i/radb11htn1cg4kdjtmqt.png)

Intercepting system calls is doable and would also work for other runtimes/VMs not just Node, but you would have to target a specific operating system and it's generally more complex. With C++ (through _node-gyp_), you won't have to target a specific operating system but you will add some additional requirements ([appmetrics](https://github.com/RuntimeTools/appmetrics) falls in this space).
Intercepting calls from the javascript realm it's not only doable but also works great for portability: you don't have to install a different runtime or need additional requirements, you monkey-patch on top of it.

## Configurability
I wanted to support different ways of running the tool for your existing application, much like `dotenv` does: include it programmatically or add an `-r instrument/config` flag for your command.
On top of these two alternatives, I've also added support for a standalone configuration file ("instrument.config.js") or you can also pass your own configuration as a parameter:

```javascript
// at your entrypoint file
require('instrument')({
  summary: true,
  output: 'instrument-logs.txt'
})
```

## My server is not a one-off
For cases when you're running a server and your process doesn't runs to completion you could still get the instrumentation running and separate it's output from your original process.

{% asciinema 362659 %}

## Reducing external noise
In most Node.js applications it's not uncommon to have a large list of dependencies, and sometimes you may not be interested in instrumenting the calls originated by your dependencies. Based on what you're looking for, you can switch on/off these logs by using the `dependencies` property from the configuration.

You can also choose the modules you want to instrument instead of having them enabled by default:

```js
require('instrument')({
  modules: ['http', 'https']
})
```

The configuration above will only instrument the modules `http` and `https`.

## Require-tree
I figured another useful feature would be to instrument the required modules. A great tool for this is [madge](https://github.com/pahen/madge), which focus on modules of your own and not `node_modules` dependencies. For my case, I choose a slightly different approach: you can choose to include the dependencies required by your dependencies or only to include one level of dependencies.

As an example, I exported the tree for running `npm ls` and graphed it using D3.js to end up with this [SVG](https://gist.githubusercontent.com/a0viedo/aa9c843e0ce3b4c7485b894530e2844d/raw/22e93c64ca9b7a015e18885876500afdfd78e7fe/temp.svg).

## Measuring the overhead
Even if this experiment is a thin layer on top of some APIs it **could** have unexpected outcomes in terms of performance. I took [fastify-benchmark](https://github.com/fastify/benchmarks) and ran a few of the most common frameworks for HTTP servers with and without `instrument` enabled:

<table>
<tbody><tr><th align="center">Library/framework</th>
<th>Throughput difference</th>
</tr>
  <tr>
    <td>built-in http
</td>
<td align="center"><center>-11.32%</center></td>
  </tr>
<tr>
    <td>connect
  </td>
<td align="center"><center>-4%</center></td>
</tr>
<tr>
    <td>express
</td>
<td align="center"><center>-0.37%</center></td>
  </tr>
<tr>
    <td>fastify</td>
<td align="center"><center>-8.8%</center></td>
  </tr>
<tr>
    <td>hapi
</td>
<td align="center"><center>-0.72%</center></td>
  </tr>
</tbody></table>

Another benchmark I tinkered with was [benchmarks-of-javascript-package-managers](https://github.com/pnpm/benchmarks-of-javascript-package-managers) but for some reason I couldn't get consistent results out of the instrumented processes.


I see this mostly as a thought exercise, since my goal with this experiment is mostly for local development environments on which the performance overhead shouldn't matter that much.

## Room for improvement
There were a lot of built-in modules or globals I didn't intercepted because I failed to see any added value (like `process`, `path`, `util` and the list goes on). It doesn't mean that those couldn't be instrumented, it just would take more time.
An interesting feature would be to measure the time for each of the calls that are instrumented but would require some extra work on figuring out how to graph the data to make sense of it.
The code is Open Source in case you want to take a look or if you found a bug:

{% github a0viedo/instrument no-readme %}

Another relevant question would be: could you monkey-patch other objects or functions that are inherent to JavaScript? The answer is _yes_!

```js
const originalReference = Promise.resolve;
Promise.resolve = (...params) => {
  console.log('calling Promise.resolve', params);
  return originalReference.call(Promise, ...params);
}
```
It doesn't mean you **should**, though. Not only I can't think of a good reason for anyone to do it but it could also clutter your stack traces severely.
In the following weeks I want to explore exporting hooks from `instrument` to be able to build a warning mechanism for filesystems with strict permissions.

----------------------------------------
_Shout-out to Marco Buono for donating the "instrument" package name!_