Detecting server-side memory leak in Vue app
By: Dmitry Savchenkov
Tags: Js, Vuejs, Nginx, Memory Leak, Nodejs

What happened

Suppose there is a Vue SSR app with lots of components and pages that runs under nginx + pm2 + Express and have pretty decent user activity. New release has been deployed… and boom! In a while pm2 process stops with following error in the log:

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed — JavaScript heap out of memory

Preparing test environment

In my case it was reproducing on production server only. A similar test environment is needed ideally with the same traffic and user activity.

nginx-mirror is to help!

Say production app is running on port=8080. We can mirror real user requests to the app on a different port (8081) on the same server:

Once the test environment has been set up with the same traffic as on production and the leak is reproducing there we can start experimenting!

Tracking memory usage

Let’s use process.memoryUsage() to see what’s happening with the memory heap. We can collect heapUsage data into a file to be able to build a chart to see how memory consumption is changing during time.

I’ve added the following debug code straight before res.end() in server.js

heap.log would have the following format

Request-number RSS heapTotal heapUsed external arrayBuffer

The most interesting thing here is heapUsed . You can read more about these measures in documentation to process.memoryUsage()

The real data will look like this:

1 81.42 47.89 32.51 1.84 0.45

2 87.28 48.27 37.04 2.65 1.27

3 87.59 48.27 37.23 2.66 1.27

It’s time to make a graphic representation of this values!

The simplest way for me to build a chart was to just copy this data to google spreadsheets and build a line-chart. I’ve got something like this (y-axis values are in megabytes):

heapUsed growth linearly, it’s definitely a memory leak!

A good idea is to compare heapUsed values with previous working version with the same user activity:

No more than 200Mb for the same time! Now we’ve got the reference.

It’s easy to compare values in these two logs by request numbers. In my case, starting from request number 1000 the difference was significant and the trend was pretty clear.

Identifying the request

First of all I’ve commented all my routes and redirected all traffic to static page (404) to check whether some specific page causing this leak or it seats somewhere globally in the app…

Leak is disappeared, so it’s caused by some page!

Then I’ve started to enabling routes by groups from the simplest to the complex ones, until I found the group and the page causing the leak. By using reference values I just have to wait for about 1000 requests.

Since we know the page — it’s easy to create load test for this specific page and quickly identify the exact place of leaking.

Load tests

It’s pretty easy to make load tests with k6.

After installing it locally I‘ve created a very simple script to test the leaking page:

It loads the page with 200 virtual users for 1 minute. In my case that was enough to see the trend and reproduce the leak.

Identifying the exact component

After using load test it was pretty easy to find the exact component by disabling different parts of the page. In my case the leak was in this component