Running cpu-heavy tasks with nodejs worker-threads
By: Christoph Salvisberg
Tags: Performance, Threading, Nodejs

Introduction

Due to the nature of nodeJS running all code/requests in a single thread, this can easily lead to latency in a server environment, as additional requests have to wait until the only thread is available to process their need.

To address this issue, we decided to extract the CPU heavy part of the excel sheet creation into a separate worker process — a feature of nodeJS which is considered stable since Node12: https://nodejs.org/docs/latest-v14.x/api/worker_threads.html#worker_threads_worker_threads.

The challenge

The challenge we faced in our nodeJS Server (koaJS) application is that the end-user might request Excel file to be downloaded. This file is currently transfered as base64 encoded string. There are multiple aspects here that are potentially cpu intensive: creating the datastructure of the excel file & writing it to the buffer (which will also include zipping, as *.xlsx files are zipped) and converting it to a base64 string.

Also the data to be exported in the excel file is not residing in our system, so we have to request it over the API of another system. This is considered an I/O heavy task, which are handled well by the nodeJS architecture and is fine to be kept on the worker thread.

When having all those things ready, we just need to ensure to kick off our worker thread with all the needed information and wait for it’s result to return it to the client. Let’s see how we do that

The solution

The central function that we want to run on the worker thread is in our case called “buildPersonExcel”.It generates the excel sheet, writes the file to the buffer and converts it to a base64 string (to be sent to the client). This can be replaced with any other cpu intensive task (i.e. prime numbers calculation) on that you need with it’s matching parameters and return type.

const buildPersonExcel = async (

persons: Persons[]

): Promise<string> => {

const workbook = new ExcelJS.Workbook(); const sheet = workbook.addWorksheet('My Sheet');

sheet.columns = [

{ header: 'Id', key: 'id', width: 10 },

{ header: 'Name', key: 'name', width: 32 },

{ header: 'Surname', key: 'surname', width: 32 },

]; persons.forEach((person, index) => {

const row = sheet.getRow(index + 2);

row.getCell(1).value = person.id;

row.getCell(2).value = person.name;

row.getCell(3).value = person.surname;

}); const buffer = await workbook.xlsx.writeBuffer();

const base64String = buffer.toString('base64');

return base64String;

};

The next step is that through the call hierarchy coming from the clients request, we will need to trigger this excel file creation. For that first the persons need to be loaded from the external person service. The next step is to wrap the worker thread execution into our own workerPromise, so we can easily await it’s result.

The worker will be created with the new Worker(…) command, pointing to the current filename and taking the prepared parameters as data. As next step, the promise’s resolve/reject are conneted to the worker callbacks. Last of all, we launch the worker by posting the message.

Now the result can conveniently be awaited!

interface ExcelData {

persons: Person[];

} public createExcelSheet = async (): Promise<string> => {

// access persons from an external resource

const persons = await this.personService.fetchPersons(); const workerData: ExcelData = { persons };

const workerPromise = new Promise<string>((resolve, reject) => {

const worker = new Worker(__filename, {

workerData: workerData,

});

worker.on('message', resolve);

worker.on('error', reject);

worker.on('exit', (code) => {

if (code !== 0) reject(new Error(`Worker stopped with exit code ${code}`));

});

worker.postMessage(workerData);

});



const result = await workerPromise;

return result;

};

Now there is one missing aspect to it all: Where is our buildPersonExcel function actually being called? There needs to be in the same file (as we spawn the worker on __filename) as the createExcelSheet some global code that needs to be executed. Therein we check if it is the mainThread or not — on the mainThread, we dont want to trigger (or rather watch for) the workerprocess execution.

Simply listening on the parentPort for triggered messages, we can run with the transfered data the cpu heavy buildPersonExcel function and once it returns the result, we post it back on the worker thread infrastructure

if (!isMainThread) {

if (parentPort) {

parentPort.once('message', async (data: ExcelData) => {

const result = await buildPersonExcel(data.consultations, data.businessPartners, data.users);

if (parentPort) {

parentPort.postMessage(result);

}

});

}

}

Conclusion

Overall we are very happy with the simplicity of the solution and the added benefit that can be gained through it, solving one of the bigger problems that one might face using nodeJS. We also appreciate it, that it can be solved with a solution provided by nodeJS itself and not some magical npm package.

Performance of this case

Most aspects that are blocking the main thread could be extracted to the worker thread. But by extracting it, the runtime environment needs to serialize the data, which again is cpu heavy on both main thread and worker thread (where we don’t care that much).

This could be also avoided by running the persons fetching inside the worker thread — but in our case we didn’t want to burden the worker threads with the complexity of main threads startup & context creation. We might need to handle database connections, read environment variables, … on that worker thread, which we felt is a overkill for the scenario the need we currently have.

Extensability of our solution

The current approach with one piece of code directly in the same file as the worker instantiation is nice and simple, but once the amount of worker usages increases, some abstraction of this will be required. Having a dedicated entrypoint (file) for all workers would allow us to better structure the code around it and not loose ourselfs in workers beeing created left and right throughout the server and thus loosing the overview of the inner workings and increasing maintenance cost.

Appendix

Setup (tech involved)

NodeJS 14

Server: https://koajs.com/ & Typescript

Excel generation: https://github.com/exceljs/exceljs

Further readings