Node.js Child Processes: Everything you need to know
By: Samer Buna
Tags: Programming, JavaScript, Software Development, Web Development, Nodejs

Single-threaded, non-blocking performance in Node.js works great for a single process. But eventually, one process in one CPU is not going to be enough to handle the increasing workload of your application.

No matter how powerful your server may be, a single thread can only support a limited load.

The fact that Node.js runs in a single thread does not mean that we can’t take advantage of multiple processes and, of course, multiple machines as well.

Using multiple processes is the best way to scale a Node application. Node.js is designed for building distributed applications with many nodes. This is why it’s named Node. Scalability is baked into the platform and it’s not something you start thinking about later in the lifetime of an application.

This article is a write-up of part of my Pluralsight course about Node.js. I cover similar content in video format there.

Please note that you’ll need a good understanding of Node.js events and streams before you read this article. If you haven’t already, I recommend that you read these two other articles before you read this one:

The Child Processes Module

We can easily spin a child process using Node’s child_process module and those child processes can easily communicate with each other with a messaging system.

The child_process module enables us to access Operating System functionalities by running any system command inside a, well, child process.

We can control that child process input stream, and listen to its output stream. We can also control the arguments to be passed to the underlying OS command, and we can do whatever we want with that command’s output. We can, for example, pipe the output of one command as the input to another (just like we do in Linux) as all inputs and outputs of these commands can be presented to us using Node.js streams.

Note that examples I’ll be using in this article are all Linux-based. On Windows, you need to switch the commands I use with their Windows alternatives.

There are four different ways to create a child process in Node: spawn() , fork() , exec() , and execFile() .

We’re going to see the differences between these four functions and when to use each.

Spawned Child Processes

The spawn function launches a command in a new process and we can use it to pass that command any arguments. For example, here’s code to spawn a new process that will execute the pwd command.

const { spawn } = require('child_process'); const child = spawn('pwd');

We simply destructure the spawn function out of the child_process module and execute it with the OS command as the first argument.

The result of executing the spawn function (the child object above) is a ChildProcess instance, which implements the EventEmitter API. This means we can register handlers for events on this child object directly. For example, we can do something when the child process exits by registering a handler for the exit event:

child.on('exit', function (code, signal) {

console.log('child process exited with ' +

`code ${code} and signal ${signal}`);

});

The handler above gives us the exit code for the child process and the signal , if any, that was used to terminate the child process. This signal variable is null when the child process exits normally.

The other events that we can register handlers for with the ChildProcess instances are disconnect , error , close , and message .

The disconnect event is emitted when the parent process manually calls the child.disconnect function.

event is emitted when the parent process manually calls the function. The error event is emitted if the process could not be spawned or killed.

event is emitted if the process could not be spawned or killed. The close event is emitted when the stdio streams of a child process get closed.

event is emitted when the streams of a child process get closed. The message event is the most important one. It’s emitted when the child process uses the process.send() function to send messages. This is how parent/child processes can communicate with each other. We’ll see an example of this below.

Every child process also gets the three standard stdio streams, which we can access using child.stdin , child.stdout , and child.stderr .

When those streams get closed, the child process that was using them will emit the close event. This close event is different than the exit event because multiple child processes might share the same stdio streams and so one child process exiting does not mean that the streams got closed.

Since all streams are event emitters, we can listen to different events on those stdio streams that are attached to every child process. Unlike in a normal process though, in a child process, the stdout / stderr streams are readable streams while the stdin stream is a writable one. This is basically the inverse of those types as found in a main process. The events we can use for those streams are the standard ones. Most importantly, on the readable streams, we can listen to the data event, which will have the output of the command or any error encountered while executing the command:

child.stdout.on('data', (data) => {

console.log(`child stdout:

${data}`);

});



child.stderr.on('data', (data) => {

console.error(`child stderr:

${data}`);

});

The two handlers above will log both cases to the main process stdout and stderr . When we execute the spawn function above, the output of the pwd command gets printed and the child process exits with code 0 , which means no error occurred.

We can pass arguments to the command that’s executed by the spawn function using the second argument of the spawn function, which is an array of all the arguments to be passed to the command. For example, to execute the find command on the current directory with a -type f argument (to list files only), we can do:

const child = spawn('find', ['.', '-type', 'f']);

If an error occurs during the execution of the command, for example, if we give find an invalid destination above, the child.stderr data event handler will be triggered and the exit event handler will report an exit code of 1 , which signifies that an error has occurred. The error values actually depend on the host OS and the type of error.

A child process stdin is a writable stream. We can use it to send a command some input. Just like any writable stream, the easiest way to consume it is using the pipe function. We simply pipe a readable stream into a writable stream. Since the main process stdin is a readable stream, we can pipe that into a child process stdin stream. For example:

const { spawn } = require('child_process');



const child = spawn('wc');



process.stdin.pipe(child.stdin)



child.stdout.on('data', (data) => {

console.log(`child stdout:

${data}`);

});