paddybyers/node — Issue #16: Running multiple node instances in a process
Author: paddybyers
Created: 2011-11-04 21:31:10+00:00
State: open

## Introduction

This is a summary of the issues and proposed solutions to be able to run multiple node instances in a single process.

The requirement arises on Android - and may also exist on other platforms - as a result of a number of resource and lifecycle platform characteristics.

The need arises primarily as a combination of two factors:

- In order to do useful work, with custom modules that can call into the Android platform, node must run in-process in an Android application or service. Only by running in-process is it possible to have access to an `android.content.Context` and other contextual attributes (such as permissions) to be able to di useful work in the platform library.

- The platform will only run at most a single instance of any given application or service process. Therefore, in order to be able to run multiple concurrent instances, or even multiple instances serially, it must be possible to initiate and cleanly exit multiple instances.

However, there are other motivations or potential benefits to doing this. First, it opens the possibility of a very lightweight fork() - which just involves the creation of a new thread and v8 isolate, which are very cheap in comparison with spawning a new process. Other helper threads are sharable between isolates.

Second, it would be possible to arrange for individual isolates to be able to run with reduced platform access - ie "sandboxed" - to execute untrusted code. I've not looked at this in detail yet, but it is clear that v8 isolates can provide a high degree of isolation - including resource constraints - between independently running instances.

The changes to run multiple instances go hand-in-hand with the idea that node is loaded and run as a library. So the multiple instance support as been implemented with an eye primarily on the library API, although a trivial main() is still included that behaves exactly the same as the existing executable.

The following sections summarise the current design and implementation. This is implemented and is running but would benefit from review and discussion of the approach, as well as a critical review of the changes and the possibility for those changes being upstreamed.

## Overview

The core of the approach is that each node instance runs in a separate v8::Isolate, with a separate uv_loop. The principal abstraction in the API is a node::Isolate which encapsulates those two together with all other per-instance state.

Each node::Isolate runs in a dedicated thread. Just as with the conventional system, the thread that enters the Isolate enters, and blocks in, the event loop until the instance dies.

The following simple test code illustrates use of the API to run two parallel http servers.

> #include
> #include
>
> void *run(void *arg);
>
> int main(int argc, char **argv) {
> pthread_t t1, t2;
> int result = node::Initialize(argc, argv);
> if(result == 0) {
> pthread_create(&t1, NULL, &run, (void *)"1337");
> pthread_create(&t2, NULL, &run, (void *)"1338");
> pthread_join(t1, NULL);
> pthread_join(t2, NULL);
> node::Dispose();
> }
> return result;
> }
>
> void *run(void *arg) {
> char *argv[] = {
> (char *)"node",
> (char *)"serv-hello.js",
> (char *)arg,
> 0
> };
> node::Isolate *isolate = node::Isolate::New();
> isolate->Start(3, argv);
> isolate->Dispose();
> return NULL;
> }

The first step is to initialize the library with `node::Initialize()`. This initialises the v8 environment, and the v8-specific options must be passed in the argv. Two threads are spawned and each creates its own isolate with `node::Isolate::New()`. The Isolate::Start() method launches that isolate.

The per-isolate argv can contain instance-specific options (such as `--eval`) as well as options for isolate-level resource constraints; the only such option today relates to stack limit. The isolate argv also obviously includes the isolate's arguments, and is in the same format as the regular node command line.

Each thread will exit the call to `Start()` under the same circumstances as with the conventional system; if the instance autonomously exists (as a result of there being no more watchers), or there was a call to process.exit() or some other fatal exception. By contrast with the conventional executable, however, it is not possible to send a signal to the hosting process and target any specific isolate; instead, there is a programmatic means to deliver a simulated signal to any created isolate through the API.

The changes have been implemented and tested in the unix universe (ie eio and ev) and there are gaps in the windows implementation. Input is welcomed to implement and test the necessary fixes.

The following sections summarise the issues and approach encountered in the implementation.

## Migration of static data to per-isolate data

Much of the static state is in node.cc itself. All static members - with a very small number of exceptions - are migrated to be members of a new node::Isolate class.

Although this state is migrated to the isolate class, there is still a single static defaultIsolate, so the storage of that state has not changed in the default isolate case.

The changes are primarily in this commit.

https://github.com/paddybyers/node/commit/76b51b5abba1406fd791eb7e6a53f9d77123c58f

## Retrieving the current isolate from an arbitrary context

Wherever possible, a pointer to the current isolate is obtained directly - for example, the Isolate associated with each of the prepare, check, and gc watchers etc is stored in the `data` field and retrieved without any lookup.

However, the entrypoints from Javascript - where we are only passed an `Arguments` - do not provide a way to pass the isolate instance pointer explicitly. In these instances the lookup is performed using v8's `v8::Isolate::GetCurrent()` which performs an optimised lookup via v8's thread-local storage. I haven't benchmarked this lookup yet.

There is also a potential optimisation in the defaultIsolate case where there is a simple test against (say) the known thread id associated with the default isolate. I haven't tried this yet - still I think the focus needs to be on optimising the general case instead of simply the single-instance case.

Some helper methods are provided to minimise the impact on the builtin module code - eg to get the uv_loop for the current isolate, or to get the last error from the uv_loop for the current isolate.

## Hard-coded assumptions about the default loop

Several of the builtin modules have a hard-coded assumption about there being only the default loop. These have been updated to use the loop associated with the current isolate.

Most of the changes are here:

https://github.com/paddybyers/node/commit/376d6c1c05cf5fba30c2925d143e01a3b48b3db8

## Default versus non-default isolate

As mentioned above, there is a default isolate, which continues to use the default uv_loop. The only practical difference this makes is that SIGCHLD handling (for spawn()) will work as now in the default isolate, but will not currently work in any non-default isolate.

## Static data for builtin modules

Nearly all of the builtin modules use statics to keep persistent handles to certain v8 objects or primitives (typically strings). These values are not sharable between v8 isolates and therefore must now be managed on a per-isolate basis.

The approach to this is to create a way to access all such values from the node::Isolate instance. This means that an isolate lookup is needed, and the values must be accessed indirectly through a `ModuleStatics` instance instead of directly from static storage. The node_extensions.h mechanism is used to build a table of references to all such objects for a given isolate, so (apart from the resolution of the isolate itself) there is no further cost to obtaining those static values.

This, unfortunately, requires a change in nearly every builtin module.

The changes are primarily in here:

https://github.com/paddybyers/node/commit/729141f683d733ad0a182e17d5286a4d8bb4281c

## eio issues with multiple loops

These issues are documented separately in these issues:

https://github.com/paddybyers/node/issues/15
https://github.com/paddybyers/node/issues/14

## Argument and options processing

The argv and options processing code is shared between the processing for the system as a whole (which applies to the v8-specific options, and also node options that are specified as isolate defaults) and per-isolate options and arguments.

Much of the processing is moved into a new `NodeOptions` class but there is still some tidying that can be done.

https://github.com/paddybyers/node/commit/cd1cefc8172c90062224a2ca2d1d50833beefc5a

## Exit handling

There are several places where node will simply exit by calling `exit()`. Obviously, this isn't going to work for a library.

Broadly, there are two sets of circumstances where this happens:

1) Fatal errors in startup. These are dealt with by setting an exit code and returning from the relevant point. Eventually this results in Start() returning with the exit code.

2) Killing the instance, from inside the event loop. This is relevant to process.exit() and other uncaught and fatal exceptions. Here, it is necessary to cause the thread to return abruptly back to the event loop (typically from deep within a v8 call stack, possibly with multiple nested invocations from javascript to native and back) and then exiting out of the event loop.

It is not possible to do this simply with an exception or longjmp because resources will be leaked.

Returning abruptly back to the event loop is performed by forcibly terminating the v8::Isolate, which causes v8 to propagate and uncatchable termination exception, and so it returns as an uncaught exception back to the event loop.

Exiting the event loop is performed via `uv_break()`.

The conventional exit() behaviour is retained, and the "library" behaviour is enabled via a preprocessor `NODE_LIBRARY` switch.

The relevant changes are here:

https://github.com/paddybyers/node/commit/f99a26d5b9c398b1825ee39fe93080dee08a0a58

## Not yet implemented

Changes to fork() to explicitly create and launch an isolate.

spawn() SIGCHILD handling for the non-default isolate.

Tidying to AtExit() in the non-library case.

Lots of others.

This has been addressed by