MLton/mlton — pull request #91 by MatthewFluet (2014-12-17 18:07:44+00:00)
Title: Cache a worker thread to service calls to `_export`-ed functions.

Each call from C to an `_export`-ed SML function resulted in the
creation of a new ML thread to service the call. An application that
made many calls to `_export`-ed SML functions could result in a
significant cost in terms of allocation and garbage collection; see
http://sourceforge.net/p/mlton/mailman/message/33116352/.

Recall the implementation of `_export`-ed SML functions. Each source
program `_export` is assigned a unique integer and the `_export`
expression becomes a function that stores (a wrapped version of) the
SML function into a global array at the appropriate index. An ML
thread is created and registered in the runtime to handle (not
service) all calls of `_export`-ed SML functions. This ML thread is a
simple loop that inspects global state to determine the index of the
`_export`-ed SML function that is being called, extracts (the wrapped
version of) the SML function from the global array, creates a new ML
thread to execute the extracted SML function (and service the call of
the `_export`-ed SML function), and switches to the new ML thread,
leaving the handler ML thread paused ready to go around the loop and
handle the next call of an `_export`-ed SML function. It is not
possible for the handler ML thread to call the extracted SML function
(and service the call), because the extracted function might call an
`_import`-ed C function which in turns calls an `_export`-ed SML
function; when trying to handle this nested call of an `_export`-ed
SML function, the handler ML thread would be paused at the call of the
`_import`-ed C function, not paused in the handler loop, and chaos
would ensue. Thus, a separate ML thread is required to service the
call of the `_export`-ed SML function, leaving the handler ML thread
ready to handle nested and subsequent calls of `_export`-ed SML
functions.

However, mutual recursion between SML and C is probably not common.
The common case is probably that of SML calling C calling SML (or, in
the case of a SML system-level library, just C calling SML). In this
common case, it shouldn't be necessary to create a new ML thread per
call of an `_export`-ed SML function.

In this common case, it would be possible for the handler ML thread to
also service the call of the `_export`-ed function. However, we still
need to handle the situation where there is deeper recursion between
SML and C. Creating and registering a new handler ML thread while the
original handler ML thread is servicing the call of the `_export`-ed
function isn't a solution to the allocation problem, since a new
handler ML thread is created per call of an `_export`-ed SML function.
Lazily creating a new handler ML thread isn't a solution either, since
we can't recover from the original handler ML thread paused at the
call of the `_import`-ed C function (and not paused in the handler
loop).

A better solution is to create a worker ML thread to service the call
of an `_export`-ed SML function and reuse this worker thread when it
is not already servicing the call of an `_export`-ed SML function.
Essentially, a `val worker: Prim.thread option ref` is created to
cache an available worker ML thread. The handler ML thread inspects
`worker` for an available worker thread: if `!worker` is `NONE`, then
a new worker ML thread is created; if `!worker` is `SOME wrk_thrd`,
then `worker` is updated to `NONE` (indicating that no worker thread
is available; the `wrk_thrd` worker thread is now servicing the call
of the `_export`-ed SML function). When a worker ML thread is about
to return to C (and pause ready to go around the loop and service the
next call of an `_export`-ed SML function), it updates `worker` to
`SOME self`, making itself available.

In the common case, a single worker ML thread is created at the first
call of an `_export`-ed SML function, and this worker ML thread is
reused for all subsequent calls of `_export`-ed SML functions. The
example program from the above message results in no additional
allocation per call of the `_export`-ed SML function (resulting in
significantly faster overall execution).

While it would be possible to replace `val worker: Prim.thread option
ref` with `val workers: Prim.thread list ref`, allowing multiple
worker ML threads to persist after some mutual recursion between SML
and C, this would result in a small amount of additional alloction per
call of `_export`-ed SML function corresponding cons-cell allocation
in the the update of `workers` to `self :: !workers`.