Araq/Nimrod — pull request #1644 by rbehrends (2014-11-08 15:20:12+00:00)
Title: Reworked the threadpool implementation.

Now uses a simple task queue protected by a monitor, which simplifies
the program logic and fixes a number of race conditions and potential
deadlock situations.

Simplifies and fixes the barrier implementation.

Fixes the load balancer so that it doesn't stop operating after 128
calls. Whether the load balancer is actually worth the overhead of
checking the CPU load periodically is not sure, but it can be disabled
by setting the minimum pool size to be the same as the maximum pool
size. The biggest problem is that under load the `advice()` operation
requires pretty all tasks to use a single shared lock, introducing a
serialization bottleneck.

Renamed the CondVar type to Semaphore, because that's what it is.

The FlowVar cleanup queue now has unbounded size, which eliminates a
bottleneck when a large number of FlowVar instances are GCed
simultaneously. It may still be worthwhile to keep track of the size of
the free list and prune it if it exceeds a certain size.

A difficult to remove bottleneck remains with the implementation of
`spawn`. The current implementation requires `spawn` to block until the
the task has started and copied all its arguments. This is a difficult
issue to resolve, but it can at least be eliminated when there are no
arguments that need to be copied and by calling `scheduleTask()` without
the optional semaphore argument, non-blocking startup is possible in
principle.