chapel-lang/chapel — pull request #257 by ronawho (2014-08-29 17:32:36+00:00)
Title: Limit the memory qthreads uses for it's memory pools

Qthreads uses memory pools for things like stacks, thread private data, runtime
data, etc. It assumes that the item size for things in the pool is small,
however our default stack size is quite large. This meant that with qthreads we
were allocating over a GB up front.

This seemed to be a real problem on a 32 bit machine, but the real problem
there ended up being the qthreads lowest common multiple routine was
overflowing (which is fixed in this commit.) Still the work was already put in
to limit the amount of memory and it's something we wanted to do anyways.

This approach modifies qthreads to take the max pool alloc size as an env var,
but if always ensures that at least two items can be allocated. Without this
env var set, qthreads behavior is unaffected. Note that Qthreads team has not
formally reviewed this as they're busy this week, but they will next week.

When we're setting up qthreads we set the max pool alloc size to be a multiple
of hardware parallelism and the call stack size, with the idea that memory will
be proportional to number of cores. We set a lower bound on this so we don't
make all qthreads pool allocations limited, and a maximum size on this for
architectures with many cores, but limited memory.