chapel-lang/chapel — pull request #156 by gbtitus (2014-08-14 22:34:19+00:00)
Title: add the runtime tasking layer's notion of max parallelism to locale models

We're moving toward having the tasking layer tell the rest of Chapel
(notably the module code) how much parallelism it thinks it can provide,
instead of the module code (et al.) deciding that on its own based on
how much hardware it sees. This is the first step along that path. It
augments the processor-counting support in the runtime so that we can
get an accurate count of the number of cores and processor units (PUs,
basically anything that can issue instructions, including hyperthreads).
And, it adds a chpl_getMaxPar() call to the tasking layer interface, to
say how much true parallelism the tasking layer implementation thinks it
can provide. Only the runtime tasking layer work is done so far. The
module code hasn't been updated to use this information. It's still looking
directly at how much hardware is configured, when deciding for example
how much parallelism to create for a forall-stmt.

The specific file changes are as follows.

runtime/include/chplsys.h
runtime/src/chplsys.c
Rename chpl_numCoresOnThisLocale() as chpl_getNumPUsOnThisNode()
(since #PUs is what it actually returned) and re-implement it for
Linux to reflect kernel scheduling affinity, not just what hardware is
present. Add a new chpl_getNumCoresOnThisNode(), which returns the
number of cores available to the process. This new function is a copy
of ...getNumPUs..., adjusted to return the number of actual cores (no
hyperthreads or equivalent). Make both of these return a regular int;
chpl_numCoresOnThisLocale() returned an int64_t, but I can't think of
a good reason to retain that.

runtime/include/chpl-tasks.h
runtime/src/tasks/fifo/tasks-fifo.c
runtime/src/tasks/massivethreads/tasks-massivethreads.c
third-party/qthread/qthread-1.10/src/interfaces/chapel/tasks-qthreads.c
Add the new tasking interface function chpl_task_getMaxPar().
Implement it in terms of chpl_getNumCoresOnThisNode() (added above)
for fifo and massivethreads, and in terms of qthread_num_workers() for
qthreads.
Also, in fifo tasking, call chpl_getNumPUsOnThisNode() instead of
chpl_numCoresOnThisLocale() (which actually returned #PUs) when
deciding whether to spin or use a pthread cond var while waiting for a
sync var to be in the desired state. And, while here, clarify the
associated comment.

modules/internal/ChapelLocale.chpl
modules/internal/localeModels/flat/LocaleModel.chpl
modules/internal/localeModels/numa/LocaleModel.chpl
Add a new maxTaskPar member to the LocaleModel module. For the flat
locale model and for numa with no sublocales (which happens for example
with tasks=fifo), initialize maxTaskPar by calling chpl_task_getMaxPar(). For
numa with sublocales, for now set the maxTaskPar value for the locale to the
number of sublocales, and for the sublocales to chpl_task_getMaxPar()
divided by the number of sublocales. We may want to get fancier later, but
this should do for now.

test/localeModels/gbt/maxTaskPar.chpl
test/localeModels/gbt/maxTaskPar.cleanfiles
test/localeModels/gbt/maxTaskPar.prediff
Add a new test to ensure that here.maxTaskPar has the expected value on
all the locales and sublocales in the locale model the program finds itself
running on.