open-mpi/ompi — pull request #326 by zerothi (2014-12-24 10:53:06+00:00)
Title: Enabled COMM_TYPE_SPLIT dependent on locality

Using the underlying hardware identification to split
communicators based on locality has been enabled using
the MPI_Comm_Split_Type function.

Currently implemented split's are:
HWTHREAD
CORE
L1CACHE
L2CACHE
L3CACHE
SOCKET
NUMA
NODE
BOARD
HOST
CU
CLUSTER

However only NODE is defined in the standard which is why the
remaning splits are referred to using the OMPI_ prefix instead
of the standard MPI_ prefix.

@hppritcha , @ggouaillardet I have tested this using --without-hwloc and --with-hwloc=
which both give the same output.

@ggouaillardet I tried moving the constants to the mpi-ext.h, however I got into troubles when compiling as the generic code then relied on mpi-ext.h. To me it seemed like a lot of OMPI_* constants which only has meaning for OpenMPI is still placed in mpi.h. Hence I have not moved the constants to mpi-ext.h.

NOTE: I think something fishy is going on in the locality operators.
In my test-program I couldn't get the correct split on these requests:
NUMA, SOCKET, L3CACHE
where I suspected a full communicator but only got one.