fabric: Define scalable endpoints

Scalable endpoints are endpoints that support multiple send and
receive queues (contexts).

Introduce the concept of transmit and receive contexts
(generalizations of send/receive queues).  Allow applications to
request that multiple contexts be associated with a single endpoint.

An endpoint has a single transport address, but the contexts allow
for multiple threads to process requests in parallel without
ordering or synchronization constraints.

This requires additional domain, endpoint, and AV attributes.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Rename EQ condread to sread and remove cond param

An EQ is for control events.  There isn't a requirement
to support a condition on the blocking read call.  Remove
the cond variable and rename the read call.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Rename CQ condread to sread to match EQ

The blocking EQ read call was renamed to sread, synchronous read.
Modify the CQ condread call to match.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Add event type parameter to EQ ops

Move the event type from the top of the various event
structures to its own parameter.  This simplifies the
interface for the applications.  An event parameter is
added to the EQ read, sread, and write calls.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Add CM protocol to EP protocol defines

IB is braindead and has an out of band connection protocol
that isn't required to be used.  For EPs with IB
protocols, we need to include the CM protocol as part of
the definition in order to ensure interoperability.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Remove FI_BLOCK flag

It is no longer used as part of the interface.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
man/fi_getinfo: Describe provider returning multiple fi_info's

Indicate that providers may return multiple fi_info structures,
and that the order of the returned data indicates a specific
priority.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
Merge branch 'dev'
Merge remote branch 'upstream/master' into dev
Merge branch 'master' into dev
fabric: Really remove FI_BLOCK flag

Previous patch removed the use, now remove the flag.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Rename fi_fdomain() to just fi_domain

The 'f' is not needed and may cause people to curse when
reading it.  It's just a domain.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Add AV support for symmetric addresses

Symmetric address supports allows an app to declare that the
number of endpoints per node will be the same, and that the
transport addresses of those endpoints will be the same.  A further
optimization allows the transport addresses to include all
addresses within a range.  This allows an AV to store the
addresses of remote endpoints in a compressed fashion, to
greatly reduce the needed memory footprint.

Define an FI_SYMMETRIC flag for use with an AV.  The flag will
indicate that endpoints are distributed equally over each network
address.  In conjunction with FI_RANGE, an app can insert a large
number of endpoint addresses into an AV with a single call.

Introduce a new AV insert all that takes as input a node and
service parameter, similar to those specified through fi_getinfo.
This simplifies things for the app and allows the insertion of
hostnames into the AV, for example.

Remove the fi_av_map macro and have users call fi_av_insert as
defined in the man pages.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
prov/verbs: Remove MLX4 direct from verbs build

This was a partial implementation that was never completed.
Remove it from the verbs provider.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
prov/verbs: Fix verbs build when set to auto

The verbs provider does not get built when set to auto,
even if verbs is installed.  HAVE_VERBS remains 0, since
enable_verbs = auto, rather than yes.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
Merge pull request #140 from shefty/master

Introduce concept of scalable endpoints