libfabric: Initial commit

libfabric overview

libfabric is an extensible framework for application access
to fabric services.  The framework supports multiple providers,
including providers built into the library.

The layout of the libfabric source tree is outlined below.
Note that this is under development, and that full functionality
is missing.

include/rdma
------------
Contains header files for the framework, including the base
framework APIs in fabric.h.  There are sample APIs available
for message queue operations, RDMA operations, and tagged
messages.

Proposed APIs and objects that support communication and data
transfer functionality are found in fi_domain.h.

fabric.h - Base framework APIs
fi_domain.h - General resource management objects
fi_socket.h - Base communication object

src
---
Contains the base implementation for the framework and kernel
supported APIs.

src/fabric - Base framework implementation
src/ucma - Interface to kernel rdma cm ABI
src/uverbs - Interface to kernel verbs ABI

examples
--------
Includes some simple examples that demonstrate how an application
can use the framework and various API sets.

examples/perf - Simple latency/bandwidth test.
examples/provinfo - List available provider information.

prov
----
Providers built into the libfabric library are under the prov
subdirectory.

prov/ibverbs -
This is a *sample* provider that sits over libibverbs.  It is NOT
meant as a real provider because of the overhead that results from
converting libfabric calls directly into libibverbs calls.  It is
intended to show how a hardware vendor can implement an optimized
version of their provider library for libfabric.

prov/mlx4 -
This is a sample provider that works in conjunction with the
ibverbs provider.  It is mostly unchanged from the existing libmlx4
verbs provider.

prov/psm -
This is a sample provider that sits over the Intel PSM library.

prov/rdmacm -
Incorporates the librdmacm functionality into libfabric.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Separate protocol fields
    
Since fi_info is passed directly into the fi_socket() call,
there's no reason to have the protocol field be split the
way that it is.  Simply create two fields, one for the
protocol capabilities, and the other for the protocol
interoperability.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Remove type mask

fi_info contains a flags field, so there's no need to
specify flags with the socket type.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Remove canonical name fields from fi_info

These are commented out, and it's unclear if they make sense
to include as part of fi_info.  Remove and add back only if
applications request it.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
prov/psm: Add FI_BUFFERED_SEND to supported flags

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Add option to get/set buffered send size

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
prov/ibverbs: Add support to set/get buffered send size

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Update FI_REMOTE flag description

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Remove FI_IOTAGGED and FI_IOTAGGEDV

These were defined as possible IOV formats for tagged
messages, but there are no current uses of them.  Remove
them until a use has been defined.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Remove FI_IOTAGGED and FI_IOTAGGEDV

These were defined as possible IOV formats for tagged
messages, but there are no current uses of them.  Remove
them until a use has been defined.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Rename AV_ATTR_SIZE to AV_ATTR_COUNT

Be consistent with the data structure.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Add priority fields to msg structures

A socket may support multiple priority queues underneath.
Expose a priority field in the msg structures, so that
the user can select an appropriate queue.  This allows
the fabric socket to abstract multiple hardware queues,
support different SLs on a single UD QP, etc.

This is a more generic version of 'urgent' data.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Add MR attributes

Provide a way to extend memory registration capabilities
by adding a mr_attr parameter to the registration calls.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Remove syncto comment

Not sure this makes sense.  If it does, we can add it back.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Remove flags field from fi_open

fi_open takes fi_info as input, which has a flags field.
Avoid the confusion of specifying flags in different places.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Rename AV_ATTR_SIZE to AV_ATTR_COUNT

Be consistent with the data structure.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Add priority fields to msg structures

A socket may support multiple priority queues underneath.
Expose a priority field in the msg structures, so that
the user can select an appropriate queue.  This allows
the fabric socket to abstract multiple hardware queues,
support different SLs on a single UD QP, etc.

This is a more generic version of 'urgent' data.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Add MR attributes

Provide a way to extend memory registration capabilities
by adding a mr_attr parameter to the registration calls.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Remove syncto comment

Not sure this makes sense.  If it does, we can add it back.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Remove flags field from fi_open

fi_open takes fi_info as input, which has a flags field.
Avoid the confusion of specifying flags in different places.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>