fabric: Add mode bits to RX/TX attributes

When an RX/TX context is opened, an application can
specify a subset of the capabilities requested of the
endpoint.  As a result, the provider may be able to
provide relaxed mode bits.

For example, an endpoint that supports both tagged
messages and RMA operations may require FI_CONTEXT
mode bits.  However, a TX context that will only be
used for RMA may not require this mode.  By removing
this mode, an application may see improved performance
when using the RMA TX without needing to allocate
fi_context structures.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
man/fi_errno: Add man page that describes list of errors

Provide a man page that defines possible fabric errors and
their general meaning.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Define FI_NOCQ

Now that the CQ was spun off the EQ, define FI_NOCQ,
similar to FI_NOEQ.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
prov/psm: re-enable fi_cq_read as a progress function

Always poll PSM if the buffer is NULL. This allows fi_cq_read
be used as a progress function when called with NULL buffer.

This was true in the past, but a recent optimization would cause
PSM polling be skipped if the cq is not empty.

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
fabric: Define FI_COMPLETION

Rather than using FI_EVENT to indicate that a completion
should be generated for a given operation, allow the user
to specify this using FI_COMPLETION.  FI_COMPLETION is
defined with the same value as FI_EVENT to avoid using
another flag bit and allow either value to be used.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Rename FI_PROV_MR_KEY to FI_PROV_MR_ATTR

FI_PROV_MR_KEY indicates that the provider controls the value of
the key associated with a memory region.  From the perspective of the
provider, this means that registered memory data must be exchanged
with a remote peer.

Support for memory registration offset is also provider specific.
Providers may require that registered memory be accessed using the
virtual address of the region, rather than a user specified offset.
Rather than introducing a new flag to indicate that the address should
be exchanged, rename FI_PROV_MR_KEY to FI_PROV_MR_ATTR to indicate
that _some_ attribute associated with registered memory must be
exchanged by the application with a remote peer.

The assumption is that it is no more difficult for the app to exchange
all MR attributes than just a single field.  And this change makes the
API simpler overall.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
Merge pull request #311 from shefty/master

Mostly unrelated set of changes
fabric: Remove op/iov size/alignment fields

These fields were added to help define the size of the
RX/TX contexts.  However, the OFIWG is examining an
alternative approach that uses calls into the provider.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
fabric: Define support for sharing RX/TX contexts

Provide a mechanism by which RX and TX contexts may be
shared among multiple endpoints.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
Merge pull request #312 from shefty/master

Updates to memory registration and support for shared RX/TX contexts
man: Fix misspelled words in man pages

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
Merge pull request #314 from shefty/master

man: Fix misspelled words in man pages
prov/psm: check for negative (unlimited) timeout in fi_cntr_wait

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: honor timeout in fi_cq_sread

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
Disable symbol versioning when the -ipo flag is in CFLAGS.

IPO is disabled by icc when asm statements are encountered and an warning is produced.
prov/psm: check for async CQ update in fi_cq_sread

A CQ could be updated asynchrously by the AM handlers (RMA, atomic
operations, etc). As the result, even if PSM polling doesn't return
anything for the current CQ, it is possible to break out from the
blocking wait.

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: set default CQ or CNTR wait object to none.

Both CQ and counter support blocking access via timed polling even if
no wait object is defined (attr.wait_obj==FI_WAIT_NONE). This can also
be applied to the FI_WAIT_UNSPEC case.

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: add domain to the waitset and pollset structure

The domain information is necessary for wait and poll operations.

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: implement the poll operation for pollset

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: enable polling counters in a pollset.

Keep track of the counter value of last access. This allows quick
checking of whether the counter has been updated.

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>