prov/psm: document the assumption on the "event_in" parameter.

This parameter was added to the event creation routine to allow
the event being written to the user buffer directly, bypassing
the steps of allocating/enqueuing/dequeuing a new event. The user
buffer only has space for the CQ entry of the current CQ format.
Any provider provate information should not be written to it.

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: implement cq_writeerr

Also allow writing multiple events with cq_write.

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: finish 'len' to 'count' conversion of CQ read functions

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: add count to event queue to support read threadhold

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: allow returning multiple events in a single CQ read call

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: use socketpair() instead of pipe() to create fd wait obj

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: implement wait obj

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: partial implementation of pollset

The actual polling part is missing. And there are missing links
between and poll_add/poll_del calls and the API definition.

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: implement cntr.readerr()

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: create a utility routine to get the wait object

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: use the new wait object mechanism in counter

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: signal wait object when events are added to cq

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
porv/psm: remove unused union member name

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: implement blocking cq read

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: replace empty av bind function with fi_no_bind

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
Merge pull request #256 from shefty/master

Manual merge of pull requests from Patrick
add support for async AV insertions
- add context fields to av_insert calls
- add fields to fi_eq_entry and fi_eq_err_entry for completion of
  vectored operations
- update man pages
- implement async insrt in usnic provider

Signed-off-by: Reese Faucette <rfaucett@cisco.com>
Merge pull request #257 from rfaucett/pr/async_av

add support for async AV insertions
Merge branch 'master' of ssh://github.com/ofiwg/libfabric
Merge pull request #262 from shefty/master

man/fi_msg: Fix typo describing struct fi_msg