prov/psm: document the assumption on the "event_in" parameter.

This parameter was added to the event creation routine to allow
the event being written to the user buffer directly, bypassing
the steps of allocating/enqueuing/dequeuing a new event. The user
buffer only has space for the CQ entry of the current CQ format.
Any provider provate information should not be written to it.

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: implement cq_writeerr

Also allow writing multiple events with cq_write.

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: finish 'len' to 'count' conversion of CQ read functions

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: add count to event queue to support read threadhold

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: allow returning multiple events in a single CQ read call

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: use socketpair() instead of pipe() to create fd wait obj

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: implement wait obj

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: partial implementation of pollset

The actual polling part is missing. And there are missing links
between and poll_add/poll_del calls and the API definition.

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: implement cntr.readerr()

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: create a utility routine to get the wait object

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: use the new wait object mechanism in counter

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: signal wait object when events are added to cq

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
porv/psm: remove unused union member name

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: implement blocking cq read

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
prov/psm: replace empty av bind function with fi_no_bind

Signed-off-by: Jianxin Xiong <jianxin.xiong@intel.com>
Merge branch 'master' of ssh://github.com/ofiwg/libfabric
Merge pull request #261 from j-xiong/master

prov/psm: waitset, pollset, blocking cq read, etc