tcp: fix thread race when progressing sends

Always lock before trying to progress sends.

Reported by Teng Wang <tzw0019@tigermail.auburn.edu>
tcp: if send has no buffers, try to progress ep

to process ack and release completed sends
tcp: fix race in RMA handling

When the initiator is multi-threaded, there is a race between queuing
the txs on the conn and then queuing the rma_op on the endpoint if
another thread was progressing the endpoint and received an ack before
the rma_op was enqueued on the endpoint.

Take the endpoint lock before queuing the txs.

Reported by Teng Wang <tzw0019@tigermail.auburn.edu>
tests: add multi-threaded RMA client

based on rma_verify. Spawn -T <num> threads to send RMAs to the
target. The target is still single-threaded.
tests: enable multi-threaded server (target)

Allow the server to accept -t <num_threads> for polling events.
tests: add verbose flag to rma_threaded

Only print small buffer and successful crc is -v is set
tcp: check socket error when connect gets POLLHUP

This fixes a bug in d56ddab49d3e3433fb5b6e13d63f13f9eda27b57
in which we check the socket error. We erroneously checked the
pointer to the error, not the error itself. This commit fixes that.
tcp: set the socket buffer size on servers

We were only using large buffers on the client.
Merge branch 'master' of github.com:CCI/cci
verbs: reduce per QP resources

to improve scalability of connections.
tcp: improve multi-threaded support

Do not progress conn sends in cci_connect()
If blocking send, detect connection closing and return error
If POLLHUP && POLLOUT && no socket error, try ust POLLOUT
If POLLHUP && conn status is ACTIVE2, the event may still be queued
Take conn slock before changing status to READY
Exit loop when found > nfds, not count
tcp: block instead of poll for is_polling token

Use pthread condition variable to wake waiters
tcp: add poll fd to conn and it use

remove the tcp conn fd and use tconn->pfd.fd
first baby step to polling per connection rather than
per endpoint
tcp: move polling per connection

vastly improve multi-threaded stability
tcp: progress in-flight RMA fragment

but don't start another if TCP_RMA_DEPTH are pending
tcp: rearrange conn to be more cache friendly
tcp: drive send progress from poll_events

In poll_events, set POLLOUT as well as POLLIN.
Call progress_conn_sends() when POLLOUT is ready.
Remove progress_sends, progress_queued, progress_pending.
tcp: remove index from conn
tcp: remove tcp_ignore_fd_locked()

Trying to avoid needing ep->lock for connection specific changes
tcp: add reference to conn for app

drop that reference in disconnect()
Also, queue the listening socket on the passive queue instead of
connected conns because the app never has a reference to it and
the conns queue should have 2 references (one for the list and one
for the app).