Fixes #4564: Move r31522 to v1.8 branch (btl/vader: improve performance under heavy)

---svn-pre-commit-ignore-below---

r31522 [[BR]]
btl/vader: improve performance under heavy load and eliminate a racy
feature

This commit should fix a hang seen when running some of the one-sided
tests. The downside of this fix is it reduces the maximum size of the
messages that use the fast boxes. I will fix this in a later commit.

To improve performance under a heavy load I introduced sequencing to
ensure messages are given to the pml in order. I have seen little-no
impact on the message rate or latency with this change and there is a
clear improvement to the heavy message rate case.

Lets let this sit in the trunk for a couple of days to ensure that
everything is working correctly.

cmr=v1.8.2:reviewer=jsquyres
Fixes #4587: Move r31576, r32158 to v1.8 branch (mca_base_var: use ERR_NOT_FOUND when var)

---svn-pre-commit-ignore-below---

r31576 [[BR]]
Fix MPI_T issues identified by friendly users.

Several fixes:

 - I was allowing an MPI_T_cvar_handle to be created for an invalid
   variable. Fixed this by checking if the variable is valid in
   mca_base_var_get.

 - Use a better error code when the caller tries to create an unbound
   pvar handle for a bound variable.

 - Return the verbosity level in MPI_T_cvar_get_info.

cmr=v1.8.2:reviewer=jsquyres

r32158 [[BR]]
mca_base_var: use ERR_NOT_FOUND when var is inactive

After discussion with Nathan, change the ERR_VALUE_OUT_OF_BOUNDS to be
ERR_NOT_FOUND, for two reasons:

1. It's consistent with other uses of ERR_NOT_FOUND in the code.
1. In this case, we could have just looked up a variable that is
   basically a "hole" -- e.g., var indexes 0, 1, 2 are valid, and 4,
   5, and 5 are valid, but 3 is invalid (e.g., 3 was de-registered --
   remember that MPI_T explicitly does not allow re-using indexes).
   So returning ERR_OUT_OF_BOUNDS seems weird -- returning
   ERR_NOT_FOUND seems a bit more natural.

cmr=v1.8.2:ticket=#4587

r32161 [[BR]]
mpit: update the return code check for mca_base_var_get

mca_base_var_get now can return OPAL_ERR_NOT_FOUND if a variable no
longer exists. This commit updates the return code check to ensure
the correct MPI_T error code is returned to the user.

cmr=v1.8.2:reviewer=jsquyres
Fixes #4766: Move r32151 to v1.8 branch (coll/ml: set n_resources to 0)

---svn-pre-commit-ignore-below---

r32151 [[BR]]
coll/ml: set n_resources to 0 when destructing an lmngr

Also keep track of the allocation base so we free the correct pointer
when cleaning up.

cmr=v1.8.2:reviewer=manjugv
Fixes #4773: Move r32156, r32159 to v1.8 branch (btl_usnic_map.c: Arrgh!  Forgot to)

---svn-pre-commit-ignore-below---

r32156 [[BR]]
usnic: add btl_usnic_connectivity_map MCA param to output link information

If the btl_usnic_connectivity_map MCA param is set to a non-NULL
value, then each MPI process will output a file named
<prefix>-<hostname>.pid<pid>.job<jobid>.mcwrank<MCW rank>.txt.  Its
contents will detail which usNIC device(s) (and therefore which
link(s)) are being used to communicate with each peer MPI process.

Here is a sample output file (named
mpi005.pid26071.job1640759297.mcwrank0.txt):

{{{
device=usnic_0,interface=eth4,ip=10.10.0.5/16,mac=24:57:20:05:20:00,mtu=9000
device=usnic_1,interface=eth5,ip=10.2.0.5/16,mac=24:57:20:05:21:00,mtu=9000
device=usnic_2,interface=eth6,ip=10.3.0.5/16,mac=24:57:20:05:50:00,mtu=9000
peer=1,hostname=mpi006,device=usnic_0@peer_ip=10.10.0.6/16@peer_mac=24:57:20:06:20:00,device=usnic_1@peer_ip=10.2.0.6/16@peer_mac=24:57:20:06:21:00,device=usnic_2@peer_ip=10.3.0.6/16@peer_mac=24:57:20:06:50:00
peer=2,hostname=mpi007,device=usnic_0@peer_ip=10.10.0.7/16@peer_mac=24:57:20:07:20:00,device=usnic_1@peer_ip=10.2.0.7/16@peer_mac=24:57:20:07:21:00,device=usnic_2@peer_ip=10.3.0.7/16@peer_mac=24:57:20:07:50:00
peer=3,hostname=mpi008,device=usnic_0@peer_ip=10.10.0.8/16@peer_mac=24:57:20:08:20:00,device=usnic_1@peer_ip=10.2.0.8/16@peer_mac=24:57:20:08:21:00,device=usnic_2@peer_ip=10.3.0.8/16@peer_mac=24:57:20:08:50:00
}}}

Reviewed by Reese Faucette

cmr=v1.8.2

r32159 [[BR]]
btl_usnic_map.c: Arrgh!  Forgot to svn add this file.

cmr=v1.8.2:ticket=#4773
Fixes #4774: Move r32160 to v1.8 branch (osc/rdma: check for more types)

---svn-pre-commit-ignore-below---

r32160 [[BR]]
osc/rdma: check for more types of window access violations

This commit adds a check to see if the target is in an access epoch. If
not we return OMPI_ERR_RMA_SYNC. This fixes test_start3 in the onesided
test suite. The cost of this extra check is 1 byte/peer for the boolean
flag indicating that the peer is in an access epoch.

I also fixed a problem where mupliple unexpected post messages are not
correctly handled.

cmr=v1.8.2:reviewer=jsquyres