Fixes #3313: Fix multi-node comm_spawn in v1.7

---svn-pre-commit-ignore-below---

r27318 [[BR]]
Cleanup comm_spawn for the multi-node case where at least one new process isn't spawned on every node. Avoid the complexities of trying to execute a daemon collective across the dynamic spawn as it becomes too hard to ensure that all daemons participate or are accounted for - instead, use a less scalable but workable solution of sending the data directly between the participating procs. Ensure that singletons get their collectives properly defined at startup so the spawned "HNP" is ready for them.

As a secondary cleanup, the HNP doesn't need to update its nidmap during an xcast as it already has an up-to-date picture of the situation. So just dump that data and move along.
Fixes #3310: Move fixes for VT build failures into v1.7 branch

---svn-pre-commit-ignore-below---

r27316 [[BR]]
Changes to OTF:
	- otfaux:
		- fixed build error on Solaris and NetBSD (removed -lm from library dependencies)
Changes to VT:
	- vtunify:
		- disable OpenMP parallelization if PGI compiler version < 9 is used (threadprivate not supported)
Fixes #3275: Move hwloc base error message fix to v1.7

---svn-pre-commit-ignore-below---

r27180 [[BR]]
Clean up the error message names from the hwloc base, and add a
missing error message.

r27306 [[BR]]
Refs #3275.

We ran into a case where the OMPI SVN trunk grew a new acceptable MCA
parameter value, but this new value was not accepted on the v1.6
branch (hwloc_base_mem_bind_failure_action -- on the trunk it accepts
the value "silent", but on the older v1.6 branch, it doesn't).  If you
set "hwloc_base_mem_bind_failure_action=silent" in the default MCA
params file and then accidentally ran with the v1.6 branch, every OMPI
executable (including ompi_info) just failed because hwloc_base_open()
would say "hey, 'silent' is not a valid value for
hwloc_base_mem_bind_failure_action!".  Kaboom.

The only problem is that it didn't give you any indication of where
this value was being set.  Quite maddening, from a user perspective.

So we changed the ompi_info handles this case.  If any framework open
function return OMPI_ERR_BAD_PARAM (either because its base MCA params
got a bad value or because one of its component register/open
functions return OMPI_ERR_BAD_PARAM), ompi_info will stop, print out
a warning that it received and error, and then dump out the parameters
that it has received so far in the framework that had a problem.

At a minimum, this will show the user the MCA param that had an error
(it's usually the last one), and ''where it was set from'' (so that
they can go fix it).  

We updated ompi_info to check for O???_ERR_BAD_PARAM from each from
the framework opens.  Also updated the doxygen docs in mca.h for this
O???_BAD_PARAM behavior.  And we noticed that mca.h had MCA_SUCCESS
and MCA_ERR_??? codes.  Why?  I think we used them in exactly one
place in the code base (mca_base_components_open.c).  So we deleted
those and just used the normal OPAL_* codes instead.

While we were doing this, we also cleaned up a little memory
management during ompi_info/orte-info/opal-info finalization.
Valgrind still reports a truckload of memory still in use at ompi_info
termination, but they mostly look to be components not freeing
memory/resources properly (and outside the scope of this fix).
Fixes #3315: Move cmd line fix to v1.7

---svn-pre-commit-ignore-below---

r26953 [[BR]]
Minor fix for the command line parser: we didn't previously
distinguish between unknown ''options'' (i.e., command line options
that are registered and have some meaning) and unknown ''tokens''
(i.e., strings that do not begin with "-").  Hence, if you did:

   mpirun --fo my_mpi_program

(when perhaps you meant to type "--foo", mpirun would complain that no
such executable "--fo" existed.  That is ''correct,'' but perhaps not
completely useful.  It is more accurate for mpirun to report that
there is no such "--fo" option.

This change to cmd_line.c makes it so that we will ''always'' report
errors regarding tokens that begin with "-".