Pamgen: Adding support for Boost, GCEI's

1) Adding TPL support for Boost to Pamgen.

2) Adding support for the generalized complete elliptic integral, computed
via Carlson's functions in Boost.
[Belos] uncaught exceptions
[Belos] restore original os flags
[Belos] pass several large data items by reference

Build/Test Cases Summary
Enabled Packages: Belos
Disabled Packages: Pamgen,Intrepid
Enabled all Forward Packages
0) MPI_DEBUG => passed: passed=479,notpassed=0 (33.70 min)
1) SERIAL_RELEASE => passed: passed=445,notpassed=0 (11.56 min)
Other local commits for this build/test group: 7019b46, 10dbbd0
[NOX] uncaught exceptions, uninitialized variables

Build/Test Cases Summary
Enabled Packages: NOX
Disabled Packages: Pamgen,Intrepid
Enabled all Forward Packages
0) MPI_DEBUG => passed: passed=182,notpassed=0 (11.90 min)
1) SERIAL_RELEASE => passed: passed=182,notpassed=0 (4.17 min)
MueLu: added default template args to ShiftedLaplacian
Tpetra::MultiVector::subView(NonConst): Improved input checking.

This commit only affects the Kokkos refactor version of Tpetra.
Anasazi: Fixed bugs in the new Tpetra SetBlock test.

It now passes for both the "classic" and Kokkos refactor versions of
Tpetra.  That means that the recently reported issues might be caused
by some other MultiVecTraits function (CloneView* perhaps?) and may
only manifest in SetBlock.  I will expand the test to see if I can
replicate observed misbehavior in a simple way.

Build/Test Cases Summary
Enabled Packages: KokkosCore, KokkosContainers, KokkosCompat, KokkosLinAlg, KokkosMpiComm, KokkosAlgorithms, Tpetra, Belos, Ifpack2, Zoltan2, Amesos2, Anasazi
Disabled Packages: FEI,Moertel,MOOCHO,NOX,Rythmos,STK
Enabled all Forward Packages
0) MPI_DEBUG => Test case MPI_DEBUG was not run! => Does not affect push readiness! (-1.00 min)
1) SERIAL_RELEASE => Test case SERIAL_RELEASE was not run! => Does not affect push readiness! (-1.00 min)
2) MPI_DEBUG_SS => FAILED: passed=995,notpassed=1 => Not ready to push! (41.09 min)
3) SERIAL_RELEASE_SS => passed: passed=946,notpassed=0 (10.40 min)
4) MPI_DEBUG_KOKKOS_REFACTOR => FAILED: passed=1001,notpassed=2 => Not ready to push! (43.69 min)
5) SERIAL_RELEASE_KOKKOS_REFACTOR => FAILED: passed=950,notpassed=2 => Not ready to push! (12.57 min)
Other local commits for this build/test group: 13cae99
WARNING: Forced the push!
Kokkos: Allow C++11 if cuda version is >= 6.5

CUDA 6.5 inofficially supports C++11 in device code. This includes
stuff like lambdas and auto types. It does not enable dispatch of
lambdas from the host, since lambdas are not allowed as arguments
to __global__ functions.

We introduce a new internal typedef: KOKKOS_HAVE_CXX11_DISPATCH_LAMBDA
which can be used to guard such lambda dispatches for example in tests
against be compiled for CUDA.
Kokkos Cuda: cleaned up internal implementation of shfl_down operations, added shfl

I simplified the internal implementation of shfl_down and added a mapping for shfl.
Both should work for any types of size 4 or 8 and fall back to using static shared
memory otherwise.
Kokkos: Guarded test of CXX11 Lambda dispatch with new macro

Using KOKKOS_HAVE_CXX11_DISPATCH_LAMBDA to guard the test which
uses it. Made the Macro have a value of 1 if defined.
Kokkos: Addint TeamVectorPolicy

TeamVectorPolicy allows nested parallelism with explicit vector loops.
For CUDA this enables in particular to have multiple cuda-threads in a warp to act
as vector lanes. The member_type of the policy has (in addition to the same functionality as
the member_type of the TeamPolicy) following functions which are only available if
C++11 is enabled:

vector_par_for(INT, LAMBDA)
vector_par_reduce(INT,LAMBDA,RESULT)
vector_par_reduce(INT,LAMBDA,INITRESULT,JOIN_OP)
vector_par_scan(INT,LAMBDA,RESULT)

Documentation is posted further down.
A usage could look like this for a sparse matvec:

void operator(TeamVector member) const {
  //get row index for this thread
  int row = member.league_rank()*member.team_size() + member.team_rank();
  double row_dot;

  //loop over the row with vector lanes of a single thread
  member.vector_par_reduce(RowLength(row),[&] (int i, double& val) {
    val+= RowValues(i)*X_Vector(RowIndex(i));
  }, row_dot);

  //let only one vector_lane write the result to the Y_Vector
  member.vector_single([&] () {
    Y_Vector(row) = row_dot;
  });
}

Note that code outside of vector_* operations is potentially executed by all vector lanes.
This is currently true on CUDA, but could be also true on other hardware/backend models if those
handel Vector Lanes explicitly.

Also Note that CUDA 6.5 supports C++11 with the major exception of submitting lambdas to
global functions. Hence you can't give a lambda to a Kokkos::parallel_for. But the now added
vector_* functions accept lambdas also with CUDA.

The TeamVectorPolicy was added to all backends including QThreads.

    /** \brief  Guarantees execution of op() with only a single vector lane of this thread. */
    template< class Operation >
    KOKKOS_INLINE_FUNCTION void vector_single(const Operation & op) const ;

    /** \brief  Intra-thread vector parallel for. Executes op(const iType i) for each i=0..N-1.
     *
     * The range i=0..N-1 is mapped to all vector lanes of the the calling thread.
     * This functionality requires C++11 support.*/
    template< typename iType, class Operation, typename ValueType >
    KOKKOS_INLINE_FUNCTION void vector_par_for(const iType n, const Operation & op) const ;

    /** \brief  Intra-thread vector parallel reduce. Executes op(const iType i, ValueType & val) for each i=0..N-1.
     *
     * The range i=0..N-1 is mapped to all vector lanes of the the calling thread and a summation of
     * val is performed and put into result. This functionality requires C++11 support.*/
    template< typename iType, class Operation, typename ValueType >
    KOKKOS_INLINE_FUNCTION void vector_par_reduce(const iType n, const Operation & op, ValueType& result) const ;

    /** \brief  Intra-thread vector parallel reduce. Executes op(const iType i, ValueType & val) for each i=0..N-1.
     *
     * The range i=0..N-1 is mapped to all vector lanes of the the calling thread and a reduction of
     * val is performed using JoinType(ValueType& val, const ValueType& update) and put into init_result.
     * The input value of init_result is used as initializer for temporary variables of ValueType. Therefore
     * the input value should be the neutral element with respect to the join operation (e.g. '0 for +-' or
     * '1 for *'). This functionality requires C++11 support.*/
    template< typename iType, class Operation, typename ValueType, class JoinType >
    KOKKOS_INLINE_FUNCTION void vector_par_reduce(const iType n, const Operation & op, ValueType& init_result, const JoinType & join) const ;

    /** \brief  Intra-thread vector parallel exclusive prefix sum. Executes op(const iType i, ValueType & val, bool final)
     *          for each i=0..N-1.
     *
     * The range i=0..N-1 is mapped to all vector lanes in the thread and a scan operation is performed.
     * Depending on the target execution space the operator might be called twice: once with final=false
     * and once with final=true. When final==true val contains the prefix sum value. The contribution of this
     * "i" needs to be added to val no matter whether final==true or not. In a serial execution
     * (i.e. team_size==1) the operator is only called once with final==true. Scan_val will be set
     * to the final sum value over all vector lanes.
     * This functionality requires C++11 support.*/
    template< typename iType, class Operation, typename ValueType >
    KOKKOS_INLINE_FUNCTION  void vector_par_scan(const iType n, const Operation & op, ValueType& scan_val) const ;
KOkkos: Allow C++11 for CUDA 6.5. Add KOKKOS_HAVE_CXX11_DISPATCH_LAMBDA

CUDA 6.5 allows C++11 in device code with the exception of giving
lambdas to __global__ functions. Hence now C++11 can be enabled for
Kokkos if using C++11 but a KOKKOS_HAVE_CXX11_DISPATCH_LAMBDA define
has been added which can be used to guard code which gives lambdas
to Kokkos::parallel_for etc.
Kokkos: CUDA Internals

Added functions which return the maximal and optimal thread number
for a given functor. Note the functions are templated on the DriverType
which is equivalent to the inernal ParallelFor<FunctorType,DeviceType>.
Kokkos: Adding tests for the new TeamVectorPolicy

The tests are only active is C++11 is used.
Kokkos: Add the actual TeamVector test file
Disable Zoltan2's use of parmetis temporarily

The fix made yesterday to Zoltan2 exposed an issue with the version of parmetis
that we have installed on our test machines. We need to update to 4.0 to satisfy
the Zoltan2 dependency. That hasn't been done yet so I am disabling Zoltan2's
use of parmetis until the new version can be installed.
Belos: Cleaned up shadowing warnings in test
Anasazi: Added new test for Tpetra MultiVecTraits::SetBlock.

The Tpetra specialization of MultiVecTraits now has a new test for
SetBlock, TpetraSetBlock4.  This test manifested the issue that Alicia
Klinvex and others discovered.

TODO: Port this test to Tpetra.  It makes an excellent test for the
postcondition of Tpetra::deep_copy, that it does not invalidate any
views of the output MultiVector dst.

TODO: Port this test to Belos (to exercise Belos' MultiVecTraits).
Tpetra::MultiVector: Fixed deep_copy.

This commit affects both the "classic" and Kokkos refactor versions of
Tpetra.

The implementation of Tpetra::deep_copy for MultiVector arguments had
a bug: it was invalidating existing views of the output argument dst.
This manifested as incorrect behavior of the Tpetra specializations of
{Anasazi, Belos}::MultiVecTraits::SetBlock.

This commit fixes this, by ensuring that the only thing that changes
in dst is the contents of its storage.  Its pointers don't change, it
doesn't reallocate memory, and it doesn't change any other fields.

The easiest and cleanest way for me to fix this was to add a public
'assign' method to MultiVector.  This takes a MultiVector of the same
type, and does a deep copy into *this.  This means that the
specialization of deep_copy for when src and dst have the same type
doesn't need an explicit instantiation; it's just a one-line call to
assign().  Of course, the "generic" version of deep_copy, for when src
and dst have different types, takes more work.