Repository: spack/spack

Issue:
sundials: use CudaPackage and ROCmPackage and add spack tests
Updates the sundials package to use inherit from CudaPackage instead of using our own cuda variant. Also adds support for ROCm and `spack test`.

CC: @gardner48 


Discussion:
Please fix the flake8 issue:

```
var/spack/repos/builtin/packages/sundials/package.py:280: [F841] local variable 'rocm_path' is assigned to but never used
```
FYI.  After running the tests with `$ spack test run --alias sundials sundials`, I get the following output in the Spack test log:

```
$ spack test results -l sundials
==> Results for test suite 'sundials':
==>   sundials-5.7.0-escynwx PASSED
==> Testing package sundials-5.7.0-escynwx
==> [2021-03-09-11:36:53.056806] test: test_nvector_serial: expect command status in [0]
==> [2021-03-09-11:36:53.057248] '$SPACK_ROOT/spack/opt/spack/linux-rhel7-broadwell/gcc-8.3.1/sundials-5.7.0-escynwxqsyr7c5xortqp4izfkrl45moo/.spack/spack-build-escynwx/examples/nvector/serial/test_nvector_serial' '10' '0'
Timer resolution: 1 ns = 1e-09 s
Testing serial N_Vector 
Vector length 10 
PASSED test -- N_VGetVectorID 
PASSED test -- N_VGetLength
PASSED test -- N_VGetCommunicator
PASSED test -- N_VCloneEmpty 
PASSED test -- N_VClone 
PASSED test -- N_VCloneEmptyVectorArray 
PASSED test -- N_VCloneVectorArray 
PASSED test -- N_VSetArrayPointer 
PASSED test -- N_VGetArrayPointer 

Testing standard vector operations:

PASSED test -- N_VConst 
PASSED test -- N_VLinearSum Case 1a 
PASSED test -- N_VLinearSum Case 1b 
PASSED test -- N_VLinearSum Case 1c 
PASSED test -- N_VLinearSum Case 2a 
PASSED test -- N_VLinearSum Case 2b 
PASSED test -- N_VLinearSum Case 2c 
PASSED test -- N_VLinearSum Case 3 
PASSED test -- N_VLinearSum Case 4a 
PASSED test -- N_VLinearSum Case 4b 
PASSED test -- N_VLinearSum Case 5a 
PASSED test -- N_VLinearSum Case 5b 
PASSED test -- N_VLinearSum Case 6a 
PASSED test -- N_VLinearSum Case 6b 
PASSED test -- N_VLinearSum Case 7 
PASSED test -- N_VLinearSum Case 8 
PASSED test -- N_VLinearSum Case 9 
PASSED test -- N_VProd 
PASSED test -- N_VDiv 
PASSED test -- N_VScale Case 1 
PASSED test -- N_VScale Case 2 
PASSED test -- N_VScale Case 3 
PASSED test -- N_VScale Case 4 
PASSED test -- N_VAbs 
PASSED test -- N_VInv 
PASSED test -- N_VAddConst 
PASSED test -- N_VDotProd 
PASSED test -- N_VMaxNorm 
PASSED test -- N_VWrmsNorm 
PASSED test -- N_VWrmsNormMask 
PASSED test -- N_VMin 
PASSED test -- N_VMin 
PASSED test -- N_VWL2Norm 
PASSED test -- N_VL1Norm 
PASSED test -- N_VCompare 
PASSED test -- N_VInvTest Case 1 
PASSED test -- N_VInvTest Case 2 
PASSED test -- N_VConstrMask Case 1 
PASSED test -- N_VConstrMask Case 2 
PASSED test -- N_VMinQuotient Case 1 
PASSED test -- N_VMinQuotient Case 2 

Testing fused and vector array operations (disabled):

PASSED test -- N_VLinearCombination Case 1a 
PASSED test -- N_VLinearCombination Case 1b 
PASSED test -- N_VLinearCombination Case 2a 
PASSED test -- N_VLinearCombination Case 2b 
PASSED test -- N_VLinearCombination Case 3a 
PASSED test -- N_VLinearCombination Case 3b 
PASSED test -- N_VLinearCombination Case 3c 
PASSED test -- N_VScaleAddMulti Case 1a 
PASSED test -- N_VScaleAddMulti Case 1b 
PASSED test -- N_VScaleAddMulti Case 2a 
PASSED test -- N_VScaleAddMulti Case 2b 
PASSED test -- N_VDotProdMulti Case 1 
PASSED test -- N_VDotProdMulti Case 2 
PASSED test -- N_VLinearSumVectorArray Case 0 
PASSED test -- N_VLinearSumVectorArray Case 1a 
PASSED test -- N_VLinearSumVectorArray Case 1b 
PASSED test -- N_VLinearSumVectorArray Case 1c 
PASSED test -- N_VLinearSumVectorArray Case 2a 
PASSED test -- N_VLinearSumVectorArray Case 2b 
PASSED test -- N_VLinearSumVectorArray Case 2c 
PASSED test -- N_VLinearSumVectorArray Case 3 
PASSED test -- N_VLinearSumVectorArray Case 4a 
PASSED test -- N_VLinearSumVectorArray Case 4b 
PASSED test -- N_VLinearSumVectorArray Case 5a 
PASSED test -- N_VLinearSumVectorArray Case 5b 
PASSED test -- N_VLinearSumVectorArray Case 6a 
PASSED test -- N_VLinearSumVectorArray Case 6b 
PASSED test -- N_VLinearSumVectorArray Case 7 
PASSED test -- N_VLinearSumVectorArray Case 8 
PASSED test -- N_VLinearSumVectorArray Case 9 
PASSED test -- N_VScaleVectorArray Case 1a 
PASSED test -- N_VScaleVectorArray Case 1b 
PASSED test -- N_VScaleVectorArray Case 2a 
PASSED test -- N_VScaleVectorArray Case 2b 
PASSED test -- N_VConstVectorArray Case 1a 
PASSED test -- N_VConstVectorArray Case 1b 
PASSED test -- N_VWrmsNormVectorArray Case 1a 
PASSED test -- N_VWrmsNormVectorArray Case 1b 
PASSED test -- N_VWrmsNormMaskVectorArray Case 1 
PASSED test -- N_VWrmsNormMaskVectorArray Case 2 
PASSED test -- N_VScaleAddMultiVectorArray Case 1a 
PASSED test -- N_VScaleAddMultiVectorArray Case 1b 
PASSED test -- N_VScaleAddMultiVectorArray Case 2a 
PASSED test -- N_VScaleAddMultiVectorArray Case 2b 
PASSED test -- N_VScaleAddMultiVectorArray Case 3a 
PASSED test -- N_VScaleAddMultiVectorArray Case 3b 
PASSED test -- N_VScaleAddMultiVectorArray Case 4a 
PASSED test -- N_VScaleAddMultiVectorArray Case 4b 
PASSED test -- N_VLinearCombinationVectorArray Case 1a 
PASSED test -- N_VLinearCombinationVectorArray Case 1b 
PASSED test -- N_VLinearCombinationVectorArray Case 2a 
PASSED test -- N_VLinearCombinationVectorArray Case 2b 
PASSED test -- N_VLinearCombinationVectorArray Case 3a 
PASSED test -- N_VLinearCombinationVectorArray Case 3b 
PASSED test -- N_VLinearCombinationVectorArray Case 4a 
PASSED test -- N_VLinearCombinationVectorArray Case 4b 
PASSED test -- N_VLinearCombinationVectorArray Case 5a 
PASSED test -- N_VLinearCombinationVectorArray Case 5b 
PASSED test -- N_VLinearCombinationVectorArray Case 6a 
PASSED test -- N_VLinearCombinationVectorArray Case 6b 
PASSED test -- N_VLinearCombinationVectorArray Case 6c 

Testing fused and vector array operations (enabled):

PASSED test -- N_VLinearCombination Case 1a 
PASSED test -- N_VLinearCombination Case 1b 
PASSED test -- N_VLinearCombination Case 2a 
PASSED test -- N_VLinearCombination Case 2b 
PASSED test -- N_VLinearCombination Case 3a 
PASSED test -- N_VLinearCombination Case 3b 
PASSED test -- N_VLinearCombination Case 3c 
PASSED test -- N_VScaleAddMulti Case 1a 
PASSED test -- N_VScaleAddMulti Case 1b 
PASSED test -- N_VScaleAddMulti Case 2a 
PASSED test -- N_VScaleAddMulti Case 2b 
PASSED test -- N_VDotProdMulti Case 1 
PASSED test -- N_VDotProdMulti Case 2 
PASSED test -- N_VLinearSumVectorArray Case 0 
PASSED test -- N_VLinearSumVectorArray Case 1a 
PASSED test -- N_VLinearSumVectorArray Case 1b 
PASSED test -- N_VLinearSumVectorArray Case 1c 
PASSED test -- N_VLinearSumVectorArray Case 2a 
PASSED test -- N_VLinearSumVectorArray Case 2b 
PASSED test -- N_VLinearSumVectorArray Case 2c 
PASSED test -- N_VLinearSumVectorArray Case 3 
PASSED test -- N_VLinearSumVectorArray Case 4a 
PASSED test -- N_VLinearSumVectorArray Case 4b 
PASSED test -- N_VLinearSumVectorArray Case 5a 
PASSED test -- N_VLinearSumVectorArray Case 5b 
PASSED test -- N_VLinearSumVectorArray Case 6a 
PASSED test -- N_VLinearSumVectorArray Case 6b 
PASSED test -- N_VLinearSumVectorArray Case 7 
PASSED test -- N_VLinearSumVectorArray Case 8 
PASSED test -- N_VLinearSumVectorArray Case 9 
PASSED test -- N_VScaleVectorArray Case 1a 
PASSED test -- N_VScaleVectorArray Case 1b 
PASSED test -- N_VScaleVectorArray Case 2a 
PASSED test -- N_VScaleVectorArray Case 2b 
PASSED test -- N_VConstVectorArray Case 1a 
PASSED test -- N_VConstVectorArray Case 1b 
PASSED test -- N_VWrmsNormVectorArray Case 1a 
PASSED test -- N_VWrmsNormVectorArray Case 1b 
PASSED test -- N_VWrmsNormMaskVectorArray Case 1 
PASSED test -- N_VWrmsNormMaskVectorArray Case 2 
PASSED test -- N_VScaleAddMultiVectorArray Case 1a 
PASSED test -- N_VScaleAddMultiVectorArray Case 1b 
PASSED test -- N_VScaleAddMultiVectorArray Case 2a 
PASSED test -- N_VScaleAddMultiVectorArray Case 2b 
PASSED test -- N_VScaleAddMultiVectorArray Case 3a 
PASSED test -- N_VScaleAddMultiVectorArray Case 3b 
PASSED test -- N_VScaleAddMultiVectorArray Case 4a 
PASSED test -- N_VScaleAddMultiVectorArray Case 4b 
PASSED test -- N_VLinearCombinationVectorArray Case 1a 
PASSED test -- N_VLinearCombinationVectorArray Case 1b 
PASSED test -- N_VLinearCombinationVectorArray Case 2a 
PASSED test -- N_VLinearCombinationVectorArray Case 2b 
PASSED test -- N_VLinearCombinationVectorArray Case 3a 
PASSED test -- N_VLinearCombinationVectorArray Case 3b 
PASSED test -- N_VLinearCombinationVectorArray Case 4a 
PASSED test -- N_VLinearCombinationVectorArray Case 4b 
PASSED test -- N_VLinearCombinationVectorArray Case 5a 
PASSED test -- N_VLinearCombinationVectorArray Case 5b 
PASSED test -- N_VLinearCombinationVectorArray Case 6a 
PASSED test -- N_VLinearCombinationVectorArray Case 6b 
PASSED test -- N_VLinearCombinationVectorArray Case 6c 

Testing local reduction operations:

PASSED test -- N_VDotProdLocal
PASSED test -- N_VMaxNormLocal
PASSED test -- N_VMinLocal
PASSED test -- N_VL1NormLocal
PASSED test -- N_VWSqrSumLocal
PASSED test -- N_VWSqrSumMaskLocal
PASSED test -- N_VInvTestLocal Case 1
PASSED test -- N_VInvTestLocal Case 2
PASSED test -- N_VConstrMaskLocal Case 1 
PASSED test -- N_VConstrMaskLocal Case 2 
PASSED test -- N_VMinQuotientLocal Case 1 
PASSED test -- N_VMinQuotientLocal Case 2 

Testing XBraid interface operations:

PASSED test -- N_VBufSize
PASSED test -- N_VBufPack
PASSED test -- N_VBufUnpack
SUCCESS: NVector module passed all tests 

PASSED

```
@gardner48 Do you want to review this PR?
> Minor typo to fix but otherwise this looks good to me.

Great catch!

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/sundials/package.py b/var/spack/repos/builtin/packages/sundials/package.py
--- a/var/spack/repos/builtin/packages/sundials/package.py
+++ b/var/spack/repos/builtin/packages/sundials/package.py
@@ -202,6 +202,8 @@ class Sundials(CMakePackage, CudaPackage, ROCmPackage):
     patch('FindPackageMultipass.cmake.patch', when='@5.0.0')
     patch('5.5.0-xsdk-patches.patch', when='@5.5.0')
     patch('0001-add-missing-README-to-examples-cvode-hip.patch', when='@5.6.0:5.7.0')
+    # remove sundials_nvecopenmp target from ARKODE SuperLU_DIST example
+    patch('remove-links-to-OpenMP-vector.patch', when='@5.5.0:5.7.0')
 
     # ==========================================================================
     # SUNDIALS Settings


Test patch: