Repository: spack/spack

Issue:
netcdf and hdf5 always have conflicting specs
Trying to build `adios` with `+netcdf+hdf5` support but I keep getting conflicting specs due to the +mpi part of the specs for both. 

### Expected Result
Concretized spec of adios with adios built with netcdf+mpi and hdf5 +mpi

### Actual Result
```console
spack install adios+hdf5+netcdf
==> Error: Conflicts in concretized spec "adios@1.13.0%cce@8.6.2+blosc~bzip2~fortran+hdf5~infiniband+lz4+mpi+netcdf patches=8ae17f655248e87cbab1d1ed794e15364a38d2f5f8d971b1086702f72d79bd42 ~shared staging= +sz~szip+zfp+zlib arch=cray-cnl6-haswell /b2brmth"

  List of matching conflicts for spec:

    netcdf@4.4.1.1.3%cce@8.6.2~dap~hdf4 maxdims=1024 maxvars=8192 +mpi+parallel-netcdf+shared arch=cray-cnl6-haswell

1. "+mpi" conflicts with "netcdf^hdf5~mpi" [netcdf+mpi requires hdf5+mpi]
```

### Steps to reproduce the issue
Log into Cori
```console
spack install adios+hdf5+netcdf
```

### Information on your system

Cori (Cray-XC40)
custom packages.yaml
```yaml
packages:
    all:
        compiler: [intel@18.0.1.163, gcc@7.1.0, cce@8.6.2]
        variants: ~shared
        providers:
            mpi: [mpich]
            mkl: [intel-mkl]
            blas: [intel-mkl, cray-libsci]
            scalapack: [intel-mkl, cray-libsci]
            pkgconfig: [pkg-config]
    cmake:
        buildable: false
        paths:
            cmake@3.5.2: /usr
    pkg-config:
        buildable: false
        paths:
            pkg-config@0.28: /usr
    mpich:
        buildable: false
        modules:
            mpich@7.6.2%gcc+pmi+romio: cray-mpich
            mpich@7.6.2%intel+mpi+romio: cray-mpich
            mpich@7.6.2%cce+mpi+romio: cray-mpich
    intel-mkl:
        buildable: false
        paths:
            intel-mkl@2018.1.163%gcc+ilp64 threads=openmp: /opt/intel
            intel-mkl@2018.1.163%intel+ilp64 threads=openmp: /opt/intel
            intel-mkl@2018.1.163%cce+ilp64 threads=openmp: /opt/intel
    fftw:
        buildable: false
        modules:
            fftw@3.3.6.2%gcc+openmp: cray-fftw
            fftw@3.3.6.2%intel+openmp: cray-fftw
            fftw@3.3.6.2%cce+openmp: cray-fftw
    hdf5:
        buildable: false
        variants: +mpi
        modules:
            hdf5@1.10.0.3%gcc+cxx+fortran+hl+szip+mpi: cray-netcdf-hdf5parallel
            hdf5@1.10.0.3%intel+cxx+fortran+hl+szip+mpi: cray-netcdf-hdf5parallel
            hdf5@1.10.0.3%cce+cxx+fortran+hl+szip+mpi: cray-netcdf-hdf5parallel
    cray-libsci:
        buildable: false
        modules:
            cray-libsci@17.09.1%intel: cray-libsci
            cray-libsci@17.09.1%gcc: cray-libsci
            cray-libsci@17.09.1%cce: cray-libsci
    netcdf:
        buildable: false
        variants: +mpi
        modules:
            netcdf@4.4.1.1.3%intel+mpi+parallel-netcdf^hdf5+mpi: cray-netcdf-hdf5parallel
            netcdf@4.4.1.1.3%gcc+mpi+parallel-netcdf^hdf5+mpi: cray-netcdf-hdf5parallel
            netcdf@4.4.1.1.3%cce+mpi+parallel-netcdf^hdf5+mpi: cray-netcdf-hdf5parallel
    petsc:
        buildable: false
        modules:
            petsc@3.7.2%intel+boost~complex+mumps~int64: cray-petsc
            petsc@3.7.2%gcc+boost~complex+mumps~int64: cray-petsc
            petsc@3.7.2%cce+boost~complex+mumps~int64: cray-petsc
            petsc@3.7.2%intel+boost~complex+mumps+int64: cray-petsc-64
            petsc@3.7.2%gcc+boost~complex+mumps+int64: cray-petsc-64
            petsc@3.7.2%cce+boost~complex+mumps+int64: cray-petsc-64
            petsc@3.7.2%intel+boost+complex+mumps~int64: cray-petsc-complex
            petsc@3.7.2%gcc+boost+complex+mumps~int64: cray-petsc-complex
            petsc@3.7.2%cce+boost+complex+mumps~int64: cray-petsc-complex
            petsc@3.7.2%intel+boost+complex+mumps+int64: cray-petsc-complex-64
            petsc@3.7.2%gcc+boost+complex+mumps+int64: cray-petsc-complex-64
            petsc@3.7.2%cce+boost+complex+mumps+int64: cray-petsc-complex-64
    papi:
        buildable: false
        modules:
            papi@5.5.1.3%gcc: papi
            papi@5.5.1.3%intel: papi
            papi@5.5.1.3%cce: papi
    trilinos:
        buildable: false
        modules:
            trilinos@12.10.1.1%intel+fortrilinos+intrepid+nox+openmp+pnetcdf+rol+shards+stk+teuchos+tpetra: cray-trilinos
            trilinos@12.10.1.1%gcc+fortrilinos+intrepid+nox+openmp+pnetcdf+rol+shards+stk+teuchos+tpetra: cray-trilinos
            trilinos@12.10.1.1%cce+fortrilinos+intrepid+nox+openmp+pnetcdf+rol+shards+stk+teuchos+tpetra: cray-trilinos
```


Discussion:
I'm having trouble reproducing this. It doesn't occur with a fresh clone of Spack and no `packages.yaml`. I couldn't use your exact `packages.yaml` because I don't have any of that software installed, but if I pare it down I can't reproduce it either.

There are numerous bugs with the concretizer that cause problems like this. For now, you should be able to get around it if you do something like:
```
$ spack spec adios +netcdf +hdf5 ^netcdf+mpi ^hdf5+mpi
```
If that doesn't work, try relaxing your package specs in `packages.yaml`.
```console
spack install adios+hdf5+netcdf ^netcdf+mpi ^hdf5+mpi
v==> Error: Conflicts in concretized spec "adios@1.13.0%cce@8.6.2+blosc~bzip2~fortran+hdf5~infiniband+lz4+mpi+netcdf patches=8ae17f655248e87cbab1d1ed794e15364a38d2f5f8d971b1086702f72d79bd42 ~shared staging= +sz~szip+zfp+zlib arch=cray-cnl6-haswell /zcfeinz"

  List of matching conflicts for spec:

    netcdf@4.4.1.1.3%cce@8.6.2~dap~hdf4 maxdims=1024 maxvars=8192 +mpi~parallel-netcdf~shared arch=cray-cnl6-haswell

1. "+mpi" conflicts with "netcdf^hdf5~mpi" [netcdf+mpi requires hdf5+mpi]
```

packages.yaml
```yaml
packages:
    all:
        compiler: [intel@18.0.1.163, gcc@7.1.0, cce@8.6.2]
        variants: ~shared
        craype_link_type: "static"
        providers:
            mpi: [mpich]
            mkl: [intel-mkl]
            blas: [intel-mkl, cray-libsci]
            scalapack: [intel-mkl, cray-libsci]
            pkgconfig: [pkg-config]
    cmake:
        buildable: false
        paths:
            cmake@3.5.2: /usr
    pkg-config:
        buildable: false
        paths:
            pkg-config@0.28: /usr
    mpich:
        buildable: false
        modules:
            mpich@7.6.2%gcc+pmi+romio: cray-mpich
            mpich@7.6.2%intel+mpi+romio: cray-mpich
            mpich@7.6.2%cce+mpi+romio: cray-mpich
    intel-mkl:
        buildable: false
        paths:
            intel-mkl@2018.1.163%gcc+ilp64 threads=openmp: /opt/intel
            intel-mkl@2018.1.163%intel+ilp64 threads=openmp: /opt/intel
            intel-mkl@2018.1.163%cce+ilp64 threads=openmp: /opt/intel
    fftw:
        buildable: false
        modules:
            fftw@3.3.6.2%gcc+openmp: cray-fftw
            fftw@3.3.6.2%intel+openmp: cray-fftw
            fftw@3.3.6.2%cce+openmp: cray-fftw
    hdf5:
        buildable: false
        modules:
            hdf5@1.10.0.3%gcc+hl: cray-netcdf-hdf5parallel
            hdf5@1.10.0.3%intel+hl: cray-netcdf-hdf5parallel
            hdf5@1.10.0.3%cce+hl: cray-netcdf-hdf5parallel
    cray-libsci:
        buildable: false
        modules:
            cray-libsci@17.09.1%intel: cray-libsci
            cray-libsci@17.09.1%gcc: cray-libsci
            cray-libsci@17.09.1%cce: cray-libsci
    netcdf:
        buildable: false
        modules:
            netcdf@4.4.1.1.3%intel: cray-netcdf-hdf5parallel
            netcdf@4.4.1.1.3%gcc: cray-netcdf-hdf5parallel
            netcdf@4.4.1.1.3%cce: cray-netcdf-hdf5parallel
    petsc:
        buildable: false
        modules:
            petsc@3.7.2%intel+boost~complex+mumps~int64: cray-petsc
            petsc@3.7.2%gcc+boost~complex+mumps~int64: cray-petsc
            petsc@3.7.2%cce+boost~complex+mumps~int64: cray-petsc
            petsc@3.7.2%intel+boost~complex+mumps+int64: cray-petsc-64
            petsc@3.7.2%gcc+boost~complex+mumps+int64: cray-petsc-64
            petsc@3.7.2%cce+boost~complex+mumps+int64: cray-petsc-64
            petsc@3.7.2%intel+boost+complex+mumps~int64: cray-petsc-complex
            petsc@3.7.2%gcc+boost+complex+mumps~int64: cray-petsc-complex
            petsc@3.7.2%cce+boost+complex+mumps~int64: cray-petsc-complex
            petsc@3.7.2%intel+boost+complex+mumps+int64: cray-petsc-complex-64
            petsc@3.7.2%gcc+boost+complex+mumps+int64: cray-petsc-complex-64
            petsc@3.7.2%cce+boost+complex+mumps+int64: cray-petsc-complex-64
    papi:
        buildable: false
        modules:
            papi@5.5.1.3%gcc: papi
            papi@5.5.1.3%intel: papi
            papi@5.5.1.3%cce: papi
    trilinos:
        buildable: false
        modules:
            trilinos@12.10.1.1%intel+fortrilinos+intrepid+nox+openmp+pnetcdf+rol+shards+stk+teuchos+tpetra: cray-trilinos
            trilinos@12.10.1.1%gcc+fortrilinos+intrepid+nox+openmp+pnetcdf+rol+shards+stk+teuchos+tpetra: cray-trilinos
            trilinos@12.10.1.1%cce+fortrilinos+intrepid+nox+openmp+pnetcdf+rol+shards+stk+teuchos+tpetra: cray-trilinos
```

I relaxed the constraints a bit but still getting that error  
Can you try removing things from your `packages.yaml` one-by-one to get down to a minimal working example? Also, are you sure you only have a single `packages.yaml` and there aren't any others that would cause `hdf5~mpi` to be preferred?
I can replicate the issue with the given `packages.yaml`. I'll be looking into this more later today.
No other packages.yaml and I got rid of the hdf5 entry but still same problem.
Adding netcdf entry seems to cause the problem
Thanks for the update. This is a Spack bug which relates to some internals: the external netcdf has all dependencies stripped off of it, this interferes with the conflict check (which is not executed with `strict=True`) so it basically is overzealous in reporting a possible conflict. I'll see if I can work on a PR later today or tomorrow morning.
Ah, I see it now. The spec is actually correct but it satisfies all of the conflicting specs. It dives into satisfies_dependencies and since we have a check if there are no dependencies we return True.

Resolving patch:
diff --git a/lib/spack/spack/spec.py b/lib/spack/spack/spec.py
--- a/lib/spack/spack/spec.py
+++ b/lib/spack/spack/spec.py
@@ -1869,9 +1869,9 @@ def concretize(self):
         matches = []
         for x in self.traverse():
             for conflict_spec, when_list in x.package.conflicts.items():
-                if x.satisfies(conflict_spec):
+                if x.satisfies(conflict_spec, strict=True):
                     for when_spec, msg in when_list:
-                        if x.satisfies(when_spec):
+                        if x.satisfies(when_spec, strict=True):
                             matches.append((x, conflict_spec, when_spec, msg))
         if matches:
             raise ConflictsInSpecError(self, matches)


Test patch: