Repository: spack/spack

Issue:
ASP based solver errors on multi-valued variants with disjoint sets of values
This error has been reported by @bvanessen When trying to concretize a spec that is modeled with multi valued variants from a disjoint set of values the ASP based solver errors out in the spec build phase (i.e. `clingo` completes successfully and the error happens when constructing the spec from its atoms)

### Steps to reproduce the issue

```console
$ spack solve mvapich2 file_systems=lustre
==> Error: the value 'auto' is mutually exclusive with any of the other values [mvapich2, variant 'file_systems']
```

### Error Message

<details>

<summary>$ spack -d solve mvapich2 file_systems=lustre </summary>

```console
==> [2021-03-25-08:12:47.446725] Imported solve from built-in commands
==> [2021-03-25-08:12:47.447836] Imported solve from built-in commands
==> [2021-03-25-08:12:47.448782] [BOOSTRAP CONFIG SCOPE] name=defaults, path=/home/culpo/PycharmProjects/spack/etc/spack/defaults
==> [2021-03-25-08:12:47.448855] [BOOSTRAP CONFIG SCOPE] name=defaults/linux, path=/home/culpo/PycharmProjects/spack/etc/spack/defaults/linux
==> [2021-03-25-08:12:47.448929] [BOOSTRAP CONFIG SCOPE] name=system, path=/etc/spack
==> [2021-03-25-08:12:47.448979] [BOOSTRAP CONFIG SCOPE] name=system/linux, path=/etc/spack/linux
==> [2021-03-25-08:12:47.449043] [BOOSTRAP CONFIG SCOPE] name=site, path=/home/culpo/PycharmProjects/spack/etc/spack
==> [2021-03-25-08:12:47.449107] [BOOSTRAP CONFIG SCOPE] name=site/linux, path=/home/culpo/PycharmProjects/spack/etc/spack/linux
==> [2021-03-25-08:12:47.449184] [BOOSTRAP CONFIG SCOPE] name=user, path=/home/culpo/.spack
==> [2021-03-25-08:12:47.449245] [BOOSTRAP CONFIG SCOPE] name=user/linux, path=/home/culpo/.spack/linux
==> [2021-03-25-08:12:47.451080] Reading config file /home/culpo/PycharmProjects/spack/etc/spack/defaults/config.yaml
==> [2021-03-25-08:12:47.477033] DATABASE LOCK TIMEOUT: 3s
==> [2021-03-25-08:12:47.477138] PACKAGE LOCK TIMEOUT: No timeout
==> [2021-03-25-08:12:47.477684] DATABASE LOCK TIMEOUT: 3s
==> [2021-03-25-08:12:47.477755] PACKAGE LOCK TIMEOUT: No timeout
==> [2021-03-25-08:12:47.553399] 'python3' '-V'
==> [2021-03-25-08:12:47.557075] 'python3' '-c' 'import readline'
==> [2021-03-25-08:12:47.591195] 'python3' '-c' 'import sqlite3'
==> [2021-03-25-08:12:47.625069] 'python3' '-c' 'import dbm'
==> [2021-03-25-08:12:47.661961] 'python3' '-c' 'import nis'
==> [2021-03-25-08:12:47.694440] 'python3' '-c' 'import zlib'
==> [2021-03-25-08:12:47.726282] 'python3' '-c' 'import bz2'
==> [2021-03-25-08:12:47.768297] 'python3' '-c' 'import lzma'
==> [2021-03-25-08:12:47.800598] 'python3' '-c' 'import ctypes'
==> [2021-03-25-08:12:47.834796] 'python3' '-c' 'import uuid'
==> [2021-03-25-08:12:47.892704] 'python3' '-c' 'import ssl'
==> [2021-03-25-08:12:47.944485] 'python3' '-c' 'import hashlib'
==> [2021-03-25-08:12:47.978741] 'python3' '-c' 'import xml.parsers.expat'
==> [2021-03-25-08:12:48.009653] 'python3' '-c' 'import xml.etree.ElementTree'
==> [2021-03-25-08:12:48.046152] 'python3' '-c' 'import tkinter'
==> [2021-03-25-08:12:48.089252] 'python3' '-c' 'import tkinter.tix'
==> [2021-03-25-08:12:48.141585] Reading config file /home/culpo/PycharmProjects/spack/etc/spack/defaults/config.yaml
==> [2021-03-25-08:12:48.194827] diffutils applying constraint iconv
==> [2021-03-25-08:12:48.222438] ncurses applying constraint pkgconfig
==> [2021-03-25-08:12:48.223292] readline applying constraint ncurses
==> [2021-03-25-08:12:48.224121] gdbm applying constraint readline
==> [2021-03-25-08:12:48.226423] perl applying constraint gdbm
==> [2021-03-25-08:12:48.227357] perl applying constraint berkeley-db
==> [2021-03-25-08:12:48.234209] gettext applying constraint iconv
==> [2021-03-25-08:12:48.237347] help2man applying constraint perl
==> [2021-03-25-08:12:48.238617] help2man applying constraint gettext
==> [2021-03-25-08:12:48.239984] help2man applying constraint perl
==> [2021-03-25-08:12:48.240864] bison applying constraint diffutils
==> [2021-03-25-08:12:48.241715] bison applying constraint m4
==> [2021-03-25-08:12:48.242550] bison applying constraint perl
==> [2021-03-25-08:12:48.243394] bison applying constraint help2man
==> [2021-03-25-08:12:48.244738] python applying constraint pkgconfig@0.9.0:
==> [2021-03-25-08:12:48.246280] python applying constraint pkgconfig@0.9.0:
==> [2021-03-25-08:12:48.248275] clingo-bootstrap applying constraint cmake@3.16.0:
==> [2021-03-25-08:12:48.249436] clingo-bootstrap applying constraint re2c@0.13:
==> [2021-03-25-08:12:48.251606] clingo-bootstrap applying constraint bison@2.5:
==> [2021-03-25-08:12:48.253399] clingo-bootstrap applying constraint python
==> [2021-03-25-08:12:48.255170] Reading config file /home/culpo/PycharmProjects/spack/etc/spack/defaults/packages.yaml
==> [2021-03-25-08:12:48.289612] Reading config file /home/culpo/.spack/packages.yaml
==> [2021-03-25-08:12:48.301704] clingo-bootstrap applying constraint cmake@3.16.0:
==> [2021-03-25-08:12:48.304159] clingo-bootstrap applying constraint re2c@0.13:
==> [2021-03-25-08:12:48.305249] clingo-bootstrap applying constraint bison@2.5:
==> [2021-03-25-08:12:48.306075] clingo-bootstrap applying constraint python
==> [2021-03-25-08:12:48.306668] python applying constraint pkgconfig@0.9.0:
==> [2021-03-25-08:12:48.311350] clingo-bootstrap applying constraint cmake@3.16.0:
==> [2021-03-25-08:12:48.312333] clingo-bootstrap applying constraint re2c@0.13:
==> [2021-03-25-08:12:48.313516] clingo-bootstrap applying constraint bison@2.5:
==> [2021-03-25-08:12:48.314464] clingo-bootstrap applying constraint python
==> [2021-03-25-08:12:48.315058] python applying constraint pkgconfig@0.9.0:
==> [2021-03-25-08:12:48.321237] clingo-bootstrap applying constraint cmake@3.16.0:
==> [2021-03-25-08:12:48.322115] clingo-bootstrap applying constraint re2c@0.13:
==> [2021-03-25-08:12:48.323043] clingo-bootstrap applying constraint bison@2.5:
==> [2021-03-25-08:12:48.323843] clingo-bootstrap applying constraint python
==> [2021-03-25-08:12:48.327842] Reading config file /home/culpo/.spack/linux/compilers.yaml
==> [2021-03-25-08:12:48.430240] clingo-bootstrap applying constraint cmake@3.16.0:
==> [2021-03-25-08:12:48.431444] clingo-bootstrap applying constraint re2c@0.13:
==> [2021-03-25-08:12:48.433107] clingo-bootstrap applying constraint bison@2.5:
==> [2021-03-25-08:12:48.434075] clingo-bootstrap applying constraint python
==> [2021-03-25-08:12:48.453689] Ordered hashes [bison]: bison/25/b72914fe38e54a6fc25f29019e0a0786705c4f61ce20d414cc2010c8d62448c7
==> [2021-03-25-08:12:48.454144] Ordered hashes [cmake]: cmake/23/b48396c0e4f61756248156b6cebe9bc0d7a22228639b47b5aa77c9330588ce88
==> [2021-03-25-08:12:48.520187] Reading config file /home/culpo/PycharmProjects/spack/etc/spack/defaults/repos.yaml
==> [2021-03-25-08:12:49.629931] Reading config file /home/culpo/.spack/linux/compilers.yaml
==> [2021-03-25-08:12:49.759667] Reading config file /home/culpo/PycharmProjects/spack/etc/spack/defaults/packages.yaml
==> [2021-03-25-08:12:49.796526] Reading config file /home/culpo/.spack/packages.yaml
==> [2021-03-25-08:12:58.188154] version_weight(autoconf, 0)
==> [2021-03-25-08:12:58.188255] version_weight(automake, 0)
==> [2021-03-25-08:12:58.188302] version_weight(berkeley-db, 0)
==> [2021-03-25-08:12:58.188353] version_weight(bison, 18)
==> [2021-03-25-08:12:58.188402] version_weight(bzip2, 0)
==> [2021-03-25-08:12:58.188444] version_weight(cmake, 0)
==> [2021-03-25-08:12:58.188484] version_weight(diffutils, 0)
==> [2021-03-25-08:12:58.188524] version_weight(expat, 0)
==> [2021-03-25-08:12:58.188563] version_weight(findutils, 0)
==> [2021-03-25-08:12:58.188602] version_weight(flex, 0)
==> [2021-03-25-08:12:58.188658] version_weight(gdbm, 0)
==> [2021-03-25-08:12:58.188713] version_weight(gettext, 0)
==> [2021-03-25-08:12:58.188776] version_weight(help2man, 0)
==> [2021-03-25-08:12:58.188834] version_weight(libbsd, 0)
==> [2021-03-25-08:12:58.188892] version_weight(libffi, 0)
==> [2021-03-25-08:12:58.188945] version_weight(libiconv, 0)
==> [2021-03-25-08:12:58.188997] version_weight(libnl, 0)
==> [2021-03-25-08:12:58.189049] version_weight(libpciaccess, 0)
==> [2021-03-25-08:12:58.189100] version_weight(libsigsegv, 0)
==> [2021-03-25-08:12:58.189151] version_weight(libtool, 0)
==> [2021-03-25-08:12:58.189203] version_weight(libxml2, 0)
==> [2021-03-25-08:12:58.189254] version_weight(m4, 0)
==> [2021-03-25-08:12:58.189306] version_weight(mvapich2, 0)
==> [2021-03-25-08:12:58.189356] version_weight(ncurses, 0)
==> [2021-03-25-08:12:58.189407] version_weight(openssl, 0)
==> [2021-03-25-08:12:58.189458] version_weight(perl, 0)
==> [2021-03-25-08:12:58.189509] version_weight(pkgconf, 0)
==> [2021-03-25-08:12:58.189561] version_weight(py-docutils, 0)
==> [2021-03-25-08:12:58.189612] version_weight(py-setuptools, 0)
==> [2021-03-25-08:12:58.189663] version_weight(python, 0)
==> [2021-03-25-08:12:58.189713] version_weight(rdma-core, 0)
==> [2021-03-25-08:12:58.189764] version_weight(readline, 0)
==> [2021-03-25-08:12:58.189815] version_weight(sqlite, 0)
==> [2021-03-25-08:12:58.189867] version_weight(tar, 0)
==> [2021-03-25-08:12:58.189917] version_weight(texinfo, 0)
==> [2021-03-25-08:12:58.189968] version_weight(util-linux-uuid, 0)
==> [2021-03-25-08:12:58.190019] version_weight(util-macros, 0)
==> [2021-03-25-08:12:58.190070] version_weight(xz, 0)
==> [2021-03-25-08:12:58.190120] version_weight(zlib, 0)
==> [2021-03-25-08:12:58.190171] version_weight(bison, -1)
==> [2021-03-25-08:12:58.191341] provider_weight(bison, 0)
==> [2021-03-25-08:12:58.191413] provider_weight(libiconv, 10)
==> [2021-03-25-08:12:58.191457] provider_weight(pkgconf, 10)
==> [2021-03-25-08:12:58.191495] provider_weight(util-linux-uuid, 10)
==> [2021-03-25-08:12:58.192740] SpecError: the value 'auto' is mutually exclusive with any of the other values [mvapich2, variant 'file_systems']
==> [2021-03-25-08:12:58.192802] Error: the value 'auto' is mutually exclusive with any of the other values [mvapich2, variant 'file_systems']
Traceback (most recent call last):
  File "/home/culpo/PycharmProjects/spack/lib/spack/spack/main.py", line 768, in main
    return _invoke_command(command, parser, args, unknown)
  File "/home/culpo/PycharmProjects/spack/lib/spack/spack/main.py", line 496, in _invoke_command
    return_val = command(parser, args)
  File "/home/culpo/PycharmProjects/spack/lib/spack/spack/cmd/solve.py", line 100, in solve
    specs, dump=dump, models=models, timers=args.timers, stats=args.stats
  File "/home/culpo/PycharmProjects/spack/lib/spack/spack/solver/asp.py", line 1619, in solve
    return driver.solve(setup, specs, dump, models, timers, stats, tests)
  File "/home/culpo/PycharmProjects/spack/lib/spack/spack/solver/asp.py", line 366, in solve
    answers = builder.build_specs(tuples)
  File "/home/culpo/PycharmProjects/spack/lib/spack/spack/solver/asp.py", line 1544, in build_specs
    action(*args)
  File "/home/culpo/PycharmProjects/spack/lib/spack/spack/solver/asp.py", line 1421, in variant_value
    self._specs[pkg].update_variant_validate(name, value)
  File "/home/culpo/PycharmProjects/spack/lib/spack/spack/spec.py", line 3012, in update_variant_validate
    self.variants[variant_name], self.package)
  File "/home/culpo/PycharmProjects/spack/lib/spack/spack/variant.py", line 132, in validate_or_raise
    self.group_validator(pkg.name, self.name, value)
  File "/home/culpo/PycharmProjects/spack/lib/spack/spack/variant.py", line 766, in _disjoint_set_validator
    raise error.SpecError(msg)
spack.error.SpecError: the value 'auto' is mutually exclusive with any of the other values [mvapich2, variant 'file_systems']
```

</details>

### Information on your system

* **Spack:** 0.16.1-1885-38841ad746
* **Python:** 3.6.9
* **Platform:** linux-ubuntu18.04-broadwell
* **Concretizer:** clingo

### Additional information

- [x] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [x] I have searched the issues of this repo and believe this is not a duplicate
- [x] I have run the failing commands in debug mode and reported the output




Discussion:
Further investigating this issue, the builder is correct since both these atoms are in the computed solution:
```
[ ... ]
==> [2021-03-25-08:46:34.299918] node_target(py-docutils, broadwell)
==> [2021-03-25-08:46:34.299970] variant_value(mvapich2, file_systems, lustre) <---
==> [2021-03-25-08:46:34.300177] variant_value(mvapich2, alloca, False)
==> [2021-03-25-08:46:34.300295] variant_value(mvapich2, ch3_rank_bits, 32)
==> [2021-03-25-08:46:34.300394] variant_value(mvapich2, cuda, False)
==> [2021-03-25-08:46:34.300479] variant_value(mvapich2, debug, False)
==> [2021-03-25-08:46:34.300559] variant_value(mvapich2, fabrics, mrail)
==> [2021-03-25-08:46:34.300653] variant_value(mvapich2, file_systems, auto) <---
[ ... ]
```
Possibly solved by an additional integrity constraint:
```diff
diff --git a/lib/spack/spack/solver/concretize.lp b/lib/spack/spack/solver/concretize.lp
index 0a73ba9eda..33f8a107e6 100644
--- a/lib/spack/spack/solver/concretize.lp
+++ b/lib/spack/spack/solver/concretize.lp
@@ -350,6 +350,11 @@ variant_set(Package, Variant) :- variant_set(Package, Variant, _).
 % A variant cannot have a value that is not also a possible value
 :- variant_value(Package, Variant, Value), not variant_possible_value(Package, Variant, Value).
 
+% If a variant has been set to something explicitly, prevent adding other values to it
+% This is relevant for multi-valued variants, in particular the ones allowing disjoint
+% sets of values
+:- variant_set(Package, Variant), variant_value(Package, Variant, Value), not variant_set(Package, Variant, Value).
+
 % variant_set is an explicitly set variant value. If it's not 'set',
 % we revert to the default value. If it is set, we force the set value
 variant_value(Package, Variant, Value)
```

Resolving patch:
diff --git a/lib/spack/spack/solver/asp.py b/lib/spack/spack/solver/asp.py
--- a/lib/spack/spack/solver/asp.py
+++ b/lib/spack/spack/solver/asp.py
@@ -593,11 +593,16 @@ def pkg_rules(self, pkg, tests):
                 values = []
             elif isinstance(values, spack.variant.DisjointSetsOfValues):
                 union = set()
-                for s in values.sets:
+                # Encode the disjoint sets in the logic program
+                for sid, s in enumerate(values.sets):
+                    for value in s:
+                        self.gen.fact(fn.variant_value_from_disjoint_sets(
+                            pkg.name, name, value, sid
+                        ))
                     union.update(s)
                 values = union
 
-            # make sure that every variant has at least one posisble value
+            # make sure that every variant has at least one possible value
             if not values:
                 values = [variant.default]
 
diff --git a/var/spack/repos/builtin.mock/packages/mvapich2/package.py b/var/spack/repos/builtin.mock/packages/mvapich2/package.py
new file mode 100644
--- /dev/null
+++ b/var/spack/repos/builtin.mock/packages/mvapich2/package.py
@@ -0,0 +1,15 @@
+# Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
+# Spack Project Developers. See the top-level COPYRIGHT file for details.
+#
+# SPDX-License-Identifier: (Apache-2.0 OR MIT)
+class Mvapich2(Package):
+    homepage = "http://www.homepage.org"
+    url = "http://www.someurl"
+
+    version('1.5', '9c5d5d4fe1e17dd12153f40bc5b6dbc0')
+
+    variant(
+        'file_systems',
+        description='List of the ROMIO file systems to activate',
+        values=auto_or_any_combination_of('lustre', 'gpfs', 'nfs', 'ufs'),
+    )


Test patch:
diff --git a/lib/spack/spack/test/concretize.py b/lib/spack/spack/test/concretize.py
--- a/lib/spack/spack/test/concretize.py
+++ b/lib/spack/spack/test/concretize.py
@@ -1138,3 +1138,35 @@ def test_error_message_for_inconsistent_variants(self, spec_str):
         s = Spec(spec_str)
         with pytest.raises(RuntimeError, match='not found in package'):
             s.concretize()
+
+    @pytest.mark.regression('22533')
+    @pytest.mark.parametrize('spec_str,variant_name,expected_values', [
+        # Test the default value 'auto'
+        ('mvapich2', 'file_systems', ('auto',)),
+        # Test setting a single value from the disjoint set
+        ('mvapich2 file_systems=lustre', 'file_systems', ('lustre',)),
+        # Test setting multiple values from the disjoint set
+        ('mvapich2 file_systems=lustre,gpfs', 'file_systems',
+         ('lustre', 'gpfs')),
+    ])
+    def test_mv_variants_disjoint_sets_from_spec(
+            self, spec_str, variant_name, expected_values
+    ):
+        s = Spec(spec_str).concretized()
+        assert set(expected_values) == set(s.variants[variant_name].value)
+
+    @pytest.mark.regression('22533')
+    def test_mv_variants_disjoint_sets_from_packages_yaml(self):
+        external_mvapich2 = {
+            'mvapich2': {
+                'buildable': False,
+                'externals': [{
+                    'spec': 'mvapich2@2.3.1 file_systems=nfs,ufs',
+                    'prefix': '/usr'
+                }]
+            }
+        }
+        spack.config.set('packages', external_mvapich2)
+
+        s = Spec('mvapich2').concretized()
+        assert set(s.variants['file_systems'].value) == set(['ufs', 'nfs'])