Repository: spack/spack

Issue:
Bug: comparisons between abstract specs can raise -> spack spec callpath fails
<!-- Explain, in a clear and concise way, the command you ran and the result you were trying to achieve.
Example: "I ran `spack find` to list all the installed packages and ..." -->

I installed Spack v0.16, and I tried to install callpath. I get the error message below.

I discussed the issue with @becker33, and he says it looks like there is a bug in the dyninst package and the elf providers, so it fails if there is a dependance on dyninst and elf. It looks like the error is releated to `elfutils`.

The error does not show up in Spack v0.15.

### Steps to reproduce the issue

Install Spack, and checkout v0.16

```console
$ spack install callpath
```

### Error Message

```console
Error: '<' not supported between instances of 'str' and 'NoneType'
```

### Information on your system

I run on CentOS 7.

<!-- Please include the output of `spack debug report` -->

<!-- If you have any relevant configuration detail (custom `packages.yaml` or `modules.yaml`, etc.) you can add that here as well. -->

### Additional information

<!-- These boxes can be checked by replacing [ ] with [x] or by clicking them after submitting the issue. -->
- [ ] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [x] I have searched the issues of this repo and believe this is not a duplicate
- [ ] I have run the failing commands in debug mode and reported the output

<!-- We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively!

If you want to ask a question about the tool (how to use it, what it can currently do, etc.), try the `#general` channel on our Slack first. We have a welcoming community and chances are you'll get your reply faster and without opening an issue.

Other than that, thanks for taking the time to contribute to Spack! -->



Discussion:
Concretizing `callpath` succeeds when built with `dyninst@:9.2` (when it `dyninst` depends on `libelf` instead of `elfutils`).

It also succeeds when either the `dyninst` or `elf` dependencies are commented out.
The problem is related to how callpath and dyninst specify their
dependencies on elf, elfutils and libelf.  In commit c786eb46bb
(PR #18063), dyninst replaced elf@0/1 with elfutils or libelf.

If you replace the callpath depends_on to elfutils, then it
concretizes (I haven't tried building).

IMO, the abstraction of elf@0/1, elfutils, and libelf has never really
worked.  Libelf and libdwarf are old, elfutils is new and pretending
they are interchangeable is a fiction.  A package needs to know if
it's using old or new.

In this case, I don't use callpath, so I don't really know if it would
work with elfutils, libelf or both.  Depends on how actively callpath
is maintained.

ping @hainest

@becker33 Btw, can we get a more informative error message here?
`'<' not supported between instances of 'str' and 'NoneType'`
doesn't really identify what the problem is.

Even `spack -d spec callpath` isn't much better.

```
  File "/home/krentel/comgr/spack-repo/lib/spack/spack/spec.py", line 2674, in _merge_dependency
    provider = self._find_provider(dep, provider_index)
  File "/home/krentel/comgr/spack-repo/lib/spack/spack/spec.py", line 2609, in _find_provider
    providers = provider_index.providers_for(vdep)
  File "/home/krentel/comgr/spack-repo/lib/spack/spack/provider_index.py", line 78, in providers_for
    return sorted(s.copy() for s in result)
  File "/home/krentel/comgr/spack-repo/lib/spack/llnl/util/lang.py", line 249, in <lambda>
    lambda s, o: o is not None and s._cmp_key() < o._cmp_key())
TypeError: '<' not supported between instances of 'NoneType' and 'str'
```

The real problem is conflicting versions of elf and elfutils, but none
of the errors identify elf as part of the problem.  The stack trace
hints at merging specs, but it never says elf.
Yeah I haven't yet been able to figure out exactly what the bug is causing that error (vs a good error message for an incompatibility). But that definitely needs to change.
Ok, so after much digging this seems to have almost nothing to do with elfutils. This is a core bug in Spack, in that comparison between Spec objects fails (with the error shown above) if we compare abstract specs and either we compare a spec with a name to an abstract spec, or we compare a spec with a namespace to one for which the namespace has not been defined yet. Expect a PR in the next ~20 minutes.

@mwkrentel @mvanwaveren 

Resolving patch:
diff --git a/lib/spack/spack/spec.py b/lib/spack/spack/spec.py
--- a/lib/spack/spack/spec.py
+++ b/lib/spack/spack/spec.py
@@ -3517,8 +3517,11 @@ def ne_dag(self, other, deptypes=True):
 
     def _cmp_node(self):
         """Comparison key for just *this node* and not its deps."""
-        return (self.name,
-                self.namespace,
+        # Name or namespace None will lead to invalid comparisons for abstract
+        # specs. Replace them with the empty string, which is not a valid spec
+        # name nor namespace so it will not create spurious equalities.
+        return (self.name or '',
+                self.namespace or '',
                 tuple(self.versions),
                 self.variants,
                 self.architecture,


Test patch:
diff --git a/lib/spack/spack/test/spec_syntax.py b/lib/spack/spack/test/spec_syntax.py
--- a/lib/spack/spack/test/spec_syntax.py
+++ b/lib/spack/spack/test/spec_syntax.py
@@ -2,7 +2,7 @@
 # Spack Project Developers. See the top-level COPYRIGHT file for details.
 #
 # SPDX-License-Identifier: (Apache-2.0 OR MIT)
-
+import itertools
 import os
 import pytest
 import shlex
@@ -806,3 +806,26 @@ def test_kv_with_spaces(self):
     ])
     def test_target_tokenization(self, expected_tokens, spec_string):
         self.check_lex(expected_tokens, spec_string)
+
+    @pytest.mark.regression('20310')
+    def test_compare_abstract_specs(self):
+        """Spec comparisons must be valid for abstract specs.
+
+        Check that the spec cmp_key appropriately handles comparing specs for
+        which some attributes are None in exactly one of two specs"""
+        # Add fields in order they appear in `Spec._cmp_node`
+        constraints = [
+            None,
+            'foo',
+            'foo.foo',
+            'foo.foo@foo',
+            'foo.foo@foo+foo',
+            'foo.foo@foo+foo arch=foo-foo-foo',
+            'foo.foo@foo+foo arch=foo-foo-foo %foo',
+            'foo.foo@foo+foo arch=foo-foo-foo %foo cflags=foo',
+        ]
+        specs = [Spec(s) for s in constraints]
+
+        for a, b in itertools.product(specs, repeat=2):
+            # Check that we can compare without raising an error
+            assert a <= b or b < a