Repository: spack/spack

Issue:
Package specs fail for yaml-cpp package when using namespaces
I've created an external repo to override native packages in spack.
I can use the namespace to properly reference most packages, but yaml-cpp fails when I add the compiler in the spec.


### Steps to reproduce the issue
For example,
`spack find builtin.yaml-cpp`

returns the proper references, but

`spack find builtin.yaml-cpp%gcc@7.2.0`

returns

`==> Error: Spec filename must end in .yaml: 'builtin.yaml-cpp%gcc@7.2.0'`

The following also return the expected results:

`spack find builitin.nalu-wind%gcc@7.2.0`
`spack find builtin.trilinos%gcc@7.2.0`

So it appears to be an issue with yaml-cpp (as far as I know). Guessing it is because there is a "*.yaml" in the spec for this corner case.

Debug output:

[myconsole]$ spack -d find builtin.yaml-cpp%gcc%7.2.0
==> [2020-01-29-14:55:12.384485] Imported find from built-in commands
==> [2020-01-29-14:55:12.386274] Imported find from built-in commands
==> [2020-01-29-14:55:12.387341] SpackError: Spec filename must end in .yaml: 'builtin.yaml-cpp%gcc%7.2.0'
==> [2020-01-29-14:55:12.387410] Error: Spec filename must end in .yaml: 'builtin.yaml-cpp%gcc%7.2.0'
Traceback (most recent call last):
  File "/projects/wind/snl-ceerws/spack/lib/spack/spack/main.py", line 711, in main
    return _invoke_command(command, parser, args, unknown)
  File "/projects/wind/snl-ceerws/spack/lib/spack/spack/main.py", line 456, in _invoke_command
    return_val = command(parser, args)
  File "/projects/wind/snl-ceerws/spack/lib/spack/spack/cmd/find.py", line 197, in find
    results = args.specs(**q_args)
  File "/projects/wind/snl-ceerws/spack/lib/spack/spack/cmd/common/arguments.py", line 69, in _specs
    qspecs = spack.cmd.parse_specs(self.values)
  File "/projects/wind/snl-ceerws/spack/lib/spack/spack/cmd/__init__.py", line 161, in parse_specs
    raise spack.error.SpackError(msg)
spack.error.SpackError: Spec filename must end in .yaml: 'builtin.yaml-cpp%gcc%7.2.0'

### Information on your system

RHEL6, custom package in repo that just allows me to access additional sha's for yaml-cpp, but inherits from yaml-cpp in builtin.  I don't think my package configuration has anything to do with this.



Discussion:
I can't dive into this now but I *believe* this is related to https://github.com/spack/spack/pull/12561.
@tgamblin finally got some time to look at this.  looks like the issue is the [regex for a fully qualified expression](https://github.com/spack/spack/blob/76abf358138927be3ebeea2bead9fd1ccd087f41/lib/spack/spack/spec.py#L3927) doesn't cover this case when called from [here]( https://github.com/spack/spack/blob/76abf358138927be3ebeea2bead9fd1ccd087f41/lib/spack/spack/spec.py#L4100).  In general this regex seems really insufficient for this test because the spec can get way more complicated than that i.e. `builtin.yaml-cpp@0.62%gcc@7.2.0 build_type=Release ^boost build_type=RelWithDebInfo arch=dummy-rehl7-x86`

Trying to think of a solution, but fancy regex is not a strong point of mine. Should we run it through the lexer first?

Resolving patch:
diff --git a/lib/spack/spack/spec.py b/lib/spack/spack/spec.py
--- a/lib/spack/spack/spec.py
+++ b/lib/spack/spack/spec.py
@@ -3945,7 +3945,8 @@ def __init__(self):
 
             # Filenames match before identifiers, so no initial filename
             # component is parsed as a spec (e.g., in subdir/spec.yaml)
-            (r'[/\w.-]+\.yaml[^\b]*', lambda scanner, v: self.token(FILE, v)),
+            (r'[/\w.-]*/[/\w/-]+\.yaml[^\b]*',
+             lambda scanner, v: self.token(FILE, v)),
 
             # Hash match after filename. No valid filename can be a hash
             # (files end w/.yaml), but a hash can match a filename prefix.
@@ -4096,11 +4097,6 @@ def spec_from_file(self):
         """
         path = self.token.value
 
-        # don't treat builtin.yaml, builtin.yaml-cpp, etc. as filenames
-        if re.match(spec_id_re + '$', path):
-            self.push_tokens([spack.parse.Token(ID, self.token.value)])
-            return None
-
         # Special case where someone omits a space after a filename. Consider:
         #
         #     libdwarf^/some/path/to/libelf.yamllibdwarf ^../../libelf.yaml
@@ -4112,7 +4108,6 @@ def spec_from_file(self):
             raise SpecFilenameError(
                 "Spec filename must end in .yaml: '{0}'".format(path))
 
-        # if we get here, we're *finally* interpreting path as a filename
         if not os.path.exists(path):
             raise NoSuchSpecFileError("No such spec file: '{0}'".format(path))
 


Test patch:
diff --git a/lib/spack/spack/test/spec_syntax.py b/lib/spack/spack/test/spec_syntax.py
--- a/lib/spack/spack/test/spec_syntax.py
+++ b/lib/spack/spack/test/spec_syntax.py
@@ -188,6 +188,32 @@ def test_full_specs(self):
             " ^_openmpi@1.2:1.4,1.6%intel@12.1~qt_4 debug=2"
             " ^stackwalker@8.1_1e arch=test-redhat6-x86")
 
+    def test_yaml_specs(self):
+        self.check_parse(
+            "yaml-cpp@0.1.8%intel@12.1"
+            " ^boost@3.1.4")
+        tempspec = r"builtin.yaml-cpp%gcc"
+        self.check_parse(
+            tempspec.strip("builtin."),
+            spec=tempspec)
+        tempspec = r"testrepo.yaml-cpp%gcc"
+        self.check_parse(
+            tempspec.strip("testrepo."),
+            spec=tempspec)
+        tempspec = r"builtin.yaml-cpp@0.1.8%gcc"
+        self.check_parse(
+            tempspec.strip("builtin."),
+            spec=tempspec)
+        tempspec = r"builtin.yaml-cpp@0.1.8%gcc@7.2.0"
+        self.check_parse(
+            tempspec.strip("builtin."),
+            spec=tempspec)
+        tempspec = r"builtin.yaml-cpp@0.1.8%gcc@7.2.0" \
+            r" ^boost@3.1.4"
+        self.check_parse(
+            tempspec.strip("builtin."),
+            spec=tempspec)
+
     def test_canonicalize(self):
         self.check_parse(
             "mvapich_foo"