Repository: spack/spack

Issue:
spack external find: Error: Invalid character
<!-- Explain, in a clear and concise way, the command you ran and the result you were trying to achieve.
Example: "I ran `spack find` to list all the installed packages and ..." -->

I decided today to look at `spack` again after having some "fun" with `brew` getting its gcc all tangled in places I didn't want. So, I thought: "Let spack manage gcc!" After running `spack spec -I gcc` and seeing it wanted to install `perl` and the like, I decided to read the docs and learned about `spack external find`. Nice. But running it led to an error. 

Just to be doubly sure it wasn't my fault, I nuked my old `.spack` directory (since I didn't know about `spack external find` my spack knowledge is out of dat) as well as re-cloned the spack repo from scratch. Same issue. To wit:

### Steps to reproduce the issue

```console
❯ spack external find
==> Error: Invalid character
```

### Error Message

<!-- If Spack reported an error, provide the error message. If it did not report an error but the output appears incorrect, provide the incorrect output. If there was no error message and no output but the result is incorrect, describe how it does not match what you expect. -->
```console
❯ spack external find
==> Error: Invalid character
❯ spack --debug --stacktrace external find
lib/spack/spack/cmd/__init__.py:122 ==> [2020-10-12-09:10:44.321955] Imported external from built-in commands
lib/spack/spack/cmd/__init__.py:122 ==> [2020-10-12-09:10:44.323702] Imported external from built-in commands
lib/spack/spack/config.py:872 ==> [2020-10-12-09:10:44.327810] Reading config file /Users/fortran/spack/etc/spack/defaults/repos.yaml
lib/spack/spack/util/executable.py:159 ==> [2020-10-12-09:10:58.174477] '/bin/bash' '--version'
lib/spack/spack/util/executable.py:159 ==> [2020-10-12-09:10:58.185217] '/usr/local/bin/bash' '--version'
lib/spack/spack/util/executable.py:159 ==> [2020-10-12-09:10:58.193121] '/usr/bin/bison' '--version'
lib/spack/spack/util/executable.py:159 ==> [2020-10-12-09:10:58.211693] '/usr/local/bin/cmake' '--version'
lib/spack/spack/util/executable.py:159 ==> [2020-10-12-09:10:58.225630] '/usr/bin/find' '--version'
lib/spack/spack/package.py:182 ==> [2020-10-12-09:10:58.232383] An error occurred when trying to detect the version of "/usr/bin/find" [Command exited with status 1:
    '/usr/bin/find' '--version'
/usr/bin/find: illegal option -- -
usage: find [-H | -L | -P] [-EXdsx] [-f path] path ... [expression]
       find [-H | -L | -P] [-EXdsx] -f path [path ...] [expression]
]
lib/spack/spack/cmd/external.py:282 ==> [2020-10-12-09:10:58.233084] The following executables in /usr/bin were decidedly not part of the package findutils: /usr/bin/find
lib/spack/spack/util/executable.py:159 ==> [2020-10-12-09:10:58.234410] '/usr/local/bin/fish' '--version'
lib/spack/spack/package.py:182 ==> [2020-10-12-09:10:58.241665] An error occurred when trying to detect the version of "/usr/local/bin/fish" [name 're' is not defined]
lib/spack/spack/cmd/external.py:282 ==> [2020-10-12-09:10:58.242005] The following executables in /usr/local/bin were decidedly not part of the package fish: /usr/local/bin/fish
lib/spack/spack/util/executable.py:159 ==> [2020-10-12-09:10:58.243394] '/usr/bin/flex' '--version'
lib/spack/spack/util/executable.py:159 ==> [2020-10-12-09:10:58.261605] '/usr/local/bin/fzf' '--version'
lib/spack/spack/main.py:765 ==> [2020-10-12-09:10:58.271625] LexError: Invalid character
lib/spack/spack/error.py:55 ==> [2020-10-12-09:10:58.272691] Error: Invalid character
Traceback (most recent call last):
  File "/Users/fortran/spack/lib/spack/spack/main.py", line 762, in main
    return _invoke_command(command, parser, args, unknown)
  File "/Users/fortran/spack/lib/spack/spack/main.py", line 490, in _invoke_command
    return_val = command(parser, args)
  File "/Users/fortran/spack/lib/spack/spack/cmd/external.py", line 337, in external
    action[args.external_command](args)
  File "/Users/fortran/spack/lib/spack/spack/cmd/external.py", line 156, in external_find
    pkg_to_entries = _get_external_packages(packages_to_check)
  File "/Users/fortran/spack/lib/spack/spack/cmd/external.py", line 279, in _get_external_packages
    pkg.determine_spec_details(prefix, exes_in_prefix))
  File "/Users/fortran/spack/lib/spack/spack/package.py", line 205, in determine_spec_details
    spec = spack.spec.Spec(
  File "/Users/fortran/spack/lib/spack/spack/spec.py", line 1019, in __init__
    spec_list = SpecParser(self).parse(spec_like)
  File "/Users/fortran/spack/lib/spack/spack/parse.py", line 151, in parse
    self.setup(text)
  File "/Users/fortran/spack/lib/spack/spack/parse.py", line 148, in setup
    self.push_tokens(self.lexer.lex(text))
  File "/Users/fortran/spack/lib/spack/spack/parse.py", line 86, in lex
    tokens = self.lex_word(word)
  File "/Users/fortran/spack/lib/spack/spack/parse.py", line 79, in lex_word
    raise LexError("Invalid character", word, word.index(remainder))
spack.parse.LexError: Invalid character
```

Staring at this, I ran the last command and:
```console
❯ /usr/local/bin/fzf --version
0.23.1 (brew)
```

### Information on your system

<!-- Please include the output of `spack debug report` -->
```console
❯ spack debug report
* **Spack:** 0.15.4-1420-db16f3e0d
* **Python:** 3.8.6
* **Platform:** darwin-catalina-skylake
```

<!-- If you have any relevant configuration detail (custom `packages.yaml` or `modules.yaml`, etc.) you can add that here as well. -->

I used to have some, but I nuked them!

### Additional information

<!-- These boxes can be checked by replacing [ ] with [x] or by clicking them after submitting the issue. -->
- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have searched the issues of this repo and believe this is not a duplicate
- [X] I have run the failing commands in debug mode and reported the output

<!-- We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively!

If you want to ask a question about the tool (how to use it, what it can currently do, etc.), try the `#general` channel on our Slack first. We have a welcoming community and chances are you'll get your reply faster and without opening an issue.

Other than that, thanks for taking the time to contribute to Spack! -->



Discussion:
I'm seeing the same error on a basically stock ubuntu 19.10:

```
~/spack develop* scogland1@chimera 10s
1 ❯ ./bin/spack debug report
* **Spack:** 0.15.4-1562-a51702e3a
* **Python:** 3.7.5
* **Platform:** linux-ubuntu19.10-zen
```

I'm guessing this is because FZF is printing the git hash in perens after the version.  May take a deeper look later, question is whether to fix this in the version parser or in the fzf version method.  I'm leaning toward parser.
I think the FZF method needs fixing. Currently it doesn't even try to match a regex, it just returns whatever `--version` returns. Instead of fixing the parser to allow "`0.23.1 (brew)`" as a valid version I think we should probably intercept this kind of failure, write it in the debug log and proceed.

Resolving patch:
diff --git a/lib/spack/spack/package.py b/lib/spack/spack/package.py
--- a/lib/spack/spack/package.py
+++ b/lib/spack/spack/package.py
@@ -202,14 +202,19 @@ def determine_spec_details(cls, prefix, exes_in_prefix):
                         external_modules = extra_attributes.pop(
                             'modules', None
                         )
-                        spec = spack.spec.Spec(
-                            spec_str,
-                            external_path=external_path,
-                            external_modules=external_modules
-                        )
-                        specs.append(spack.spec.Spec.from_detection(
-                            spec, extra_attributes=extra_attributes
-                        ))
+                        try:
+                            spec = spack.spec.Spec(
+                                spec_str,
+                                external_path=external_path,
+                                external_modules=external_modules
+                            )
+                        except Exception as e:
+                            msg = 'Parsing failed [spec_str="{0}", error={1}]'
+                            tty.debug(msg.format(spec_str, str(e)))
+                        else:
+                            specs.append(spack.spec.Spec.from_detection(
+                                spec, extra_attributes=extra_attributes
+                            ))
 
                 return sorted(specs)
 
diff --git a/var/spack/repos/builtin/packages/fzf/package.py b/var/spack/repos/builtin/packages/fzf/package.py
--- a/var/spack/repos/builtin/packages/fzf/package.py
+++ b/var/spack/repos/builtin/packages/fzf/package.py
@@ -2,12 +2,10 @@
 # Spack Project Developers. See the top-level COPYRIGHT file for details.
 #
 # SPDX-License-Identifier: (Apache-2.0 OR MIT)
-
 import os
+import re
 import shutil
 
-from spack import *
-
 
 class Fzf(MakefilePackage):
     """fzf is a general-purpose command-line fuzzy finder."""
@@ -37,7 +35,9 @@ class Fzf(MakefilePackage):
 
     @classmethod
     def determine_version(cls, exe):
-        return Executable(exe)('--version', output=str, error=str).rstrip()
+        candidate = Executable(exe)('--version', output=str, error=str)
+        match = re.match(r'(^[\d.]+)', candidate)
+        return match.group(1) if match else None
 
     @when('@:0.17.5')
     def patch(self):


Test patch: