Repository: spack/spack

Issue:
py-transformers unable to be found
### Steps to reproduce

This is quite a simple error to reproduce, and has consistently been an issue for me for a while now with various versions of spack.
```console
$ spack spec py-transformers
```
or, to see this error in an environment:
```console
$ spack env create -d ./test-env
$ spacktivate -p ./test-env
$ spack add py-transformers
$ spack concretize -f
```


### Error message

```console
$ spack --debug --stacktrace spec py-transformers
```
<details> <summary>Full error output</summary>

```console
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:24.766764] Reading config file /Users/ruth521/Projects/spack/spack/etc/spack/defaults/config.yaml
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:24.794562] Reading config file /Users/ruth521/Projects/spack/spack/etc/spack/defaults/concretizer.yaml
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:24.802347] Reading config file /Users/ruth521/Projects/spack/test-env/spack.yaml
lib/spack/spack/environment/environment.py:173 ==> [2022-08-30-11:22:24.804814] Using environment '/Users/ruth521/Projects/spack/test-env'
lib/spack/spack/cmd/__init__.py:124 ==> [2022-08-30-11:22:24.813121] Imported spec from built-in commands
lib/spack/spack/cmd/__init__.py:124 ==> [2022-08-30-11:22:24.814080] Imported spec from built-in commands
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:24.848930] Reading config file /Users/ruth521/Projects/spack/spack/etc/spack/defaults/bootstrap.yaml
lib/spack/spack/environment/environment.py:197 ==> [2022-08-30-11:22:24.857615] Deactivated environment '/Users/ruth521/Projects/spack/test-env'
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:24.900861] Reading config file /Users/ruth521/Projects/spack/spack/etc/spack/defaults/config.yaml
lib/spack/spack/database.py:389 ==> [2022-08-30-11:22:24.919398] DATABASE LOCK TIMEOUT: 3s
lib/spack/spack/database.py:395 ==> [2022-08-30-11:22:24.919670] PACKAGE LOCK TIMEOUT: No timeout
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:24.920386] Reading config file /Users/ruth521/Projects/spack/spack/etc/spack/defaults/bootstrap.yaml
lib/spack/spack/database.py:389 ==> [2022-08-30-11:22:24.926917] DATABASE LOCK TIMEOUT: 3s
lib/spack/spack/database.py:395 ==> [2022-08-30-11:22:24.927213] PACKAGE LOCK TIMEOUT: No timeout
lib/spack/spack/bootstrap.py:626 ==> [2022-08-30-11:22:24.927477] [BOOTSTRAP CONFIG SCOPE] name=_builtin
lib/spack/spack/bootstrap.py:637 ==> [2022-08-30-11:22:24.928027] [BOOTSTRAP CONFIG SCOPE] name=defaults, path=/Users/ruth521/Projects/spack/spack/etc/spack/defaults
lib/spack/spack/bootstrap.py:638 ==> [2022-08-30-11:22:24.928253] [BOOTSTRAP CONFIG SCOPE] name=defaults/darwin, path=/Users/ruth521/Projects/spack/spack/etc/spack/defaults/darwin
lib/spack/spack/bootstrap.py:637 ==> [2022-08-30-11:22:24.928475] [BOOTSTRAP CONFIG SCOPE] name=bootstrap, path=/Users/ruth521/.spack/bootstrap/config
lib/spack/spack/bootstrap.py:638 ==> [2022-08-30-11:22:24.928687] [BOOTSTRAP CONFIG SCOPE] name=bootstrap/darwin, path=/Users/ruth521/.spack/bootstrap/config/darwin
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:24.933957] Reading config file /Users/ruth521/.spack/bootstrap/config/darwin/compilers.yaml
lib/spack/spack/bootstrap.py:763 ==> [2022-08-30-11:22:25.002845] [BOOTSTRAP ROOT SPEC] clingo-bootstrap@spack+python %apple-clang target=x86_64
lib/spack/spack/bootstrap.py:522 ==> [2022-08-30-11:22:25.003185] [BOOTSTRAP MODULE clingo] Try importing from Python
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:25.003926] Reading config file /Users/ruth521/Projects/spack/spack/etc/spack/defaults/bootstrap.yaml
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:25.342635] Reading config file /Users/ruth521/Projects/spack/spack/etc/spack/defaults/config.yaml
lib/spack/spack/bootstrap.py:107 ==> [2022-08-30-11:22:25.500305] [BOOTSTRAP MODULE clingo] The installed spec "clingo-bootstrap@spack+python %apple-clang target=x86_64 ^python@3.8 /p5on7i4hejl775ezndzfdkhvwra3hatn" provides the "clingo" Python module
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:25.501973] Reading config file /Users/ruth521/Projects/spack/spack/etc/spack/defaults/config.yaml
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:25.520729] Reading config file /Users/ruth521/Projects/spack/test-env/spack.yaml
lib/spack/spack/environment/environment.py:173 ==> [2022-08-30-11:22:25.523291] Using environment '/Users/ruth521/Projects/spack/test-env'
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:25.523640] Reading config file /Users/ruth521/Projects/spack/spack/etc/spack/defaults/concretizer.yaml
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:25.528254] Reading config file /Users/ruth521/Projects/spack/spack/etc/spack/defaults/repos.yaml
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:25.553097] Reading config file /Users/ruth521/Projects/spack/spack/etc/spack/defaults/mirrors.yaml
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:29.111862] Reading config file /Users/ruth521/.spack/darwin/compilers.yaml
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:29.132916] Reading config file /Users/ruth521/Projects/spack/spack/etc/spack/defaults/packages.yaml
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:29.163155] Reading config file /Users/ruth521/Projects/spack/spack/etc/spack/defaults/darwin/packages.yaml
lib/spack/spack/config.py:1008 ==> [2022-08-30-11:22:29.173765] Reading config file /Users/ruth521/.spack/packages.yaml
lib/spack/spack/main.py:964 ==> [2022-08-30-11:22:44.591636] SpackError: No versions available for package 'py-transformers'
lib/spack/spack/error.py:54 ==> [2022-08-30-11:22:44.596360] Error: No versions available for package 'py-transformers'
Traceback (most recent call last):
  File "/Users/ruth521/Projects/spack/spack/lib/spack/spack/cmd/__init__.py", line 227, in parse_specs
    spec.concretize(tests=tests)  # implies normalize
  File "/Users/ruth521/Projects/spack/spack/lib/spack/spack/spec.py", line 2924, in concretize
    self._new_concretize(tests)
  File "/Users/ruth521/Projects/spack/spack/lib/spack/spack/spec.py", line 2899, in _new_concretize
    result = solver.solve([self], tests=tests)
  File "/Users/ruth521/Projects/spack/spack/lib/spack/spack/solver/asp.py", line 2380, in solve
    result, _, _ = self.driver.solve(setup, specs, reuse=reusable_specs, output=output)
  File "/Users/ruth521/Projects/spack/spack/lib/spack/spack/solver/asp.py", line 706, in solve
    answers = builder.build_specs(tuples)
  File "/Users/ruth521/Projects/spack/spack/lib/spack/spack/solver/asp.py", line 2232, in build_specs
    action(*args)
  File "/Users/ruth521/Projects/spack/spack/lib/spack/spack/solver/asp.py", line 2066, in error
    raise UnsatisfiableSpecError(msg)
spack.solver.asp.UnsatisfiableSpecError: No versions available for package 'py-transformers'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ruth521/Projects/spack/spack/lib/spack/spack/main.py", line 961, in main
    return _main(argv)
  File "/Users/ruth521/Projects/spack/spack/lib/spack/spack/main.py", line 916, in _main
    return finish_parse_and_run(parser, cmd_name, env_format_error)
  File "/Users/ruth521/Projects/spack/spack/lib/spack/spack/main.py", line 944, in finish_parse_and_run
    return _invoke_command(command, parser, args, unknown)
  File "/Users/ruth521/Projects/spack/spack/lib/spack/spack/main.py", line 608, in _invoke_command
    return_val = command(parser, args)
  File "/Users/ruth521/Projects/spack/spack/lib/spack/spack/cmd/spec.py", line 104, in spec
    concretized_specs = spack.cmd.parse_specs(args.specs, concretize=True)
  File "/Users/ruth521/Projects/spack/spack/lib/spack/spack/cmd/__init__.py", line 241, in parse_specs
    raise spack.error.SpackError(msg)
spack.error.SpackError: No versions available for package 'py-transformers'
```
</details>

### Information on your system

I have seen this error on multiple systems:

On my local mac:
```console
$ spack debug report
* **Spack:** 0.19.0.dev0 (08c67302fc61e18c32be3ff5a0cf20c96db5e26e)
* **Python:** 3.8.5
* **Platform:** darwin-catalina-skylake
* **Concretizer:** clingo
```

On HPC clusters:

```console
$ spack debug report
* **Spack:** 0.19.0.dev0
* **Python:** 3.9.5
* **Platform:** linux-centos7-zen2
* **Concretizer:** clingo
```

### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have searched the issues of this repo and believe this is not a duplicate
- [X] I have run the failing commands in debug mode and reported the output


Discussion:
@adamjstewart is the maintainer of the relevant package, and replied in the slack when I asked about this issue in #general.

I get this behaviour with a fresh clone of spack, and it is painfully easy to re-produce for me.

I am able to run `spack edit py-transformers` just fine and see that the `package.py` exists, but for some reason spack cannot concretize the package itself.
A workaround that I have imagined would be to install the relevant stack without `py-transformers` using spack, and then to install the transformers library using `pip`.

This might work, but it would be 1000% easier to have this working within spack.
... aaaand all of a sudden I'm magically able to reproduce this. I don't think anything changed on my end, but I'm glad that you're not the only one experiencing this anymore.

@becker33 @alalazo is it possible that this error message is a red herring? The package clearly has versions, so I'm guessing it's more something like "no version can be concretized successfully".
Any progress on this? I would love to manage transformers through spack.
Concretizer error message is still useless...

Going to try to add a newer version and see if it's easier to concretize. What is almost certainly going on here is that two of the dependencies require versions that conflict with each other. The tricky part is figuring out which versions conflict.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/py-huggingface-hub/package.py b/var/spack/repos/builtin/packages/py-huggingface-hub/package.py
--- a/var/spack/repos/builtin/packages/py-huggingface-hub/package.py
+++ b/var/spack/repos/builtin/packages/py-huggingface-hub/package.py
@@ -14,13 +14,18 @@ class PyHuggingfaceHub(PythonPackage):
     homepage = "https://github.com/huggingface/huggingface_hub"
     pypi = "huggingface_hub/huggingface_hub-0.0.10.tar.gz"
 
+    version("0.10.1", sha256="5c188d5b16bec4b78449f8681f9975ff9d321c16046cc29bcf0d7e464ff29276")
     version("0.0.10", sha256="556765e4c7edd2d2c4c733809bae1069dca20e10ff043870ec40d53e498efae2")
     version("0.0.8", sha256="be5b9a7ed36437bb10a780d500154d426798ec16803ff3406f7a61107e4ebfc2")
 
+    depends_on("python@3.7:", when="@0.10:", type=("build", "run"))
     depends_on("python@3.6:", type=("build", "run"))
     depends_on("py-setuptools", type="build")
     depends_on("py-filelock", type=("build", "run"))
     depends_on("py-requests", type=("build", "run"))
     depends_on("py-tqdm", type=("build", "run"))
+    depends_on("py-pyyaml@5.1:", when="@0.10:", type=("build", "run"))
+    depends_on("py-typing-extensions@3.7.4.3:", when="@0.10:", type=("build", "run"))
     depends_on("py-typing-extensions", when="@0.0.10:", type=("build", "run"))
     depends_on("py-importlib-metadata", when="^python@:3.7", type=("build", "run"))
+    depends_on("py-packaging@20.9:", when="@0.10:", type=("build", "run"))
diff --git a/var/spack/repos/builtin/packages/py-tokenizers/package.py b/var/spack/repos/builtin/packages/py-tokenizers/package.py
--- a/var/spack/repos/builtin/packages/py-tokenizers/package.py
+++ b/var/spack/repos/builtin/packages/py-tokenizers/package.py
@@ -13,15 +13,15 @@ class PyTokenizers(PythonPackage):
     homepage = "https://github.com/huggingface/tokenizers"
     pypi = "tokenizers/tokenizers-0.6.0.tar.gz"
 
+    version("0.13.1", sha256="3333d1cee5c8f47c96362ea0abc1f81c77c9b92c6c3d11cbf1d01985f0d5cf1d")
     version("0.10.3", sha256="1a5d3b596c6d3a237e1ad7f46c472d467b0246be7fd1a364f12576eb8db8f7e6")
     version("0.6.0", sha256="1da11fbfb4f73be695bed0d655576097d09a137a16dceab2f66399716afaffac")
     version("0.5.2", sha256="b5a235f9c71d04d4925df6c4fa13b13f1d03f9b7ac302b89f8120790c4f742bc")
 
     depends_on("py-setuptools", type="build")
     depends_on("py-setuptools-rust", type="build")
-    depends_on("rust@nightly", type="build")
 
     # TODO: This package currently requires internet access to install.
-    # Also, a nightly or dev version of rust is required to build.
+    # Also, a nightly or dev version of rust is required to build older versions.
     # https://github.com/huggingface/tokenizers/issues/176
     # https://github.com/PyO3/pyo3/issues/5
diff --git a/var/spack/repos/builtin/packages/py-transformers/package.py b/var/spack/repos/builtin/packages/py-transformers/package.py
--- a/var/spack/repos/builtin/packages/py-transformers/package.py
+++ b/var/spack/repos/builtin/packages/py-transformers/package.py
@@ -16,24 +16,32 @@ class PyTransformers(PythonPackage):
 
     maintainers = ["adamjstewart"]
 
+    version("4.24.0", sha256="486f353a8e594002e48be0e2aba723d96eda839e63bfe274702a4b5eda85559b")
     version("4.6.1", sha256="83dbff763b7e7dc57cbef1a6b849655d4fcab6bffdd955c5e8bea12a4f76dc10")
     version("2.8.0", sha256="b9f29cdfd39c28f29e0806c321270dea337d6174a7aa60daf9625bf83dbb12ee")
 
+    depends_on("python@3.7:", when="@4.24:", type=("build", "run"))
     depends_on("python@3.6:", type=("build", "run"))
     depends_on("py-setuptools", type="build")
-
-    depends_on("py-dataclasses", when="^python@:3.6", type=("build", "run"))
-    depends_on("py-importlib-metadata", when="@4.6.1: ^python@:3.7", type=("build", "run"))
+    depends_on("py-importlib-metadata", when="@4.6: ^python@:3.7", type=("build", "run"))
     depends_on("py-filelock", type=("build", "run"))
-    depends_on("py-huggingface-hub@0.0.8", when="@4.6.1:", type=("build", "run"))
+    depends_on("py-huggingface-hub@0.10:0", when="@4.24:", type=("build", "run"))
+    depends_on("py-huggingface-hub@0.0.8", when="@4.6.1", type=("build", "run"))
+    depends_on("py-numpy@1.17:", when="@4.6:", type=("build", "run"))
     depends_on("py-numpy", type=("build", "run"))
-    depends_on("py-numpy@1.17:", when="@4.6.1:", type=("build", "run"))
-    depends_on("py-packaging", when="@4.6.1:", type=("build", "run"))
+    depends_on("py-packaging@20:", when="@4.24:", type=("build", "run"))
+    depends_on("py-packaging", when="@4.6.1", type=("build", "run"))
+    depends_on("py-pyyaml@5.1:", when="@4.24:", type=("build", "run"))
     depends_on("py-regex@:2019.12.16,2019.12.18:", type=("build", "run"))
     depends_on("py-requests", type=("build", "run"))
-    depends_on("py-sacremoses", type=("build", "run"))
+    depends_on("py-tokenizers@0.11.1:0.11.2,0.11.4:0.13", when="@4.24:", type=("build", "run"))
+    depends_on("py-tokenizers@0.10.1:0.10", when="@4.6.1", type=("build", "run"))
     depends_on("py-tokenizers@0.5.2", when="@2.8.0", type=("build", "run"))
-    depends_on("py-tokenizers@0.10.1:0.10", when="@4.6.1:", type=("build", "run"))
     depends_on("py-tqdm@4.27:", type=("build", "run"))
+
+    # Historical requirements
+    depends_on("py-dataclasses", when="@4.6.1 ^python@:3.6", type=("build", "run"))
+    depends_on("py-sacremoses", when="@:4.6", type=("build", "run"))
     depends_on("py-boto3", when="@2.8.0", type=("build", "run"))
+    depends_on("py-dataclasses", when="@2.8.0 ^python@:3.6", type=("build", "run"))
     depends_on("py-sentencepiece", when="@2.8.0", type=("build", "run"))


Test patch: