Repository: Lightning-AI/torchmetrics

Issue:
packaging.metadata: `_tests` in `provides-extra` breaks pep-685 compliance
## 🐛 Bug

When building the `.whl` and installing it, I get the following error:
```
packaging.metadata.ExceptionGroup: ('invalid metadata', [InvalidMetadata("'_tests' is invalid for 'provides-extra'")])
```
I guess that [the choice of `_tests` as a key in the extra dependencies dict](https://github.com/Lightning-AI/torchmetrics/blob/fd2e332b66df1b484728efedad9d430c7efae990/setup.py#L171) is not supported by the python tooling.
I would suggest to rename it to `tests` or `domain_tests`.

For context, this happens when building the python package `torchmetrics` in the nixpkgs repo.

Full stacktrace:
```
Checking runtime dependencies for torchmetrics-1.3.0-py3-none-any.whl
Traceback (most recent call last):
  File "/nix/store/lq6lxp60dvapb70bmidj7cbf7m274081-python-runtime-deps-check-hook.py", line 93, in <module>
    metadata = get_metadata(args.wheel)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/lq6lxp60dvapb70bmidj7cbf7m274081-python-runtime-deps-check-hook.py", line 58, in get_metadata
    metadata = Metadata.from_raw(raw)
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/y16vmwsy7c4s870pm6yapqwbm1i59apm-python3.11-packaging-23.2/lib/python3.11/site-packages/packaging/metadata.py", line 722, in from_raw
    raise ExceptionGroup("invalid metadata", exceptions)
packaging.metadata.ExceptionGroup: ('invalid metadata', [InvalidMetadata("'_tests' is invalid for 'provides-extra'")])
```


Discussion:
Hi! thanks for your contribution!, great first issue!
Specification for `provides-extra`: https://packaging.python.org/en/latest/specifications/core-metadata/#provides-extra-multiple-use
@GaetanLepage thank you for reporting your concerns. Could you please share the steps to reproduce this and why would you need it? Since we are using some links to source code for testing with packages without updates, this installing via extra won't be complete, and we encourage you to use the raw `.txt` files

Trying to replicate in #2306 but it seems to be passing
Well, this is not that easy to replicate.
Indeed, I faced this error within the nix build pipeline. I was able to understand what it was doing in the background:
- We first build the `.whl` package for `torchmetrics`
- Then, we are [reading the content of the produced package](https://github.com/NixOS/nixpkgs/blob/a382cd09c265d1951c3ea6c7a1de61d8318b1353/pkgs/development/interpreters/python/hooks/python-runtime-deps-check-hook.py#L36-L49) and [building a `packaging.Metadata` object out of it](https://github.com/NixOS/nixpkgs/blob/a382cd09c265d1951c3ea6c7a1de61d8318b1353/pkgs/development/interpreters/python/hooks/python-runtime-deps-check-hook.py#L58)

This is where it crashes. It fails because the actual metadata from `torchmetrics` (more precisely, the `provides-extra` entry) does not respect [PEP685](https://peps.python.org/pep-0685/).
I suggest to simply rename this `_tests` key to another name that satisfies the norm (see [PEP508](https://peps.python.org/pep-0508/)).
Some clarifications wrto scoping:

- Build generation works fine: tested both `python -m pip wheel git+https://github.com/Lightning-AI/torchmetrics` and `python -m build`
- Wheel installation e.g. via `pip install` works fine
- Metadata inspection using `packaging.metadata` misbehaves in some very specific ways.

To summarize @GaetanLepage's findings, the reproducer is:

```
❯ python -m build
❯ pip install ./dist/torchmetrics-1.4.0.dev0-py3-none-any.whl
❯ unzip dist/torchmetrics-1.4.0.dev0-py3-none-any.whl torchmetrics-1.4.0.dev0.dist-info/METADATA
❯ python
>>> import packaging.metadata
>>> from packaging.metadata import parse_email
>>> with open("torchmetrics-1.4.0.dev0.dist-info/METADATA", "r") as f: txt = f.read()
>>> packaging.metadata.Metadata.from_raw(parse_email(txt)[0])
...
    raise ExceptionGroup("invalid metadata", exceptions)
packaging.metadata.ExceptionGroup: ('invalid metadata', [InvalidMetadata("'_tests' is invalid for 'provides-extra'")])
```

Otherwise the nixpkgs internals are irrelevant here. I'd rename the issue and prefix it e.g. with "(packaging.metadata|pep-685) (compatibility|support|compliance): "
Thank you for the clarification @SomeoneSerge 
ok so unless you would play some advanced package analyses you would not face this issue, correct? I know that if you try to install it with `.[_tests]` you will not see it...
> ok so unless you would play some advanced package analyses you would not face this issue, correct

That is my (limited) understanding. This doesn't seem to break pip or pypa-build, but `packaging.metadata` is there for python tooling developers(?), so this impacts "tooling" which may or may not be a sizeable thing

I do notice however that installation of `[_tests]` from the wheel fails obscurely (whereas `[tests]` "successfully" installs nothing):

```
❯ pip install dist/torchmetrics-1.4.0.dev0-py3-none-any.whl'[_tests]'
ERROR: Exception:
Traceback (most recent call last):
  File "/home/ss/Unsorted/repro-torchmetrics/.venv/lib/python3.11/site-packages/pip/_vendor/packaging/requirements.py", line 102, in __init__
    req = REQUIREMENT.parseString(requirement_string)
...
pip._vendor.pyparsing.exceptions.ParseException: Expected string_end, found '['  (at char 11), (line:1, col:12)
...
    return Requirement(req_string)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ss/Unsorted/repro-torchmetrics/.venv/lib/python3.11/site-packages/pip/_vendor/packaging/requirements.py", line 104, in __init__
    raise InvalidRequirement(
pip._vendor.packaging.requirements.InvalidRequirement: Parse error at "'[_tests]'": Expected string_end
```

(Offtopic: `_tests/` isn't present in the `.whl` even though it's mentioned in the `METADATA`, idk if that's how it's meant to be used?)
> I do notice however that installation of `[_tests]` from the wheel fails obscurely (whereas `[tests]` "successfully" installs nothing)

yes that is something we were aware of at the time we made this change

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -190,7 +190,7 @@ def _prepare_extras(skip_pattern: str = "^_", skip_files: Tuple[str] = ("base.tx
     # create an 'all' keyword that install all possible dependencies
     extras_req["all"] = list(chain([pkgs for k, pkgs in extras_req.items() if k not in ("_test", "_tests")]))
     extras_req["dev"] = extras_req["all"] + extras_req["_tests"]
-    return extras_req
+    return {k: v for k, v in extras_req.items() if not k.startswith("_")}
 
 
 # https://packaging.python.org/discussions/install-requires-vs-requirements /


Test patch: