Pytest -Werror fails with versions <9.0 due to faulthandler

When trying to turn warnings into errors with `pytest -Werror`, there is an error with `faulthandler` (`faulthandler` was introduced in #32776) and the tests don't run.

Updating pytest to >=9.0 fixes the problem.

Here's the traceback for when I used pytest==8.3.1:

```bs
➜  scikit-learn git:(TargetEncoder_groups) ✗ pytest sklearn/tests/test_metaestimators_metadata_routing.py -Werror
======================== test session starts ========================
platform linux -- Python 3.12.2, pytest-8.3.1, pluggy-1.5.0
rootdir: /home/stefanie/code/scikit-learn_dev/scikit-learn
configfile: pyproject.toml
plugins: xdist-3.6.1, anyio-4.9.0, cov-5.0.0
collected 271 items                                                                                                                      
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/stefanie/.pyenv/versions/scikit-learn_dev/lib/python3.12/site-packages/_pytest/main.py", line 283, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/stefanie/.pyenv/versions/scikit-learn_dev/lib/python3.12/site-packages/_pytest/main.py", line 336, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/home/stefanie/.pyenv/versions/scikit-learn_dev/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/stefanie/.pyenv/versions/scikit-learn_dev/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/stefanie/.pyenv/versions/scikit-learn_dev/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/home/stefanie/.pyenv/versions/scikit-learn_dev/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/stefanie/.pyenv/versions/scikit-learn_dev/lib/python3.12/site-packages/_pytest/logging.py", line 792, in pytest_collection
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/home/stefanie/.pyenv/versions/scikit-learn_dev/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/stefanie/.pyenv/versions/scikit-learn_dev/lib/python3.12/site-packages/_pytest/warnings.py", line 121, in pytest_collection
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/home/stefanie/.pyenv/versions/scikit-learn_dev/lib/python3.12/site-packages/pluggy/_callers.py", line 124, in _multicall
INTERNALERROR>     teardown.send(result)  # type: ignore[union-attr]
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/stefanie/.pyenv/versions/scikit-learn_dev/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1415, in pytest_collection
INTERNALERROR>     self._validate_config_options()
INTERNALERROR>   File "/home/stefanie/.pyenv/versions/scikit-learn_dev/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1437, in _validate_config_options
INTERNALERROR>     self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")
INTERNALERROR>   File "/home/stefanie/.pyenv/versions/scikit-learn_dev/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1476, in _warn_or_fail_if_strict
INTERNALERROR>     self.issue_config_time_warning(PytestConfigWarning(message), stacklevel=3)
INTERNALERROR>   File "/home/stefanie/.pyenv/versions/scikit-learn_dev/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1528, in issue_config_time_warning
INTERNALERROR>     warnings.warn(warning, stacklevel=stacklevel)
INTERNALERROR> pytest.PytestConfigWarning: Unknown config option: faulthandler_exit_on_timeout

========================= no tests ran in 0.47s =========================
```

In `pyproject.toml`, we mention `faulthandler` to only work with `pytest` newer than 9.0.

We came across this with @AnneBeyer and wondered whether that's a good enough reason to update/bump the min dev depenency to pytest>9.0 or if there is another better way to deal with that?

Maybe @ogrisel or @lesteve , you know more, since you were involved in PR #32776 adding `faulthandler`?

COMMENT:
Ah I tried to see whether it had any implications when using pytest<9 but  I did not think of a potential warning that could be turned into an error by `pytest -Werror`.

At this point, I think upgrading pytest for anyone bothered by `pytest -Werror` is the short-term option. I don't know about the use case, but you can very often use `pytest -Werror:some message:YourWarningType` depending on which warning you care about.

Longer term we could indeed say use `pytest >= 9` if you want to contribute to scikit-learn, but I would wait a bit because it's very recent and IIRC there were a few things like unittest.SkipTest behaviour that were not really decided on (they broke something in pytest 9 and reverted it for 9.0.1 but it was not clear about the long-term change)...

Labels: Needs Triage