Upgrade free-threading CI to run with pytest-freethreaded instead of pytest-xdist

There is a new kid on the block that should help us find out whether scikit-learn and its dependencies can be reliably considered free-threading compatible:

https://pypi.org/project/pytest-freethreaded/

Let's try to adopt it in scikit-learn.

Here is a possible plan:

- first run the tests locally a few times and see if they are tests (or a set of interacting tests) that cause a crash or a failure, open an issue for each of them, possibly upstream and then mark them as skipped under free-threading builds with a reference to the issue in the "reason" field;
- then upgrade our nightly free-threading scheduled CI run to use `pytest-freethreaded`.

Any comments @lesteve @jeremiedbb @ngoldbaum?

EDIT: anyone interested in getting hands on the first item can find this resource useful:

https://py-free-threading.github.io/

EDIT 2: there is also the [pytest-run-parallel](https://github.com/Quansight-Labs/pytest-run-parallel) plugin that can serve a similar purpose.

EDIT3: here is a TODO/plan for this problem:



- [x] resync and simplify #30041 to leverage https://github.com/Quansight-Labs/pytest-run-parallel/pull/19, https://github.com/Quansight-Labs/pytest-run-parallel/pull/33 and https://github.com/Quansight-Labs/pytest-run-parallel/pull/34
- [x] configure the `[free-threading]` CI to run with `pytest-run-parallel` #32023
- [x] enable the free-threading flag in Cython extension #31342
  - [x] investigate and fix crashes and test failures or open tracking issues and mark tests as `thread_unsafe` ;)
- [ ] benchmark the use of threading, in particular when we we expect nested parallelism between Python threads (e.g. in `GridSearchCV` with the "threading" backend of joblib) and BLAS or OpenMP native threading in the underlying estimators.
- [ ] communicate results on a blog post / pydata presentation / social media.

COMMENT:
```python
import pytest


@pytest.mark.parametrize("value", list("abcd"))
def test_simple(tmp_path, value):
    file_path = tmp_path / "file.txt"
    assert not file_path.exists()
    file_path.write_text(value)
    read_value = file_path.read_text()
    assert read_value == value
```

```
$ pip install pytest-xdist pytest-repeat
```

```
$ pytest -n 8 --count 2 -v test_tmp_path.py
========================================================================== test session starts ==========================================================================
platform darwin -- Python 3.13.0, pytest-8.3.3, pluggy-1.5.0 -- /Users/ogrisel/miniforge3/envs/py313/bin/python3.13
cachedir: .pytest_cache
rootdir: /private/tmp/testfolder
plugins: repeat-0.9.3, xdist-3.6.1
8 workers [8 items]     
scheduling tests via LoadScheduling

test_tmp_path.py::test_simple[c-1-2] 
test_tmp_path.py::test_simple[d-1-2] 
test_tmp_path.py::test_simple[a-2-2] 
[gw4] [ 12%] PASSED test_tmp_path.py::test_simple[c-1-2] 
test_tmp_path.py::test_simple[d-2-2] 
test_tmp_path.py::test_simple[b-2-2] 
test_tmp_path.py::test_simple[c-2-2] 
test_tmp_path.py::test_simple[b-1-2] 
test_tmp_path.py::test_simple[a-1-2] 
[gw0] [ 25%] PASSED test_tmp_path.py::test_simple[a-1-2] 
[gw3] [ 37%] PASSED test_tmp_path.py::test_simple[b-2-2] 
[gw5] [ 50%] PASSED test_tmp_path.py::test_simple[c-2-2] 
[gw1] [ 62%] PASSED test_tmp_path.py::test_simple[a-2-2] 
[gw7] [ 75%] PASSED test_tmp_path.py::test_simple[d-2-2] 
[gw6] [ 87%] PASSED test_tmp_path.py::test_simple[d-1-2] 
[gw2] [100%] PASSED test_tmp_path.py::test_simple[b-1-2] 

=========================================================================== 8 passed in 0.52s ===========================================================================
```

Labels: Build / CI, Needs Decision, module:test-suite, free-threading