BUG: pytest error when loading conftest (seemingly platform-specific)

### Describe the bug

I'm seeing errors on my Windows machine when running `pytest` (does not work with only `pytest`, and does not work for directories that has `conftest.py`). This seems to be a platform-specific problem, since CI is not complaining. I investigated a bit and found https://github.com/pytest-dev/pytest/issues/9765, but it doesn't look like `pytest` is planning to fix it, at least for now. I tried downgrading to `pytest==7.0.1` and everything worked smoothly, but in https://github.com/scikit-learn/scikit-learn/pull/26373 the minimum version of `pytest` has already been `7.1.2` for scikit-learn (due to some CI errors for `pytest==5.x.x`), so scikit-learn is raising error:

https://github.com/scikit-learn/scikit-learn/blob/5c4288dba42cb67d954cb56c2cebfbf25c05ef89/sklearn/conftest.py#L30-L34

I'm wondering if it is possible to pin `pytest==7.0.1` or at least relax the minimum requirement a bit to `PYTEST_MIN_VERSION = "7.0.1"`? Or are there any other suggestions how I may resolve this issue? @glemaitre who bumped the minimum version of `pytest` to 7.1.2. Truly sorry for the inconvenience caused by my annoying Windows machine.

### Steps/Code to Reproduce

```bash
pytest
```

or

```bash
pytest sklearn/utils/tests
```

Running `pytest` on a single file works correctly.

### Expected Results

No error is thrown.

### Actual Results

For the first example,

```pytb
❯ pytest
Traceback (most recent call last):
  File "D:\Downloads\mambaforge\envs\sklearn-env\Scripts\pytest-script.py", line 9, in <module>
    sys.exit(console_main())
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 192, in console_main
    code = main()
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 150, in main
    config = _prepareconfig(args, plugins)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 331, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_callers.py", line 130, in _multicall
    teardown[0].send(outcome)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\helpconfig.py", line 104, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_result.py", line 114, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 1075, in pytest_cmdline_parse
    self.parse(args)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 1425, in parse
    self._preparse(args, addopts=addopts)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 1327, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_callers.py", line 152, in _multicall
    return outcome.get_result()
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_result.py", line 114, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 1153, in pytest_load_initial_conftests
    self.pluginmanager._set_initial_conftests(
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 563, in _set_initial_conftests
    self._try_load_conftest(anchor, importmode, rootpath)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 585, in _try_load_conftest
    self._getconftestmodules(x, importmode, rootpath)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 609, in _getconftestmodules
    mod = self._importconftest(conftestpath, importmode, rootpath)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 654, in _importconftest
    assert mod not in mods
AssertionError
```

For the second example,

```pytb
❯ pytest sklearn/utils/tests
================================================================== test session starts ===================================================================
platform win32 -- Python 3.9.18, pytest-7.4.3, pluggy-1.3.0
rootdir: D:\ossd\scikit-learn-yxiao
configfile: setup.cfg
plugins: cov-4.1.0
collected 1957 items / 1 error

========================================================================= ERRORS =========================================================================
____________________________________________________ ERROR collecting sklearn/utils/tests/conftest.py ____________________________________________________
D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\runner.py:341: in from_call
    result: Optional[TResult] = func()
D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\runner.py:372: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\doctest.py:560: in collect
    module = self.config.pluginmanager._importconftest(
D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py:654: in _importconftest
    assert mod not in mods
E   AssertionError
================================================================ short test summary info =================================================================
ERROR sklearn/utils/tests/conftest.py - AssertionError
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================== 1 warning, 1 error in 1.43s ===============================================================
```

### Versions

```shell
❯ python -c "import sklearn; sklearn.show_versions()"

System:
    python: 3.9.18 | packaged by conda-forge | (main, Aug 30 2023, 03:40:31) [MSC v.1929 64 bit (AMD64)]
executable: D:\Downloads\mambaforge\envs\sklearn-env\python.exe
   machine: Windows-10-10.0.19045-SP0

Python dependencies:
      sklearn: 1.4.dev0
          pip: 23.2.1
   setuptools: 68.2.2
        numpy: 1.26.0
        scipy: 1.11.2
       Cython: 3.0.2
       pandas: 2.1.1
   matplotlib: 3.8.0
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: mkl
    num_threads: 6
         prefix: libblas
       filepath: D:\Downloads\mambaforge\envs\sklearn-env\Library\bin\libblas.dll
        version: 2022.1-Product
threading_layer: intel

       user_api: openmp
   internal_api: openmp
    num_threads: 12
         prefix: vcomp
       filepath: D:\Downloads\mambaforge\envs\sklearn-env\vcomp140.dll
        version: None
```

COMMENT:
@lesteve This is how I modified the `pytest` code:

<details>
<summary>Code</summary>

<p></p>

```python
    def _importconftest(
        self, conftestpath: Path, importmode: Union[str, ImportMode], rootpath: Path
    ) -> types.ModuleType:
        print()
        print(f"_importconftest({conftestpath=}, {importmode=}, {rootpath=})")
        existing = self.get_plugin(str(conftestpath))
        if existing is not None:
            return cast(types.ModuleType, existing)

        pkgpath = resolve_package_path(conftestpath)
        if pkgpath is None:
            _ensure_removed_sysmodule(conftestpath.stem)

        try:
            mod = import_path(conftestpath, mode=importmode, root=rootpath)
        except Exception as e:
            assert e.__traceback__ is not None
            exc_info = (type(e), e, e.__traceback__)
            raise ConftestImportFailure(conftestpath, exc_info) from e

        self._check_non_top_pytest_plugins(mod, conftestpath)

        self._conftest_plugins.add(mod)
        dirpath = conftestpath.parent
        print(f"   {mod=}")
        print(f"   {dirpath=}, {self._dirpath2confmods=}")
        if dirpath in self._dirpath2confmods:
            print(f"   Iterating over self._dirpath2confmods")
            for path, mods in self._dirpath2confmods.items():
                print(f"        {path=}, {mods=}")
                if dirpath in path.parents or path == dirpath:
                    assert mod not in mods, f"{mod} should not be in {mods}"
                    mods.append(mod)
        self.trace(f"loading conftestmodule {mod!r}")
        self.consider_conftest(mod)
        return mod
```

</details>

This is what happens on the main branch when running `pytest` on the main branch:

<details>
<summary>Printout</summary>
<p></p>

```
_importconftest(conftestpath=WindowsPath('D:/ossd/scikit-learn-yxiao/conftest.py'), importmode='prepend', rootpath=WindowsPath('D:/ossd/scikit-learn-yxiao'))
   mod=<module 'conftest' from 'D:\\ossd\\scikit-learn-yxiao\\conftest.py'>
   dirpath=WindowsPath('D:/ossd/scikit-learn-yxiao'), self._dirpath2confmods={}

_importconftest(conftestpath=WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn/conftest.py'), importmode='prepend', rootpath=WindowsPath('D:/ossd/scikit-learn-yxiao'))
   mod=<module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'>
   dirpath=WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn'), self._dirpath2confmods={}

_importconftest(conftestpath=WindowsPath('D:/ossd/scikit-learn-yxiao/conftest.py'), importmode='prepend', rootpath=WindowsPath('D:/ossd/scikit-learn-yxiao'))

_importconftest(conftestpath=WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn/conftest.py'), importmode='prepend', rootpath=WindowsPath('D:/ossd/scikit-learn-yxiao'))
   mod=<module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'>
   dirpath=WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn'), self._dirpath2confmods={WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn'): [<module 'conftest' from 'D:\\ossd\\scikit-learn-yxiao\\conftest.py'>, <module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'>]}
   Iterating over self._dirpath2confmods
        path=WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn'), mods=[<module 'conftest' from 'D:\\ossd\\scikit-learn-yxiao\\conftest.py'>, <module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'>]
```

</details>

<details>
<summary>Traceback</summary>
<p></p>

```pytb
Traceback (most recent call last):
  File "D:\Downloads\mambaforge\envs\sklearn-env\Scripts\pytest-script.py", line 9, in <module>
    sys.exit(console_main())
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 192, in console_main
    code = main()
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 150, in main
    config = _prepareconfig(args, plugins)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 331, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_callers.py", line 130, in _multicall
    teardown[0].send(outcome)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\helpconfig.py", line 104, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_result.py", line 114, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 1081, in pytest_cmdline_parse
    self.parse(args)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 1431, in parse
    self._preparse(args, addopts=addopts)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 1333, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_callers.py", line 152, in _multicall
    return outcome.get_result()
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_result.py", line 114, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 1159, in pytest_load_initial_conftests
    self.pluginmanager._set_initial_conftests(
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 563, in _set_initial_conftests
    self._try_load_conftest(anchor, importmode, rootpath)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 585, in _try_load_conftest
    self._getconftestmodules(x, importmode, rootpath)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 609, in _getconftestmodules
    mod = self._importconftest(conftestpath, importmode, rootpath)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 660, in _importconftest
    assert mod not in mods, f"{mod} should not be in {mods}"
AssertionError: <module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'> should not be in [<module 'conftest' from 'D:\\ossd\\scikit-learn-yxiao\\conftest.py'>, <module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'>]
```

</details>

This is what happens when running `pytest sklearn` on the main branch:

<details>
<summary>Printout</summary>
<p></p>

```
_importconftest(conftestpath=WindowsPath('D:/ossd/scikit-learn-yxiao/conftest.py'), importmode='prepend', rootpath=WindowsPath('D:/ossd/scikit-learn-yxiao'))
   mod=<module 'conftest' from 'D:\\ossd\\scikit-learn-yxiao\\conftest.py'>
   dirpath=WindowsPath('D:/ossd/scikit-learn-yxiao'), self._dirpath2confmods={}

_importconftest(conftestpath=WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn/conftest.py'), importmode='prepend', rootpath=WindowsPath('D:/ossd/scikit-learn-yxiao'))
   mod=<module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'>
   dirpath=WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn'), self._dirpath2confmods={}

_importconftest(conftestpath=WindowsPath('D:/ossd/scikit-learn-yxiao/conftest.py'), importmode='prepend', rootpath=WindowsPath('D:/ossd/scikit-learn-yxiao'))

_importconftest(conftestpath=WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn/conftest.py'), importmode='prepend', rootpath=WindowsPath('D:/ossd/scikit-learn-yxiao'))
   mod=<module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'>
   dirpath=WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn'), self._dirpath2confmods={WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn'): [<module 'conftest' from 'D:\\ossd\\scikit-learn-yxiao\\conftest.py'>, <module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'>]}
   Iterating over self._dirpath2confmods
        path=WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn'), mods=[<module 'conftest' from 'D:\\ossd\\scikit-learn-yxiao\\conftest.py'>, <module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'>]
```

</details>

<details>
<summary>Traceback</summary>
<p></p>

```pybt
Traceback (most recent call last):
  File "D:\Downloads\mambaforge\envs\sklearn-env\Scripts\pytest-script.py", line 9, in <module>
    sys.exit(console_main())
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 192, in console_main
    code = main()
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 150, in main
    config = _prepareconfig(args, plugins)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 331, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_callers.py", line 130, in _multicall
    teardown[0].send(outcome)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\helpconfig.py", line 104, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_result.py", line 114, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 1081, in pytest_cmdline_parse
    self.parse(args)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 1431, in parse
    self._preparse(args, addopts=addopts)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 1333, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_callers.py", line 152, in _multicall
    return outcome.get_result()
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_result.py", line 114, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\pluggy\_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 1159, in pytest_load_initial_conftests
    self.pluginmanager._set_initial_conftests(
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 563, in _set_initial_conftests
    self._try_load_conftest(anchor, importmode, rootpath)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 585, in _try_load_conftest
    self._getconftestmodules(x, importmode, rootpath)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 609, in _getconftestmodules
    mod = self._importconftest(conftestpath, importmode, rootpath)
  File "D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py", line 660, in _importconftest
    assert mod not in mods, f"{mod} should not be in {mods}"
AssertionError: <module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'> should not be in [<module 'conftest' from 'D:\\ossd\\scikit-learn-yxiao\\conftest.py'>, <module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'>]
```

</details>

This is what happens when running `pytest sklearn/utils/tests` on the main branch (the above two examples directly raise, while this example captures the error when collecting tests):

<details>
<summary>pytest printout</summary>
<p></p>

```pybt
_importconftest(conftestpath=WindowsPath('D:/ossd/scikit-learn-yxiao/conftest.py'), importmode='prepend', rootpath=WindowsPath('D:/ossd/scikit-learn-yxiao'))
================================================================== test session starts ===================================================================
platform win32 -- Python 3.9.18, pytest-7.4.3, pluggy-1.3.0
rootdir: D:\ossd\scikit-learn-yxiao
configfile: setup.cfg
plugins: cov-4.1.0
collected 1972 items / 1 error

========================================================================= ERRORS =========================================================================
____________________________________________________ ERROR collecting sklearn/utils/tests/conftest.py ____________________________________________________
D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\runner.py:341: in from_call
    result: Optional[TResult] = func()
D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\runner.py:372: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\doctest.py:560: in collect
    module = self.config.pluginmanager._importconftest(
D:\Downloads\mambaforge\envs\sklearn-env\lib\site-packages\_pytest\config\__init__.py:660: in _importconftest
    assert mod not in mods, f"{mod} should not be in {mods}"
E   AssertionError: <module 'sklearn.utils.tests.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\utils\\tests\\conftest.py'> should not be in [<module 'conftest' from 'D:\\ossd\\scikit-learn-yxiao\\conftest.py'>, <module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'>, <module 'sklearn.utils.tests.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\utils\\tests\\conftest.py'>]
-------------------------------------------------------------------- Captured stdout ---------------------------------------------------------------------

_importconftest(conftestpath=WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn/utils/tests/conftest.py'), importmode='prepend', rootpath=WindowsPath('D:/ossd/scikit-learn-yxiao'))
   mod=<module 'sklearn.utils.tests.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\utils\\tests\\conftest.py'>
   dirpath=WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn/utils/tests'), self._dirpath2confmods={WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn/utils/tests'): [<module 'conftest' from 'D:\\ossd\\scikit-learn-yxiao\\conftest.py'>, <module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'>, <module 'sklearn.utils.tests.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\utils\\tests\\conftest.py'>], WindowsPath('D:/ossd/scikit-learn-yxiao'): [<module 'conftest' from 'D:\\ossd\\scikit-learn-yxiao\\conftest.py'>]}
   Iterating over self._dirpath2confmods
        path=WindowsPath('D:/ossd/scikit-learn-yxiao/sklearn/utils/tests'), mods=[<module 'conftest' from 'D:\\ossd\\scikit-learn-yxiao\\conftest.py'>, <module 'sklearn.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\conftest.py'>, <module 'sklearn.utils.tests.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\utils\\tests\\conftest.py'>]
================================================================ short test summary info =================================================================
ERROR sklearn/utils/tests/conftest.py - AssertionError: <module 'sklearn.utils.tests.conftest' from 'd:\\ossd\\scikit-learn-yxiao\\sklearn\\utils\\tests\\conftest.py'> should not be in [<mod...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================== 1 warning, 1 error in 1.58s ===============================================================
```

</details>

Labels: Bug