Repository: pytest-dev/pytest
Pull request: fix `AttributeError`  crash when using `--import-mode=importlib`
closes #13026 .


- [X] Include new tests or update existing tests when applicable.
- [X] Allow maintainers to push and squash when merging my commits.
- [X] Create a new changelog file in the `changelog` folder.


Only parent modules with the `__path__` attribute can be used by the `find_spec` function, and most of the standard library does not meet this condition.


https://github.com/python/cpython/blob/3.9/Lib/importlib/_bootstrap_external.py#L1223
```python
   def _find_parent_path_names(self):
        """Returns a tuple of (parent-module-name, parent-path-attr-name)"""
        parent, dot, me = self._name.rpartition('.')
        if dot == '':
            # This is a top-level module. sys.path contains the parent path.
            return 'sys', 'path'
        # Not a top-level module. parent-module.__path__ contains the
        #  parent path.
        return parent, '__path__'
```


So this PR adds a condition check.



Problem statement:
Upgrade pytest from 8.3.3 to 8.3.4 triggers `AttributeError: module 'code' has no attribute '__path__'. Did you mean: '__all__'?`
- [x] a detailed description of the bug or problem you are having

I have a number of pytest tests that used to work in 8.3.3.

Since I upgraded to pytest 8.3.4, these tests fail reliably in 2 different projects.
(see details below).

- [x] output of `pip list` from the virtual environment you are using

```
poetry run pip list
Package                   Version
------------------------- --------------
anyio                     4.6.2.post1
appnope                   0.1.4
argon2-cffi               23.1.0
argon2-cffi-bindings      21.2.0
arrow                     1.3.0
asttokens                 3.0.0
async-lru                 2.0.4
attrs                     24.2.0
babel                     2.16.0
beautifulsoup4            4.12.3
black                     24.10.0
bleach                    6.2.0
boto3                     1.35.74
botocore                  1.35.74
build                     1.2.2.post1
CacheControl              0.14.0
certifi                   2024.8.30
cffi                      1.17.1
charset-normalizer        3.4.0
cleo                      2.1.0
click                     8.1.7
comm                      0.2.2
crashtest                 0.4.1
debugpy                   1.8.9
decorator                 5.1.1
defusedxml                0.7.1
distlib                   0.3.9
dulwich                   0.21.7
executing                 2.1.0
fastjsonschema            2.21.1
filelock                  3.16.1
fqdn                      1.5.1
h11                       0.14.0
httpcore                  1.0.7
httpx                     0.28.0
idna                      3.10
iniconfig                 2.0.0
installer                 0.7.0
ipykernel                 6.29.5
ipython                   8.30.0
isoduration               20.11.0
jaraco.classes            3.4.0
jedi                      0.19.2
Jinja2                    3.1.4
jmespath                  1.0.1
json5                     0.10.0
jsonpointer               3.0.0
jsonschema                4.23.0
jsonschema-specifications 2024.10.1
jupyter_client            8.6.3
jupyter_core              5.7.2
jupyter-events            0.10.0
jupyter-lsp               2.2.5
jupyter_server            2.14.2
jupyter_server_terminals  0.5.3
jupyterlab                4.3.2
jupyterlab_pygments       0.3.0
jupyterlab_server         2.27.3
keyring                   24.3.1
MarkupSafe                3.0.2
matplotlib-inline         0.1.7
mistune                   3.0.2
more-itertools            10.5.0
msgpack                   1.1.0
mypy-extensions           1.0.0
nbclient                  0.10.1
nbconvert                 7.16.4
nbformat                  5.10.4
nest-asyncio              1.6.0
notebook_shim             0.2.4
numpy                     2.1.3
overrides                 7.7.0
packaging                 24.2
pandas                    2.2.3
pandocfilters             1.5.1
parso                     0.8.4
pathspec                  0.12.1
pexpect                   4.9.0
pip                       24.2
pkginfo                   1.11.2
platformdirs              4.3.6
plotly                    5.24.1
pluggy                    1.5.0
poetry                    1.8.4
poetry-core               1.9.1
poetry-plugin-export      1.8.0
prometheus_client         0.21.1
prompt_toolkit            3.0.48
psutil                    6.1.0
ptyprocess                0.7.0
pure_eval                 0.2.3
pycparser                 2.22
Pygments                  2.18.0
pyproject_hooks           1.2.0
pytest                    8.3.3
python-dateutil           2.9.0.post0
python-json-logger        2.0.7
pytz                      2024.2
PyYAML                    6.0.2
pyzmq                     26.2.0
RapidFuzz                 3.10.1
referencing               0.35.1
requests                  2.32.3
requests-toolbelt         1.0.0
rfc3339-validator         0.1.4
rfc3986-validator         0.1.1
rpds-py                   0.22.3
s3transfer                0.10.4
Send2Trash                1.8.3
setuptools                75.6.0
shellingham               1.5.4
six                       1.16.0
sniffio                   1.3.1
soupsieve                 2.6
stack-data                0.6.3
tenacity                  9.0.0
terminado                 0.18.1
tinycss2                  1.4.0
tokenize_rt               6.1.0
tomlkit                   0.13.2
tornado                   6.4.2
traitlets                 5.14.3
trove-classifiers         2024.10.21.16
types-python-dateutil     2.9.0.20241003
tzdata                    2024.2
uri-template              1.3.0
urllib3                   2.2.3
virtualenv                20.27.1
wcwidth                   0.2.13
webcolors                 24.11.1
webencodings              0.5.1
websocket-client          1.8.0
xattr                     1.1.0
```

- [x] pytest and operating system versions

```
pytest 8.3.4
Mac OS X updated
python 3.13.0
```

Maybe related to this code below:

**[tool.pytest.ini_options]
addopts = [
    "--import-mode=importlib",
]
pythonpath = "code"**


```
✗ cat pyproject.toml
[tool.poetry]
name = "..."
version = "0.1.0"
description = "Analysis for ..."
authors = ["Peter Vandenabeele <peter.vandenabeele@...>"]
license = "Proprietary"
readme = "README.md"
package-mode = false

[tool.poetry.dependencies]
python = "^3.13"
numpy = "^2.1.2"
pandas = "^2.2.3"
jupyterlab = "^4.2.5"
boto3 = "^1.35.51"
plotly = "^5.24.1"

[tool.poetry.group.dev.dependencies]
pytest = "8.3.4"  # FAILS with 8.3.4 ;  SUCCESS with 8.3.3 
black = {version="*", extras=["jupyter"]}

[tool.pytest.ini_options]
addopts = [
    "--import-mode=importlib",
]
pythonpath = "code"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
```

- [x] minimal example if possible

```
___________________________________________ ERROR collecting code/tests/..._test.py ___________________________________________
<frozen importlib._bootstrap_external>:1564: in find_spec
    ???
<frozen importlib._bootstrap_external>:1362: in __init__
    ???
<frozen importlib._bootstrap_external>:1378: in _get_parent_path
    ???
E   AttributeError: module 'code' has no attribute '__path__'. Did you mean: '__all__'?
======================================================== short test summary info =========================================================
ERROR code/tests/....._test.py - AttributeError: module 'code' has no attribute '__path__'. Did you mean: '__all__'?
same 2/4
same 3/4
same 4/4
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 4 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================================== 4 errors in 0.09s ============================================================

✗ vim pyproject.toml  # CHANGE pytest back to 8.3.3

✗ poetry lock
Updating dependencies
Resolving dependencies... (4.0s)
Writing lock file

✗ poetry install
Installing dependencies from lock file

Package operations: 0 installs, 1 update, 0 removals

  - Downgrading pytest (8.3.4 -> 8.3.3)

✗ pytest
========================================================== test session starts ===========================================================
platform darwin -- Python 3.13.0, pytest-8.3.3, pluggy-1.5.0
rootdir: /Users/peter_v/BF/github/...
configfile: pyproject.toml
plugins: anyio-4.6.2.post1
collected 58 items

code/tests/..._test.py ..                                                                                         [  3%]
code/tests/..._test.py ......                                                                                        [ 13%]
code/tests/..._test.py ..............                                                                              [ 37%]
code/tests/..._test.py ....................................                                                             [100%]

=========================================================== 58 passed in 0.39s ===========================================================
```

Hints from issue discussion:
When the top-level directory is named `code`, the issue can be reproduced, for example:  
```  
code/foo/test_some.py  
```  

When the top-level directory is not named `code`, it executes normally, such as:  
```  
my_code/foo/test_some.py  
foo/code/test_some.py  
```  

I am currently trying to trace further details.  

-----

UPDATE：

`code` is a Python standard library. When other code uses this library, it causes the following code to evaluate as false.  
https://github.com/pytest-dev/pytest/blob/main/src/_pytest/pathlib.py#L670-L671

That is, `sys.modules['code']` refers to the standard library rather than reloading the user's code file.  


----
If `parent_module` is a standard library, should we attempt to reload it and modify `sys.modules`?  

Sounds a bit risky—does anyone else have any thoughts?

Commit message: fix #13026

Patch to review:
diff --git a/changelog/13026.bugfix.rst b/changelog/13026.bugfix.rst
new file mode 100644
index 00000000000..a7a79234588
--- /dev/null
+++ b/changelog/13026.bugfix.rst
@@ -0,0 +1 @@
+Fixed ``AttributeError``  crash when using ``--import-mode=importlib`` when top-level directory same name as the standard library.
diff --git a/src/_pytest/pathlib.py b/src/_pytest/pathlib.py
index 55b27985d31..23bb4c29986 100644
--- a/src/_pytest/pathlib.py
+++ b/src/_pytest/pathlib.py
@@ -668,7 +668,7 @@ def _import_module_using_spec(
     parent_module: ModuleType | None = None
     if parent_module_name:
         parent_module = sys.modules.get(parent_module_name)
-        if parent_module is None:
+        if parent_module is None or not hasattr(parent_module, "__path__"):
             # Get parent_location based on location, get parent_path based on path.
             if module_path.name == "__init__.py":
                 # If the current module is in a package,
diff --git a/testing/test_pathlib.py b/testing/test_pathlib.py
index 436a6b9cae3..a21d1edf970 100644
--- a/testing/test_pathlib.py
+++ b/testing/test_pathlib.py
@@ -922,6 +922,37 @@ def test_my_test():
         result = pytester.runpytest("--import-mode=importlib")
         result.stdout.fnmatch_lines("* 1 passed *")
 
+    @pytest.mark.parametrize("name", ["code", "time", "math"])
+    def test_importlib_same_name_as_stl(
+        self, pytester, ns_param: bool, tmp_path: Path, name: str
+    ):
+        """Import a namespace package with the same name as the standard library (#13026)."""
+        file_path = pytester.path / f"{name}/foo/test_demo.py"
+        file_path.parent.mkdir(parents=True)
+        file_path.write_text(
+            dedent(
+                """
+            def test_demo():
+                pass
+            """
+            ),
+            encoding="utf-8",
+        )
+
+        # unit test
+        __import__(name)  # import standard library
+
+        import_path(  # import user files
+            file_path,
+            mode=ImportMode.importlib,
+            root=pytester.path,
+            consider_namespace_packages=ns_param,
+        )
+
+        # E2E test
+        result = pytester.runpytest("--import-mode=importlib")
+        result.stdout.fnmatch_lines("* 1 passed *")
+
     def create_installed_doctests_and_tests_dir(
         self, path: Path, monkeypatch: MonkeyPatch
     ) -> tuple[Path, Path, Path]:


Code review comments:
diff_hunk: @@ -668,7 +668,7 @@ def _import_module_using_spec(
     parent_module: ModuleType | None = None
     if parent_module_name:
         parent_module = sys.modules.get(parent_module_name)
-        if parent_module is None:
+        if parent_module is None or not hasattr(parent_module, "__path__"):
path: src/_pytest/pathlib.py
text: @user1:
Can we have a comment explaining the `or not hasattr` bit? Also probably good to move it to a well named variable.


```suggestion
        # ...
        stdlib_module = not hasattr(parent_module, "__path__")
        if parent_module is None or stdlib_module:
```

While one can use `git blame` to discover that, this is the kind of thing that tends to get buried/hidden as code is moved/changed.


@author:
Agreed. I suggest using `need_reimport` as the variable name. The key issue is the lack of the `__path__` attribute, regardless of the module's source. If a module with the same name exists in the current working directory and is imported first, the same error can occur.

diff_hunk: @@ -0,0 +1 @@
+Fixed ``AttributeError``  crash when using ``--import-mode=importlib`` when top-level directory same name as the standard library.
path: changelog/13026.bugfix.rst
text: @user1:
```suggestion
Fixed :class:`AttributeError`  crash when using ``--import-mode=importlib`` when top-level directory same name as another module of the standard library.
```