Repository: pylint-dev/astroid

Issue:
astroid 2.9.1 breaks pylint with missing __init__.py: F0010: error while code parsing: Unable to load file __init__.py
### Steps to reproduce
> Steps provided are for Windows 11, but initial problem found in Ubuntu 20.04

> Update 2022-01-04: Corrected repro steps and added more environment details

1. Set up simple repo with following structure (all files can be empty):
```
root_dir/
|--src/
|----project/ # Notice the missing __init__.py
|------file.py # It can be empty, but I added `import os` at the top
|----__init__.py
```
2. Open a command prompt
3. `cd root_dir`
4. `python -m venv venv`
5. `venv/Scripts/activate`
6. `pip install pylint astroid==2.9.1` # I also repro'd on the latest, 2.9.2
7. `pylint src/project` # Updated from `pylint src`
8. Observe failure:
```
src\project\__init__.py:1:0: F0010: error while code parsing: Unable to load file src\project\__init__.py:
```

### Current behavior
Fails with `src\project\__init__.py:1:0: F0010: error while code parsing: Unable to load file src\project\__init__.py:`

### Expected behavior
Does not fail with error.
> If you replace step 6 with `pip install pylint astroid==2.9.0`, you get no failure with an empty output - since no files have content

### `python -c "from astroid import __pkginfo__; print(__pkginfo__.version)"` output
2.9.1

`python 3.9.1`
`pylint 2.12.2 `



This issue has been observed with astroid `2.9.1` and `2.9.2`


Discussion / hints:
I can't seem to reproduce this in my `virtualenv`. This might be specific to `venv`? Needs some further investigation.
@interifter Which version of `pylint` are you using?
Right, ``pip install pylint astroid==2.9.0``, will keep the local version if you already have one, so I thought it was ``2.12.2`` but that could be false. In fact it probably isn't 2.12.2. For the record, you're not supposed to set the version of ``astroid`` yourself, pylint does, and bad thing will happen if you try to set the version of an incompatible astroid. We might want to update the issue's template to have this information next.
My apologies... I updated the repro steps with a critical missed detail: `pylint src/project`, instead of `pylint src`

But I verified that either with, or without, `venv`, the issue is reproduced.

Also, I never have specified the `astroid` version, before. 

However, this isn't the first time the issue has been observed.
Back in early 2019, a [similar issue](https://stackoverflow.com/questions/48024049/pylint-raises-error-if-directory-doesnt-contain-init-py-file) was observed with either `astroid 2.2.0` or `isort 4.3.5`, which led me to try pinning `astroid==2.9.0`, which worked.
> @interifter Which version of `pylint` are you using?

`2.12.2`

Full env info:

```
Package           Version
----------------- -------
astroid           2.9.2
colorama          0.4.4
isort             5.10.1
lazy-object-proxy 1.7.1
mccabe            0.6.1
pip               20.2.3
platformdirs      2.4.1
pylint            2.12.2
setuptools        49.2.1
toml              0.10.2
typing-extensions 4.0.1
wrapt             1.13.3
```




Fix (patch):
diff --git a/ChangeLog b/ChangeLog
index 6c56d234f9..783391c360 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -7,6 +7,14 @@ What's New in astroid 2.10.0?
 Release date: TBA
 
 
+What's New in astroid 2.9.3?
+============================
+Release date: TBA
+
+* Fixed regression where packages without a ``__init__.py`` file were
+  not recognized or imported correctly.
+
+  Closes #1327
 
 What's New in astroid 2.9.2?
 ============================
diff --git a/astroid/modutils.py b/astroid/modutils.py
index 6d69846925..b20a184021 100644
--- a/astroid/modutils.py
+++ b/astroid/modutils.py
@@ -297,6 +297,9 @@ def _get_relative_base_path(filename, path_to_check):
     if os.path.normcase(real_filename).startswith(path_to_check):
         importable_path = real_filename
 
+    # if "var" in path_to_check:
+    #     breakpoint()
+
     if importable_path:
         base_path = os.path.splitext(importable_path)[0]
         relative_base_path = base_path[len(path_to_check) :]
@@ -307,8 +310,11 @@ def _get_relative_base_path(filename, path_to_check):
 
 def modpath_from_file_with_callback(filename, path=None, is_package_cb=None):
     filename = os.path.expanduser(_path_from_filename(filename))
+    paths_to_check = sys.path.copy()
+    if path:
+        paths_to_check += path
     for pathname in itertools.chain(
-        path or [], map(_cache_normalize_path, sys.path), sys.path
+        paths_to_check, map(_cache_normalize_path, paths_to_check)
     ):
         if not pathname:
             continue


Test changes:
diff --git a/tests/unittest_modutils.py b/tests/unittest_modutils.py
index d6fcb68e25..01d5e5b91e 100644
--- a/tests/unittest_modutils.py
+++ b/tests/unittest_modutils.py
@@ -30,6 +30,7 @@
 import tempfile
 import unittest
 import xml
+from pathlib import Path
 from xml import etree
 from xml.etree import ElementTree
 
@@ -189,6 +190,30 @@ def test_load_from_module_symlink_on_symlinked_paths_in_syspath(self) -> None:
         # this should be equivalent to: import secret
         self.assertEqual(modutils.modpath_from_file(symlink_secret_path), ["secret"])
 
+    def test_load_packages_without_init(self) -> None:
+        """Test that we correctly find packages with an __init__.py file.
+
+        Regression test for issue reported in:
+        https://github.com/PyCQA/astroid/issues/1327
+        """
+        tmp_dir = Path(tempfile.gettempdir())
+        self.addCleanup(os.chdir, os.curdir)
+        os.chdir(tmp_dir)
+
+        self.addCleanup(shutil.rmtree, tmp_dir / "src")
+        os.mkdir(tmp_dir / "src")
+        os.mkdir(tmp_dir / "src" / "package")
+        with open(tmp_dir / "src" / "__init__.py", "w", encoding="utf-8"):
+            pass
+        with open(tmp_dir / "src" / "package" / "file.py", "w", encoding="utf-8"):
+            pass
+
+        # this should be equivalent to: import secret
+        self.assertEqual(
+            modutils.modpath_from_file(str(Path("src") / "package"), ["."]),
+            ["src", "package"],
+        )
+
 
 class LoadModuleFromPathTest(resources.SysPathSetup, unittest.TestCase):
     def test_do_not_load_twice(self) -> None: