Repository: pyinstaller/pyinstaller

Issue:
site-packages\django\contrib\sites has managment.py file which is conflicting with managment directory and causing a issue in tree get issue in _pyi_pkgutil_iter_modules as dirretory name and file are same 
When I execute a generated exe from pyinstaller, it is failing as django.contrib.sites has mangment.py file which is conflicting with package which by default will have management as directory.

as technically there is no management folder but when find_commands whose iterator is override by _pyi_pkgutil_iter_modules id expecting a tree node with dict but it has a file with managment.py so it is failing as str error.

If any app has management folder and also managment.py it will yield in to this issue. We can rename custom apps to avoid the file name duplication with management folder but how to deal with standard dajngo apps like django.contrib.sites

here is the stack trace 



-> for pkg_name_part in pkg_prefix.parts:
(Pdb) pkg_prefix.parts
('django', 'contrib', 'sites', 'management', 'commands')          ====>    management here is expected a dict , as dict is not there it will be there in toc but as a file is there. it finds the file as str instead of dict and so is the assert.
(Pdb) c
Traceback (most recent call last):
  File "manage.py", line 19, in <module>
    execute_from_command_line(sys.argv)
  File "django\core\management\__init__.py", line 442, in execute_from_command_line
  File "django\core\management\__init__.py", line 424, in execute
  File "django\core\management\__init__.py", line 222, in main_help_text
  File "django\core\management\__init__.py", line 78, in get_commands
  File "django\core\management\__init__.py", line 35, in find_commands
  File "django\core\management\__init__.py", line 35, in <listcomp>
  File "C:\PythonWebCode\testone\Lib\site-packages\PyInstaller\hooks\rthooks\pyi_rth_pkgutil.py", line 102, in _pyi_pkgutil_iter_modules
    for pkg_name_part in pkg_prefix.parts:
AttributeError: 'str' object has no attribute 'get'

How can we address this issue ?




Discussion:
The following check 
https://github.com/pyinstaller/pyinstaller/blob/7b3e34758d330764c52b3ac8767fb0e8ff995f37/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py#L102
should be changed into 
```python
if not isinstance(tree_node, dict):
```

This way, we handle both the case when the path does not exist (`None`) and the case when the path corresponds to a module (in PYZ) instead of a package (`str`).

Resolving patch:
diff --git a/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py b/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py
--- a/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py
+++ b/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py
@@ -99,7 +99,10 @@ def _pyi_pkgutil_iter_modules(path=None, prefix=''):
                 tree_node = importer.toc_tree
                 for pkg_name_part in pkg_prefix.parts:
                     tree_node = tree_node.get(pkg_name_part)
-                    if tree_node is None:
+                    if not isinstance(tree_node, dict):
+                        # This check handles two cases:
+                        #  a) path does not exist (`tree_node` is `None`)
+                        #  b) path corresponds to a module instead of a package (`tree_node` is a leaf node (`str`)).
                         tree_node = {}
                         break
 
diff --git a/PyInstaller/utils/hooks/django.py b/PyInstaller/utils/hooks/django.py
--- a/PyInstaller/utils/hooks/django.py
+++ b/PyInstaller/utils/hooks/django.py
@@ -89,7 +89,7 @@ def _remove_class(class_name):
     if hasattr(settings, 'TEMPLATES'):
         for templ in settings.TEMPLATES:
             backend = _remove_class(templ['BACKEND'])
-            hiddenimports += backend
+            hiddenimports.append(backend)
             # Include context_processors.
             if hasattr(templ, 'OPTIONS'):
                 if hasattr(templ['OPTIONS'], 'context_processors'):


Test patch:
diff --git a/tests/functional/test_pkgutil.py b/tests/functional/test_pkgutil.py
--- a/tests/functional/test_pkgutil.py
+++ b/tests/functional/test_pkgutil.py
@@ -171,3 +171,45 @@ def test_pkgutil_iter_modules_macos_app_bundle(script_dir, tmpdir, pyi_builder,
         print("RESULTS", results_frozen, "\n\n")
 
         assert results_unfrozen == results_frozen
+
+
+# Two tests that reproduce the situation from #8191. In the first test, `pkgutil.iter_modules()` is called on a path
+# that corresponds to a module instead of the package. In the second test, we add a sub-directory component to the path
+# that corresponds to a module. Both cases should be handled gracefully by our `iter_modules` override.
+def test_pkgutil_iter_modules_with_module_path(pyi_builder):
+    pyi_builder.test_source(
+        """
+        import os
+        import pkgutil
+        import json.encoder  # Our test module
+
+        # Path to iterate over; sys._MEIPASS/json/encoder
+        search_path, _ = os.path.splitext(json.encoder.__file__)
+
+        # pkgutil.iter_modules()
+        print("Search path:", search_path)
+        entries = list(pkgutil.iter_modules([search_path]))
+        print("Entries:", entries)
+        assert len(entries) == 0, "Expected no entries!"
+        """
+    )
+
+
+def test_pkgutil_iter_modules_with_module_path_subdir(pyi_builder):
+    pyi_builder.test_source(
+        """
+        import os
+        import pkgutil
+        import json.encoder  # Our test module
+
+        # Path to iterate over; sys._MEIPASS/json/encoder/nonexistent
+        search_path, _ = os.path.splitext(json.encoder.__file__)
+        search_path = os.path.join(search_path, 'nonexistent')
+
+        # pkgutil.iter_modules()
+        print("Search path:", search_path)
+        entries = list(pkgutil.iter_modules([search_path]))
+        print("Entries:", entries)
+        assert len(entries) == 0, "Expected no entries!"
+        """
+    )