Repository: pyinstaller/pyinstaller

Issue:
`hook-pkg_resources.py` is including hidden imports that are not available
https://github.com/pyinstaller/pyinstaller/blob/3b5d233d02472ad9d589a798d19ad3d3df645223/PyInstaller/hooks/hook-pkg_resources.py#L20

and

https://github.com/pyinstaller/pyinstaller/blob/3b5d233d02472ad9d589a798d19ad3d3df645223/PyInstaller/hooks/hook-pkg_resources.py#L30

Refer to https://github.com/pypa/setuptools/issues/1963 for the past need for including `py2_warn` and [the subsequent removal of py2_warn from setuptools](https://github.com/pypa/setuptools/pull/2238).

The `pkg_resources.py2_warn` submodule was introduced in v45.0.0 and removed in v49.1.1. Therefore, an check for those versions would look like this:
```python
if is_module_satisfies("setuptools >= 45.0.0, < 49.1.1"):
    hiddenimports.append('pkg_resources.py2_warn')
```

As for `pkg_resources.markers`, I haven't found any references to any `pkg_resources.markers` packages through Google searches. I'm not convinced that it ever existed at any point. Perhaps it was being confused for (or just a typo of) `packaging.markers`, which does exist. Since `hook-pkg_resources.py` already collects the submodules of `packaging`, there doesn't need to be an extra hidden import added for `packaging.markers`. Therefore, I think that the `pkg_resources.markers` hidden import line can be removed completely.

Another module that it is trying to include is `pkg_resources._vendor.pyparsing.diagrams`. This module can only be used when the `railroad` module is available. Therefore, a check should be added to make sure that the `railroad` module is available, and filter out the diagrams module when it isn't.

For example:
```python
from PyInstaller.utils.hooks import (
    collect_submodules,
    is_module_satisfies,
    can_import_module,
    is_module_or_submodule
)

hiddenimports = []

# pkg_resources keeps vendored modules in its _vendor subpackage and does sys.meta_path based import magic to expose
# them as pkg_resources.extern.*
if not can_import_module('railroad'):
    # The `railroad` package is an optional requirement for `pyparsing`. `pyparsing.diagrams` depends on `railroad`, so
    # filter it out when `railroad` is not available.
    hiddenimports += collect_submodules(
        'pkg_resources._vendor',
        lambda name: is_module_or_submodule(name, 'pkg_resources._vendor.pyparsing.diagrams'))
else:
    hiddenimports += collect_submodules('pkg_resources._vendor')
```

PyInstaller: v5.1


Discussion:
Hmm, strictly speaking yes you're right. Missing hidden imports only lead to warnings and PyInstaller's warnings are pretty useless across the board so I feel that it's not particularly worth the developer time required hounding them all out. That said, I'll happily accept a pull request if you disagree.
I don't necessarily disagree with you, but since I already have some changes that seem to fix the warnings, I might as well make a PR for them. I'll leave it up to you and the other maintainers to decide to review and merge it (or reject it).
Sure, if there's a PR, I'll happily merge it.

Resolving patch:
diff --git a/PyInstaller/hooks/hook-pkg_resources.py b/PyInstaller/hooks/hook-pkg_resources.py
--- a/PyInstaller/hooks/hook-pkg_resources.py
+++ b/PyInstaller/hooks/hook-pkg_resources.py
@@ -9,26 +9,32 @@
 # SPDX-License-Identifier: (GPL-2.0-or-later WITH Bootloader-exception)
 #-----------------------------------------------------------------------------
 
-from PyInstaller.utils.hooks import collect_submodules, is_module_satisfies
+from PyInstaller.utils.hooks import collect_submodules, is_module_satisfies, can_import_module
 
 # pkg_resources keeps vendored modules in its _vendor subpackage, and does sys.meta_path based import magic to expose
 # them as pkg_resources.extern.*
-hiddenimports = collect_submodules('pkg_resources._vendor')
+
+# The `railroad` package is an optional requirement for `pyparsing`. `pyparsing.diagrams` depends on `railroad`, so
+# filter it out when `railroad` is not available.
+if can_import_module('railroad'):
+    hiddenimports = collect_submodules('pkg_resources._vendor')
+else:
+    hiddenimports = collect_submodules(
+        'pkg_resources._vendor', filter=lambda name: 'pkg_resources._vendor.pyparsing.diagram' not in name
+    )
 
 # pkg_resources v45.0 dropped support for Python 2 and added this module printing a warning. We could save some bytes if
 # we would replace this by a fake module.
-hiddenimports.append('pkg_resources.py2_warn')
+if is_module_satisfies('setuptools >= 45.0.0, < 49.1.1'):
+    hiddenimports.append('pkg_resources.py2_warn')
 
 excludedimports = ['__main__']
 
 # Some more hidden imports. See:
 # https://github.com/pyinstaller/pyinstaller-hooks-contrib/issues/15#issuecomment-663699288 `packaging` can either be
-# its own package, or embedded in `pkg_resources._vendor.packaging`, or both. Assume the worst and include both if
-# present.
+# its own package, or embedded in `pkg_resources._vendor.packaging`, or both.
 hiddenimports += collect_submodules('packaging')
 
-hiddenimports += ['pkg_resources.markers']
-
 # As of v60.7, setuptools vendored jaraco and has pkg_resources use it. Currently, the pkg_resources._vendor.jaraco
 # namespace package cannot be automatically scanned due to limited support for pure namespace packages in our hook
 # utilities.


Test patch: