Repository: marcelotduarte/cx_Freeze (python)

Issue:
cx_Freeze 7.0.0 and PyTorch 2.2.2+cu118: TypeError: argument docstring of add_docstring should be a str
**Describe the bug**
An error occurs when try to freeze pytorch project with cx-Freeze.

```
copying C:\Project\_build\venv\lib\site-packages\torch\functional.py -> C:\Project\_build\out\lib\torch\functional.py
Traceback (most recent call last):
  File "C:\Project\_build\venv\Lib\site-packages\cx_Freeze\initscripts\__startup__.py", line 138, in run
    module_init.run(name + "__main__")
  File "C:\Project\_build\venv\Lib\site-packages\cx_Freeze\initscripts\console.py", line 17, in run
    exec(code, module_main.__dict__)
  File "project.py", line 1, in <module>
    import torch
  File "C:/Users/bw7715/OneDrive - Zebra Technologies/Projects/python_projects/cx_freeze_issues/p39_cx70_tf_np/_build/venv/lib/site-packages/torch/__init__.py", line 1215, in <module>
    from .storage import _StorageBase, TypedStorage, _LegacyStorage, UntypedStorage, _warn_typed_storage_removal
  File "C:\Project\_build\venv\lib\site-packages\torch\storage.py", line 14, in <module>
    import numpy as np
  File "C:\Project\_build\venv\lib\site-packages\numpy\__init__.py", line 173, in <module>
    from . import core
  File "C:\Project\_build\venv\lib\site-packages\numpy\core\__init__.py", line 24, in <module>
    from . import multiarray
  File "C:\Project\_build\venv\lib\site-packages\numpy\core\multiarray.py", line 86, in <module>
    def empty_like(prototype, dtype=None, order=None, subok=None, shape=None):
  File "C:\Project\_build\venv\lib\site-packages\numpy\core\overrides.py", line 178, in decorator
    return array_function_dispatch(
  File "C:\Project\_build\venv\lib\site-packages\numpy\core\overrides.py", line 158, in decorator
    add_docstring(implementation, dispatcher.__doc__)
TypeError: argument docstring of add_docstring should be a str
```

**To Reproduce**
requirements.txt
```
--find-links https://download.pytorch.org/whl/cu118/torch_stable.html

cx_Freeze==7.0.0
torch==2.2.2+cu118
torchvision==0.17.2+cu118
```

freeze.bat
```
echo off
set WORKSPACE=%~dp0

set BUILD_DIRPATH=%WORKSPACE%\_build
set BASE_PYTHON_DIRPATH=C:\Program Files\Python39

set VENV_DIRPATH=%BUILD_DIRPATH%\venv
set TARGET_DIR=%BUILD_DIRPATH%\out

set SCRIPTS_DIRPATH=%VENV_DIRPATH%\Scripts
set PYTHON_EXE_PATH="%SCRIPTS_DIRPATH%\python.exe"
set FREEZER_EXE_PATH="%SCRIPTS_DIRPATH%\cxfreeze.exe"

set TARGET_NAME=project_out.exe
set PROJECT_SOURCE_PY_PATH=project.py

set PACKAGES=torch


"%BASE_PYTHON_DIRPATH%\python" -m venv "%VENV_DIRPATH%"
%PYTHON_EXE_PATH% -m pip install -r requirements.txt

rem %PYTHON_EXE_PATH% -m pip install --pre --extra-index-url https://marcelotduarte.github.io/packages/ cx_Freeze --upgrade

%FREEZER_EXE_PATH% build -O1 -O2 --include-msvcr --build-exe="%TARGET_DIR%" --target-name=%TARGET_NAME% --packages=%PACKAGES% --script=%PROJECT_SOURCE_PY_PATH%


"%TARGET_DIR%\%TARGET_NAME%"
```

project.py
```python
import torch

print(torch.__version__)
```


**Desktop (please complete the following information):**
 - Platform information: Windows 10  and  Ubuntu Linux 22.04
 - OS architecture (e.g. amd64): 64bit
 - cx_Freeze version: 7.0.0
 - Python version: 3.9

pip list:
```
.\python.exe -m pip list
Package           Version
----------------- ------------
cx_Freeze         7.0.0
cx_Logging        3.2.0
filelock          3.13.4
fsspec            2024.3.1
Jinja2            3.1.3
lief              0.14.1
MarkupSafe        2.1.5
mpmath            1.3.0
networkx          3.2.1
numpy             1.26.4
pillow            10.3.0
pip               22.0.4
setuptools        69.5.1
sympy             1.12
torch             2.2.2+cu118
torchvision       0.17.2+cu118
typing_extensions 4.11.0
wheel             0.43.0
```

With `cx_Freeze==6.15.16` this project works.


Discussion:
When try to use `pip install --force --no-cache --pre --extra-index-url https://marcelotduarte.github.io/packages/ cx_Freeze` with  `cx_Freeze-7.1.0.dev3-cp39-cp39-win_amd64` I get the same error.
I noticed also that with `cx_Freeze-7.1.0.dev3` there is an error during freezing:
```
Looking in indexes: https://pypi.org/simple, https://marcelotduarte.github.io/packages/
Collecting cx_Freeze
  Downloading https://marcelotduarte.github.io/packages/cx-freeze/cx_Freeze-7.1.0.dev3-cp39-cp39-win_amd64.whl (2.0 MB)
     ---------------------------------------- 2.0/2.0 MB 1.4 MB/s eta 0:00:00
Collecting wheel<=0.43.0,>=0.42.0
  Downloading wheel-0.43.0-py3-none-any.whl (65 kB)
     ---------------------------------------- 65.8/65.8 KB 107.6 kB/s eta 0:00:00
Collecting setuptools<70,>=62.6
  Downloading setuptools-69.5.1-py3-none-any.whl (894 kB)
     ---------------------------------------- 894.6/894.6 KB 1.2 MB/s eta 0:00:00
Collecting lief<=0.15.0,>=0.12.0
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)'))': /latest/lief/lief-0.15.0-cp39-cp39-win_amd64.whl
  WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)'))': /latest/lief/lief-0.15.0-cp39-cp39-win_amd64.whl
  WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)'))': /latest/lief/lief-0.15.0-cp39-cp39-win_amd64.whl
  WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)'))': /latest/lief/lief-0.15.0-cp39-cp39-win_amd64.whl
  WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)'))': /latest/lief/lief-0.15.0-cp39-cp39-win_amd64.whl
ERROR: Could not install packages due to an OSError: HTTPSConnectionPool(host='lief.s3-website.fr-par.scw.cloud', port=443): Max retries exceeded with url: /latest/lief/lief-0.15.0-cp39-cp39-win_amd64.whl (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))
```

But once it has appeared, the process goes on.



In version 7.0.0 these errors and warnings do not appear 
```
Looking in links: https://download.pytorch.org/whl/cu118/torch_stable.html
Collecting cx_Freeze==7.0.0
  Using cached cx_Freeze-7.0.0-cp39-cp39-win_amd64.whl (2.0 MB)
Collecting numpy==1.26.4
  Using cached numpy-1.26.4-cp39-cp39-win_amd64.whl (15.8 MB)
Collecting torch==2.2.2+cu118
  Using cached https://download.pytorch.org/whl/cu118/torch-2.2.2%2Bcu118-cp39-cp39-win_amd64.whl (2704.2 MB)
Collecting torchvision==0.17.2+cu118
  Using cached https://download.pytorch.org/whl/cu118/torchvision-0.17.2%2Bcu118-cp39-cp39-win_amd64.whl (4.9 MB)
Collecting lief<0.15.0,>=0.12.0
  Using cached lief-0.14.1-cp39-cp39-win_amd64.whl (2.2 MB)
Collecting cx-Logging>=3.1
  Using cached cx_Logging-3.2.0-cp39-cp39-win_amd64.whl (26 kB)
```
> TypeError: argument docstring of add_docstring should be a str

I had already commented [here](https://github.com/marcelotduarte/cx_Freeze/issues/2280#issuecomment-1980922796):
"""
numpy uses its docstring, so remove the optimization that it works. The optimization works like described [here](https://docs.python.org/3/using/cmdline.html#cmdoption-O).
See also: https://github.com/numpy/numpy/issues/13248#issuecomment-480412876
"""
And complementing the comment, I can say that in the previous version, when you used -O -OO in reality only -O ended up being used and it gave you the impression of using complete optimization. Therefore with numpy only -O1 can be used. I'll see if there's a way to restrict the optimization to just the numpy module.

> ERROR: Could not install packages due to an OSError: HTTPSConnectionPool(host='lief.s3-website.fr-par.scw.cloud', ...

This is a server error when the protocols, certificates, have nothing to do with cx_Freeze. Alias is a pip command.


Patch:
diff --git a/cx_Freeze/hooks/numpy.py b/cx_Freeze/hooks/numpy.py
--- a/cx_Freeze/hooks/numpy.py
+++ b/cx_Freeze/hooks/numpy.py
@@ -87,6 +87,20 @@ def load_numpy_core__add_newdocs(
     finder.include_module("numpy.core._multiarray_tests")
 
 
+def load_numpy_core_overrides(finder: ModuleFinder, module: Module) -> None:
+    """Recompile the numpy.core.overrides module to limit optimization by
+    avoiding removing docstrings, which are required for this module.
+    """
+    code_string = module.file.read_text(encoding="utf_8")
+    module.code = compile(
+        code_string.replace("dispatcher.__doc__", "dispatcher.__doc__ or ''"),
+        os.fspath(module.file),
+        "exec",
+        dont_inherit=True,
+        optimize=min(finder.optimize, 1),
+    )
+
+
 def load_numpy__distributor_init(finder: ModuleFinder, module: Module) -> None:
     """Fix the location of dependent files in Windows and macOS."""
     if IS_LINUX or IS_MINGW:


Test patch:
diff --git a/tests/test_hooks_pandas.py b/tests/test_hooks_pandas.py
--- a/tests/test_hooks_pandas.py
+++ b/tests/test_hooks_pandas.py
@@ -24,7 +24,7 @@
 @pytest.mark.datafiles(SAMPLES_DIR / "pandas")
 def test_pandas(datafiles: Path) -> None:
     """Test that the pandas/numpy is working correctly."""
-    output = run_command(datafiles)
+    output = run_command(datafiles, "python setup.py build_exe -O2")
     executable = datafiles / BUILD_EXE_DIR / f"test_pandas{SUFFIX}"
     assert executable.is_file()