Repository: pyinstaller/pyinstaller

Issue:
Packaged binary doesn't support long filenames/paths under Windows
<!--
Welcome to the PyInstaller issue tracker! Before creating an issue, please heed the following:

1. This tracker should only be used to report bugs and request features / enhancements to PyInstaller
    - For questions and general support, use the mailing list, see
	  <http://www.pyinstaller.org/support.html#mailing-list>
2. Use the search function before creating a new issue. Duplicates will be closed and directed to
   the original discussion.
3. When making a bug report, make sure you provide all required information. The easier it is for
   maintainers to reproduce, the faster it'll be fixed.
-->

## Description of the issue

We have a long standing bug that [rdiff-backup](https://github.com/rdiff-backup/rdiff-backup) doesn't support long paths (above 255 characters) even if the option is set accordingly in Windows. As you can read at https://github.com/rdiff-backup/rdiff-backup/issues/236 I found at last that the issue only appears when packaged with PyInstaller, and not when calling directly the program from source code, but I don't understand what the difference might be in this regard.

One possible explanation would be that the application manifest of the "compiled" executable is missing the `longPathAware` element described in https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation. If this is the case, given that Python seems to have this capability, the executable compiled by PyInstaller could also have it without complication, but this is just a guess on my side.

## Context information (for bug reports)

* Output of `pyinstaller --version`: ```4.1```
* Version of Python: ```Python 3.9.1 (tags/v3.9.1:1e5d33e, Dec  7 2020, 17:08:21) [MSC v.1927 64 bit (AMD64)]```
* Platform: Windows-10-10.0.17763
* Did you also try this on another platform? Does it work there? n/a (Windows specific)


* follow *all* the instructions in our "If Things Go Wrong" Guide => n/a (it's not a simple packaging issue, the program is used since years by many people, it works but it doesn't support long filenames/paths)

### Make sure [everything is packaged correctly](https://github.com/pyinstaller/pyinstaller/wiki/How-to-Report-Bugs#make-sure-everything-is-packaged-correctly)

  * [x] start with clean installation
  * [x] use the latest development version
  * [x] Run your frozen program **from a command window (shell)** — instead of double-clicking on it
  * [x] Package your program in **--onedir mode**
  * [x] Package **without UPX**, say: use the option `--noupx` or set `upx=False` in your .spec-file
  * [x] Repackage you application in **verbose/debug mode**. For this, pass the option `--debug` to `pyi-makespec` or `pyinstaller` or use `EXE(..., debug=1, ...)` in your .spec file.

### A minimal example program which shows the error

I understand what you are asking for but without being able to run the program in debug mode, I have currently no clue what the issue might be and can't hence create a reproducer. Any hint is more than welcome.

The issue itself is described at https://github.com/rdiff-backup/rdiff-backup/issues/236 and the instructions on how to setup the same environment are available at https://github.com/rdiff-backup/rdiff-backup/tree/ericzolf-win-py39-64bits/tools/windows

For the purpose of this bug report, I've just replaced the pyinstaller parameters in `playbook-build-rdiff-backup.yml`:

```diff
-        pyinstaller --onefile
+        pyinstaller --onedir --noupx --debug all
```

### Stacktrace / full error message

The issue is that there is no error message, the bundled binary is just misbehaving, but here you are, perhaps it gives you an idea:

[rdiff-backup_pyinstaller.debug.txt](https://github.com/pyinstaller/pyinstaller/files/5750077/rdiff-backup_pyinstaller.debug.txt)


Discussion:
Confirmed. Here's a reproducer (using a slightly modified long filename from the linked issue):

```python
import os

filename = 'c:/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/rrrrrrrrrrrrrrrrrrr/rdiff-backup-data/increments/PortableApps/PortableApps/LibreOfficePortablePrevious/App/DefaultData/settings/user/extensions/bundled/registry/com.sun.star.comp.deployment.configuration.PackageRegistryBackend/lu23864u7l9b0.tmp'
dirname = os.path.dirname(filename)

print(f"Filename length: {len(filename)}")
print(f"Dirname length: {len(dirname)}")

os.makedirs(dirname, exist_ok=True)
with open(filename, 'w') as f:
    f.write("Hello world!")   
with open(filename, 'r') as f:
    data = f.read()
    print("Read back:", data)
```

Running unfrozen version results in:
```
Filename length: 278
Dirname length: 260
Read back: Hello world!
```

In a frozen application, the `os.makedirs()` call raises exception:
```
Filename length: 278
Dirname length: 260
Traceback (most recent call last):
  File "test_fname.py", line 10, in <module>
    os.makedirs(dirname, exist_ok=True)
  File "os.py", line 225, in makedirs
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'c:/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/rrrrrrrrrrrrrrrrrrr/rdiff-backup-data/increments/PortableApps/PortableApps/LibreOfficePortablePrevious/App/DefaultData/settings/user/extensions/bundled/registry/com.sun.star.comp.deployment.configuration.PackageRegistryBackend'
[372] Failed to execute script test_fname
```

Your initial analysis appears to be correct, and the issue is due to `longPathAware` missing from the frozen application's manifest.

Adding a `windowsSettings` tag with `longPathAware` at the end of `program.exe.manifest` for onedir build of `program.exe`, i.e.,
```xml
<assembly [...]>
  [...]
  <application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
        <ws2:longPathAware>true</ws2:longPathAware>
    </windowsSettings>
   </application>
</assembly>
```
 fixes the problem for the above example.

Resolving patch:
diff --git a/PyInstaller/utils/win32/winmanifest.py b/PyInstaller/utils/win32/winmanifest.py
--- a/PyInstaller/utils/win32/winmanifest.py
+++ b/PyInstaller/utils/win32/winmanifest.py
@@ -896,6 +896,23 @@ def todom(self):
         cE.aChild(caE)
         docE.aChild(cE)
 
+        # Add application.windowsSettings section to enable longPathAware
+        # option (issue #5423).
+        if self.manifestType == "assembly":
+            aE = doc.cE("application")
+            aE.setAttribute("xmlns", "urn:schemas-microsoft-com:asm.v3")
+            wsE = doc.cE("windowsSettings")
+            lpaE = doc.cE("longPathAware")
+            lpaE.setAttribute(
+                "xmlns",
+                "http://schemas.microsoft.com/SMI/2016/WindowsSettings"
+            )
+            lpaT = doc.cT("true")
+            lpaE.aChild(lpaT)
+            wsE.aChild(lpaE)
+            aE.aChild(wsE)
+            docE.aChild(aE)
+
         return doc
 
     def toprettyxml(self, indent="  ", newl=os.linesep, encoding="UTF-8"):


Test patch: