Repository: pyinstaller/pyinstaller

Issue:
Adding support for Windows ARM64 native
### Discussed in https://github.com/orgs/pyinstaller/discussions/7188

<div type='discussions-op-text'>

<sup>Originally posted by **infinitepower18** October 25, 2022</sup>
Python 3.11 is the first version to officially support ARM64 Windows. I have tried to build an exe with a simple test program but it crashes when I try to run it. However it works fine if I use the AMD64 Python. So is ARM64 supported by Pyinstaller currently?</div>

To get PyInstaller to build native arm64 applications we need to first figure out how to compile the bootloaders for arm64, preferably in a way that doesn't deviate too far from our current waf based build system.

Seemingly no developers own a Windows arm machine but Windows arm can be virtualised on macOS M1 using [this guide](https://docs.getutm.app/guides/windows/). (Thanks for pointing that out @infinitepower18.)

--- 

FYI: For people wanting to support Windows on ARM now, 32bit intel applications can be ran under emulation on Windows ARM so building using a 32 bit version of Python on a regular intel machine will produce an application which runs, albeit slower, on Windows ARM.



Discussion:
waf doesn't seem to be able to detect MSVC's arm SDK. With [a minimal example waf project](https://github.com/pyinstaller/pyinstaller/discussions/5956#discussioncomment-922571), it goes for the `x64` tool chain by default and finds nothing at all if explicitly told to use `--msvc_targets=arm`. I have managed to get something working by circumventing waf and doing my usual ridiculous oneliner from an ARM MSVC developer prompt:
```
cl -DWIN32 -DHAVE_STRNLEN -Izlib src\main.c src\pyi_apple_events.c src\pyi_archive.c src\pyi_exception_dialog.c src\pyi_global.c src\pyi_launch.c src\pyi_main.c src\pyi_path.c src\pyi_python.c src\pyi_pythonlib.c src\pyi_splash.c src\pyi_splashlib.c    src\pyi_utils.c src\pyi_win32_utils.c zlib\adler32.c zlib\crc32.c zlib\inffast.c zlib\inflate.c zlib\inftrees.c zlib\zutil.c /link /OUT:..\PyInstaller\bootloader\Windows-64bit-arm\run.exe user32.lib advapi32.lib
```
> waf doesn't seem to be able to detect MSVC's arm SDK. With [a minimal example waf project](https://github.com/pyinstaller/pyinstaller/discussions/5956#discussioncomment-922571), it goes for the `x64` tool chain by default and finds nothing at all if explicitly told to use `--msvc_targets=arm`.

Shouldn't that be `arm64`? 
`--msvc_targets=arm64` doesn't work either. `python waf --help` suggests `arm` under the help for `--msvc_targets`.
gcc doesn't support ARM but LLVM+clang does. Installing [LLVM-15.0.2-woa64.exe](https://github.com/llvm/llvm-project/releases/download/llvmorg-15.0.2/LLVM-15.0.2-woa64.exe) gives me an installation of clang which works. waf is able to detect this and build the hello world project. Our bootloader building waf script falls over but this works well:
```
clang -o ../PyInstaller/bootloader/Windows-64bit-arm/run.exe -Izlib/ -DHAVE_STRNLEN src/*.c zlib/*.c -ladvapi32
```
I'll see what I can do to get our waf script to work with Windows+clang.
It would appear that I'm being blocked by a `.a` vs `.lib` suffix convention mismatch 🙄:
```bash
brenainn@WINDOWS-KNDK3G4  /c/pyinstaller/bootloader (develop)
$ python waf --clang --static-zlib all -v
[22/22] Linking build\debug\run_d.exe
18:14:49 runner ['C:\\Program Files\\LLVM\\bin\\clang.exe', '-Wl,--enable-auto-import', '-m64', '-municode', 'src\\main.c.3.o', 'src\\pyi_apple_events.c.2.o', 'src\\pyi_archive.c.2.o', 'src\\pyi_exception_dialog.c.2.o', 'src\\pyi_global.c.2.o', 'src\\pyi_launch.c.2.o', 'src\\pyi_main.c.2.o', 'src\\pyi_path.c.2.o', 'src\\pyi_python.c.2.o', 'src\\pyi_pythonlib.c.2.o', 'src\\pyi_splash.c.2.o', 'src\\pyi_splashlib.c.2.o', 'src\\pyi_utils.c.2.o', 'src\\pyi_win32_utils.c.2.o', '-oC:\\pyinstaller\\bootloader\\build\\debug\\run_d.exe', '-Wl,-Bstatic', '-L.', '-lstatic_zlib.lib', '-Wl,-Bdynamic', '-luser32', '-lcomctl32', '-lkernel32', '-ladvapi32', '-lgdi32']
lld-link: warning: ignoring unknown argument '--enable-auto-import'
lld-link: warning: ignoring unknown argument '-Bstatic'
lld-link: warning: ignoring unknown argument '-Bdynamic'
lld-link: error: could not open 'static_zlib.lib': no such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Waf: Leaving directory `C:\pyinstaller\bootloader\build\debug'
Build failed
 -> task in 'run_d' failed with exit status 1:
        {task 2078845916720: cprogram main.c.3.o,pyi_apple_events.c.2.o,pyi_archive.c.2.o,pyi_exception_dialog.c.2.o,pyi_global.c.2.o,pyi_launch.c.2.o,pyi_main.c.2.o,pyi_path.c.2.o,pyi_python.c.2.o,pyi_pythonlib.c.2.o,pyi_splash.c.2.o,pyi_splashlib.c.2.o,pyi_utils.c.2.o,pyi_win32_utils.c.2.o -> run_d.exe}
['C:\\Program Files\\LLVM\\bin\\clang.exe', '-Wl,--enable-auto-import', '-m64', '-municode', 'src\\main.c.3.o', 'src\\pyi_apple_events.c.2.o', 'src\\pyi_archive.c.2.o', 'src\\pyi_exception_dialog.c.2.o', 'src\\pyi_global.c.2.o', 'src\\pyi_launch.c.2.o', 'src\\pyi_main.c.2.o', 'src\\pyi_path.c.2.o', 'src\\pyi_python.c.2.o', 'src\\pyi_pythonlib.c.2.o', 'src\\pyi_splash.c.2.o', 'src\\pyi_splashlib.c.2.o', 'src\\pyi_utils.c.2.o', 'src\\pyi_win32_utils.c.2.o', '-oC:\\pyinstaller\\bootloader\\build\\debug\\run_d.exe', '-Wl,-Bstatic', '-L.', '-lstatic_zlib.lib', '-Wl,-Bdynamic', '-luser32', '-lcomctl32', '-lkernel32', '-ladvapi32', '-lgdi32']

brenainn@WINDOWS-KNDK3G4  /c/pyinstaller/bootloader (develop)
$ /bin/find | grep static_zlib
./build/debug/libstatic_zlib.lib
```

That one was solvable quite easily by telling waf not to build a library archive for zlib - rather just use the object files directly during linking. The next pain in the backside is that there is no arm equivalent to the `-m64`/`-m32` arch flag.
This would be so much simpler if that waf script wasn't littered with cross compilation.
Turns out that it'll happily swallow the `-m64` flag. I had expected there to be a `-marm64` option which is what I had in my head as gcc and clang having on Linux but I'm not sure where I got that idea because a quick check shows that they don't.
Got something working now. Test suite is about 3rd of the way through and looking encouraging.

Resolving patch:
diff --git a/PyInstaller/_shared_with_waf.py b/PyInstaller/_shared_with_waf.py
--- a/PyInstaller/_shared_with_waf.py
+++ b/PyInstaller/_shared_with_waf.py
@@ -45,10 +45,14 @@ def _pyi_machine(machine, system):
         # This explicitly inhibits cross compiling the bootloader for or on SunWay and LoongArch machine.
         return platform.machine()
 
+    if system == "Windows":
+        if machine.lower().startswith("arm"):
+            return "arm"
+        else:
+            return "intel"
+
     if system != "Linux":
         # No architecture specifier for anything par Linux.
-        # - Windows only has one 32 and one 64 bit architecture, but lots of aliases for each so it is both pointless
-        #   and painful to give Windows an architecture specifier.
         # - macOS is on two 64 bit architectures, but they are merged into one "universal2" bootloader.
         # - BSD supports a wide range of architectures, but according to PyPI's download statistics, every one of our
         #   BSD users are on x86_64. This may change in the distant future.
diff --git a/bootloader/waflib/Tools/msvc.py b/bootloader/waflib/Tools/msvc.py
--- a/bootloader/waflib/Tools/msvc.py
+++ b/bootloader/waflib/Tools/msvc.py
@@ -28,9 +28,9 @@
 version vfw32 wbemuuid  webpost wiaguid wininet winmm winscard winspool winstrm
 wintrust wldap32 wmiutils wow32 ws2_32 wsnmp32 wsock32 wst wtsapi32 xaswitch xolehlp
 '''.split()
-all_msvc_platforms = [('x64', 'amd64'), ('x86', 'x86'), ('ia64', 'ia64'), ('x86_amd64', 'amd64'), ('x86_ia64', 'ia64'),
-                      ('x86_arm', 'arm'), ('x86_arm64', 'arm64'), ('amd64_x86', 'x86'), ('amd64_arm', 'arm'),
-                      ('amd64_arm64', 'arm64')]
+all_msvc_platforms = [('arm64', 'arm64'), ('x64', 'amd64'), ('x86', 'x86'), ('ia64', 'ia64'), ('x86_amd64', 'amd64'),
+                      ('x86_ia64', 'ia64'), ('x86_arm', 'arm'), ('x86_arm64', 'arm64'), ('amd64_x86', 'x86'),
+                      ('amd64_arm', 'arm'), ('amd64_arm64', 'arm64')]
 all_wince_platforms = [('armv4', 'arm'), ('armv4i', 'arm'), ('mipsii', 'mips'), ('mipsii_fp', 'mips'),
                        ('mipsiv', 'mips'), ('mipsiv_fp', 'mips'), ('sh4', 'sh'), ('x86', 'cex86')]
 all_icl_platforms = [('intel64', 'amd64'), ('em64t', 'amd64'), ('ia32', 'x86'), ('Itanium', 'ia64')]
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -145,8 +145,9 @@ def has_bootloaders(cls):
 # Map PyInstaller platform names to their setuptools counterparts. Other OSs can be added as and when we start shipping
 # wheels for them.
 PLATFORMS = {
-    "Windows-64bit": "win_amd64",
-    "Windows-32bit": "win32",
+    "Windows-64bit-intel": "win_amd64",
+    "Windows-32bit-intel": "win32",
+    "Windows-64bit-arm": "win_arm64",
     # The manylinux version tag depends on the glibc version compiled against. If we ever change the docker image used
     # to build the bootloaders, we must check/update this tag. These are the only architectures currently supported
     # by manylinux. Other platforms must use the generic bdist_wheel command, which will produce a wheel that is not
@@ -213,7 +214,8 @@ def finalize_options(self):
 wheel_commands["wheel_darwin_64bit"] = bdist_macos
 
 wheel_commands["wheel_darwin_64bit"].ICON_TYPES = ["icns"]
-wheel_commands["wheel_windows_32bit"].ICON_TYPES = wheel_commands["wheel_windows_64bit"].ICON_TYPES = ["ico"]
+for name in ["wheel_windows_64bit_intel", "wheel_windows_32bit_intel", "wheel_windows_64bit_arm"]:
+    wheel_commands[name].ICON_TYPES = ["ico"]
 
 
 class bdist_wheels(Command):


Test patch:
diff --git a/tests/unit/test_compat.py b/tests/unit/test_compat.py
--- a/tests/unit/test_compat.py
+++ b/tests/unit/test_compat.py
@@ -52,7 +52,11 @@ def test_linux_machine(input, output):
     assert _pyi_machine(input, "Linux") == output
 
 
-def test_non_linux_machine():
+def test_windows_machine():
+    assert _pyi_machine("AMD64", "Windows") == "intel"
+    assert _pyi_machine("ARM64", "Windows") == "arm"
+
+
+def test_other_os_machine():
     assert _pyi_machine("foo", "Darwin") is None
-    assert _pyi_machine("foo", "Windows") is None
     assert _pyi_machine("foo", "FreeBSD") is None