Repository: nvaccess/nvda

Issue:
Update setup script to be compatible with Py2exe for Python 3.7
Follow up of https://github.com/nvaccess/nvda/issues/7105#issuecomment-395424373

## Background
For years, NVDA has used Py2exe to package Python code into something that is executable on a system that doesn't have Python installed. For Python 2.7, we have been at Py2exe 0.6.9 (introduced in 2008) for a long time. For Python 3, a new version has been introduced in 2014 which looks like a rewrite. The last official release (0.9.2.2) was introduced in 2014.

Starting with Python 3.6 (end 2016) however, Python contains [a backwards incompatible change of the byte code format](https://docs.python.org/3/whatsnew/3.6.html#cpython-bytecode-changes) that is not supported by the official py2exe.

Note that many parts of the build process of NVDA are handled by SCons. This includes:

* Compiling, linking and code signing all the additional libraries, such as NVDAHelper, liblouis, minhook, ESPeak
* Compiling ESpeak dictionaries, CLDR data and gettext translations
* Converting documentation to HTML using TXT2Tags, see #8734

In summary, Scons deals with everything that is required to run NVDA from source, Py2exe is not involved in that process. Py2exe gets called as soon as a distribution of NVDA is created. The portable distribution that Py2exe creates is then packed into a launcher using NSiS.

## Requirements
Before deciding on what packaging tool to use, this section lists the requirements for a packaging tool, based on the work that is now performed by Py2exe.

1. Byte compile every source file into a *.pyc or *.pyo file. Note that for Python 3, only the *pyc files remain.
2. Byte compile every Python dependency (comtypes, pyserial, etc.) required to run NVDA.
3. Collect Python extension modules (*.pyd files)
4. Bundle the several byte compiled files into a library.zip file in the main folder of the distribution.
5. Exclude several unneeded modules
6. Create executables from certain source files:

	source | destination(s) | description
	-----|-----|-----
	nvda.pyw | nvda_noUIAccess.exe, nvda_uiAccess.exe | Main executables (see below)
	nvda_eoaProxy.pyw | nvda_eoaProxy.exe | Used to integrate in Windows 7 EOA center
	nvda_slave.pyw | nvda_slave.exe | Used for add-on installation from shell, elevated rights during installation, etc.

7. Bundle a manifest with the executables. Theoretically, this can also be done using SCons, though in contrast to what I thought earlier, there is no code within the current scons structure of NVDA that can be reused for this.
8. Make sure that the UIAccess flag is added to the manifest of nvda_uiAccess.exe and nvda_eoaProxy.exe. Strictly spoken, this is code that has been added to our subclass of build_exe.py2exe in order to inject the UIAccess flag into the manifest created by py2exe. However, droppign py2exe in favour of another tool might force us to completely rewrite this logic.
9. Embed the NVDA logo into the executables as icon.
10. Set several version info flags in the version info resource of the executables.
11. Collect several system dll's not available on every system. This includes visual C++ redistributables.
12. Collect data files created using SCons, such as several libraries, language files, documentation, etc.

## Alternatives
According to research and ideas from others, there are four packaging/freezing options that can be used with Python 3. The following sections provide short descriptions along with pros and cons.

### Py2exe fork by @albertosottile
See https://github.com/albertosottile/py2exe

Thankfully, @albertosottile contacted us in https://github.com/nvaccess/nvda/issues/8375#issuecomment-476612491, commenting that he picked up Py2exe where it has been abandoned in 2014. This resulted into a [release of Py2exe 0.93.0](https://github.com/albertosottile/py2exe/releases/tag/v0.9.3.0). This version is said to work with Python 3.6 and 3.7.

#### Pros
* Stick to the same tool as used before. Though heavily changed under the hood, it still promises to be mostly compatible with older setup files. According to the [original readme](https://github.com/albertosottile/py2exe/blob/master/README_ORIGINAL.rst):
    > It is planned to achive full compatibility with the setup-scripts for Python 2; however this is probably not yet the case.
* Probably the least efford if @albertosottile's fork proves to be working.

#### Cons
* A look at the code of Py2exe for Python 3 seems to reveal that the support to create manifests is not yet implemented, that is, there is lots of commented out code that seems to origin from the Python 2 version. Having said that, the logic to embed resource files in the executable still seems to be available and an example setup files shows an alternative method to embed the manifest. It's just not compatible with the current approach in our setup.py.
* @albertosottile's fork of this project is pretty new. Some details:
	+ It is [not yet listed on PyPI](https://pypi.org/project/py2exe/). Note the version number
	+ It has only 3 watchers, 5 stars and 2 forks on github
	+ It has [an open issue](https://github.com/albertosottile/py2exe/issues/7) regarding wxPython support.

#### Requirements scheme
Feature | supported
---|---
Create multiple executables | Yes
Bundle manifest | Yes, but differently than before, might require pr to upstream
UIAccess | No explicit support, see manifest
Embed logo | Yes
Set version info | Yes
Collect system dll's | Yes
Collect external Python dependencies and *.pyd files | Yes
Collect data files | Yes
Library file | Yes

### [PyInstaller](https://www.pyinstaller.org/)
PyInstaller is an alternative tool to build executables. It also supports byte code encryption, though that seems not very useful in the case of NVDA.

#### Pros
* This tool supports most functionality we need out of the box, including stuff that's now handled by scons, such as:
	+ Code signing
	+ Full automatic support for CRTs
	+ Manifests with UAC
* It is very actively maintained.

#### Cons
* Building multiple executables at once seems to be broken
* It does not integrate very well with distutils. It looks like the PyInstaller documentation suggests using a batch file to write down command line commands. Alternatively, you can use a spec file. Writing a setup.py for your package seems unsupported. Having said that, we're now calling a python interpretter from scons to execute the setup script, so basically, we're using a command line call already. This is therefore no show stopper, just a change of approach.
* More importantly, I've seen ridiculous tracebacks with one of my test applications referring to pyinstaller:

	```
	Traceback (most recent call last):
	  File "main.py", line 1, in <module>
	  File "c:\python36\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 631, in 		exec_module
	    exec(bytecode, module.__dict__)
	  File "core.py", line 5, in <module>
	  File "c:\python36\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 631, in exec_module
	    exec(bytecode, module.__dict__)
	```

	It looks like every import is somehow redirected to pyInstaller. I'm pretty sure this will cause issues with our use of comtypes com interface modules, globalPlugins, etc. which rely on appending paths to the module __path__ variable. Showing raw paths to the python directory also looks a bit cheap, but I recall that with py2exe, we also had to work around this.

#### 	 Requirements scheme
[Here is the official features page from PyInstaller](https://www.pyinstaller.org/features.html)

Feature | supported
---|---
Create multiple executables | No, broken in v3.0, see pyinstaller/pyinstaller#1527
Bundle manifest | Yes
UIAccess | Yes
Embed logo | Yes
Set version info | Yes
Collect system dll's | Yes
Collect external Python dependencies and *.pyd files | Yes
Collect data files | Yes
Library file | No, seems to require a custom implementation using scripting according to features section, but no information found in documentation so far

### [cx_Freeze](https://anthony-tuininga.github.io/cx_Freeze/)
This is an alternative that integrates well with distutils. Note that it is far from compatible with py2exe, so setup.py would still require an overhaul. Running cx_Freeze also results into a package structure that is very similar to the one py2exe creates, including a librar .zip file in the lib sub directory. The default settings do not create a zip file, but that can be customized easily.

cx_Freeze is [hosted on GitHub](https://github.com/anthony-tuininga/cx_Freeze.git). We can include it as a separate submodule. Though it requires some c code to be compiled, that shouldn't be that much of a problem and takes less than 20 seconds, so that doesn't justify an inclusion in misc deps in my opinion.

#### Pros
* We're using it for a @babbagecom Python project including wx and comtypes, and it works pretty well for our aims. So, I do have experience with it.
* It can replace paths shown in tracebacks
* Can include MSVC redistributable automatically
* Can include additional files in the library zip file

#### Cons
* No official build supporting Python 3.7, we'd have to build from source.
* There is no built-in support for manifests and setting the UI Access privilege level. We will probably have to implement this manually, either by expanding setup.py or by building the base executables ourselves.
* By default, cx_freeze assumes that all python stuff is in a lib sub directory (i.e. in lib/library.zip) rather than in the root of the folder where the executables live. This is not easily customisable and seems to be hard-coded behaviour. We're using the lib folder for our x86 libraries, storing the python library in there might be a bit ugly.
* When there is a startup error, cx_Freeze shows an error message box including the name of cx_Freeze itself. This might be confusing for users.
* The last commit for the project dates from september 2018. There are 260 open issues, including two from myself:
	+ https://github.com/anthony-tuininga/cx_Freeze/issues/385
	+ https://github.com/anthony-tuininga/cx_Freeze/issues/413

	Both issues haven't gotten any attention.

#### 	 Requirements scheme
Feature | supported
---|---
Create multiple executables | Yes
Bundle manifest | No, should probably be embedded in base executables
UIAccess | No
Embed logo | Yes
Set version info | Yes
Collect system dll's | Yes
Collect external Python dependencies and *.pyd files | Yes
Collect data files | Yes
Library file | Yes

###  [Nuitka](http://nuitka.net/)
Suggestion by @ctoth in https://github.com/nvaccess/nvda/issues/8375#issuecomment-410810518

Nuitka chooses a completely different approach. Instead of compiling the Python code to byte code and bundling a loader with it, it converts all code to c and then compiles it using VC++ and links it against the Python library. It also claims to be faster than CPython.

#### PROS
* Nuitka might improve performance.

#### Cons
* As all python code has to be converted to C++ and then compiled, creating a distribution of NVDA is going to be a time-consuming process. We might be able to provide pre-compiled obj files for static dependencies (such as wx and comtypes). This requires some investigation, to find out and is currently undocumented.
* It looks like Nuitka doesn't support changing the path of modules. I'm not even sure whether it would be possible to make Nuitka import python code bundled in add-ons. This is something to find out if we decide to explore this direction further.
* THere is a huge runtime difference between running NVDA from source and from a distribution.

#### 	 Requirements scheme
Feature | supported
---|---
Create multiple executables | Yes
Bundle manifest | No
UIAccess | No
Embed logo | Yes
Set version info | No
Collect system dll's | No
Collect external Python dependencies and *.pyd files | Unsure
Collect data files | No
Library file | No

## HO to proceed?
Based on the above outline, I propose the following order of investigation of our options:

1. Devote time to  investigate using the Py2exe fork by @albertosottile. He seems to be pretty interested in our possible efforts to freeze NVDA with his fork. It might also be the least effort on our end. However, it would help to know what his goals are with the project, and whether he's willing to maintain it in the near and far future. @albertosottile: Would you be able to elaborate on this?
2. Try to get in touch with the PyInstaller developers to point them at the main cons listed above. The most major question would be whether there would be support for addons and custom comtypes com interfaces.
3. Try to get in touch with @anthony-tuininga to ask him about whether he's intending to continue the active development of cx_Freeze.
4. Based on the outcome of 2 and 3 when 1 fails, decide between PyInstaller and cx_Freeze. In the current state, cx_Freeze has my preference, but that preference may change.

Note that option 1, 2 and 3 could be worked on in parallel. I'm tempted to consider Nuitka being out of scope for now.
Again remove Handy Tech com server from NVDA distributions
### Is your feature request related to a problem? Please describe.
Though we have a native driverfor Handy Tech braille displays, the com server used for the legacy driver is still kept for ease of use, per request of handy tech. See #8213 

### Describe the solution you'd like
Again, I'd like to consider removing the com server from NVDA as part of the Threshold/Python 3 transition. At the time #8213 was filed, there was a valid reason for it, namely that the new native driver didn't offer support for older displays. This was fixed in #8016. Automatic braille display detection also works perfectly with HT displays. I believe there are currently two reasons for which a user might prefer the old driver:

* Using the built-in braille monitor. Note that that monitor has some braille translation issues.
* Using ATC with displays that don't support activating it in hardware, such as the Modular Evolution.

With removing the COM server, I'd like to urge Handy Tech to implement basic ATC support in the native driver.

cc @FelixGruetzmacher @michaelDCurran 


Discussion:
CC @CToth, @TSpivey and others.
[**@leonardder**](https://github.com/leonardder) commented on [8 Jun 2018, 08:16 CEST](https://github.com/nvaccess/nvda/issues/8375#issue-330532522 "2018-06-08T06:16:38Z - Replied by Github Reply Comments"):
> #### [cx_Freeze](https://anthony-tuininga.github.io/cx_Freeze/)
> Compared to PyInstaller and py2exe, there are some drawbacks:
> 
> * There is nou built-in support for manifests and settings the UI Access privilege level. We will probably have to implement this manually, either by expanding setup.py or by building the base executables ourselves.
> * By default, cx_freeze assumes that all python stuff is in a lib sub directory (i.e. in lib/library.zip) rather than in the root of the folder where the executables live. This is not easily customisable and seems to be hard-coded behaviour.

I guess all this can be solved by building our own base executables, adding the manifests to these. This integrates quite well within the scons environment and might even lead to less complex code as the current code in setup.py.

There is currently no Python 3.7 support (see https://github.com/anthony-tuininga/cx_Freeze/issues/399)
Explore the possibility of using Nuitka.
It continues to improve, now has python 3.7 support, and the author is extremely extremely interested in working with people who have issues.



> On Aug 6, 2018, at 12:01 PM, Leonard de Ruijter <notifications@github.com> wrote:
> 
> @leonardder commented on 8 Jun 2018, 08:16 CEST:
> 
> cx_Freeze
> 
> Compared to PyInstaller and py2exe, there are some drawbacks:
> 
> There is nou built-in support for manifests and settings the UI Access privilege level. We will probably have to implement this manually, either by expanding setup.py or by building the base executables ourselves.
> By default, cx_freeze assumes that all python stuff is in a lib sub directory (i.e. in lib/library.zip) rather than in the root of the folder where the executables live. This is not easily customisable and seems to be hard-coded behaviour.
> I guess all this can be solved by building our own base executables, adding the manifests to these. This integrates quite well within the scons environment and might even lead to less complex code as the current code in setup.py.
> 
> There is currently no Python 3.7 support (see anthony-tuininga/cx_Freeze#399)
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.

@ctoth: Thanks for pointing this out. [Here](http://nuitka.net/doc/user-manual.html) is the user manual.

@feerrenrut and @michaeldcurran, what are your thoughts regarding this? I assume that this issue has to be settled before Python 3 builds can be provided to end users.
Py2exe seems to have been foreked and is on pypy supporting Python 3.3 
and newer:
https://pypi.org/project/py2exe/


[**@michaelDCurran**](https://github.com/michaelDCurran) commented on [7 aug. 2018 07:09 CEST](https://github.com/nvaccess/nvda/issues/8375#issuecomment-410934972 "2018-08-07T05:09:30Z - Replied by Github Reply Comments"):
> Py2exe seems to have been foreked and is on pypy supporting Python 3.3 
> and newer:
> https://pypi.org/project/py2exe/

I'm afraid that the most recent version is from 2014. Python 3.6 contained major byte code format changes and therefore won't work with this version of py2exe. This applies to the version on pypy as well as github repositories I've been able to find quickly. The py2exe sourceforge page doesn't provide more hope that this project will ever be developed further.
Ah, Fair enough.


I think we will have to try a few of these to really understand the trade-offs.

> I assume that this issue has to be settled before Python 3 builds can be provided to end users.

We could try to stick with py2exe and upgrade to python 3.3. After the dust settles from this we can look to swap to a new tool. Then upgrade to a later version of python.
Hi, I advise against this route, as Python 3.3 is no longer supported by Python Software Foundation. See https://www.python.org/dev/peps/pep-0398/. Thanks.

Thanks for pointing that out @josephsl, I'll hide this section of the conversation, since its a bit of a red herring.
[**@ctoth**](https://github.com/ctoth) commented on [6 Aug 2018, 20:41 CEST](https://github.com/nvaccess/nvda/issues/8375#issuecomment-410810518 "2018-08-06T18:41:22Z - Replied by Github Reply Comments"):
> Explore the possibility of using Nuitka.
> It continues to improve, now has python 3.7 support, and the author is extremely extremely interested in working with people who have issues.

I've played a bit with Nuitka. It is a really impressive and interesting piece of software.

A major downside is that all python modules are converted to c++ code and than recompiled. Though a major performance boost is promised, it takes a significant time to run the distribution creation process. I also got errors like this:

`fatal error C1002: compiler is out of heap space in pass 2`

It might help when I start compiling again with less concurrent jobs. But the time increase is concerning, as it will take much more time to create try builds, etc.
Hmm, integrating the creation of a custom cx_freeze base in our scons workflow isn't as easy as I thought, it causes weird string conversion warnings I don't expect, probably because I'm missing some essential requirements to have scons build stuff that should be linked against the python library.

I filed https://github.com/anthony-tuininga/cx_Freeze/issues/413
I did some additional research into pyinstaller,, but at least its default configuration shows full paths in trace backs and raises disgusting PyInstallerImportErrors. Furthermore, it seems it is either a nvda.exe with everything included, or a nvda.exe with nothing included. Last but not least, the installation creation process doesn't feel very pythonic.

It seems it is either something that is ugly but has features (pyinstaller), or something that is much more elegant but has less features than py2exe and hasn't received official releases during last year (CX_freeze). The master branch of the latter is python 3.7 compatible, though.
If I may, I have been working on resurrecting py2exe and make it compatible with the latest versions of python. If you want, you could give a try at the prebuilt wheels that I just released here: https://github.com/albertosottile/py2exe/releases/tag/v0.9.3.0 

Please, let me know if you find any errors when packaging NVDA. This is a big project and I am sure it will provide a good benchmark for these new wheels.
This is really interesting. We'll keep this in mind as soon as the Py3 
transition begins.


For everyone who is interested in this discussion, I've updated https://github.com/nvaccess/nvda/issues/8375#issue-330532522 to contain an in depth overview of our options to replace Py2exe 0.6.9.

@albertosottile: You seem to be pretty interested in our possible efforts to freeze NVDA with your fork. It might also be the least effort on our end. However, it would help to know what your goals are with the project, and whether you're willing to maintain it in the near and far future. Would you be able to elaborate on this?
I think a missing requirement is  to collect any dependent python extention dlls (.pyd files). I don't think I saw that in your list?
> I think a missing requirement is to collect any dependent python extention dlls (.pyd files). I don't think I saw that in your list?

You're correct, I'll at that.
> You seem to be pretty interested in our possible efforts to freeze NVDA with your fork. It might also be the least effort on our end. However, it would help to know what your goals are with the project, and whether you're willing to maintain it in the near and far future. Would you be able to elaborate on this?

Thanks for the tag. I read the first post carefully and it seems that there are several potential issues in using the forked `py2exe` for your project, both technical and political. I will try to address them hereafter.

First of all, I never tried manifests and to be honest I was not even aware of this feature in py2exe. As far as I know, this could be the most serious technical hurdle you (we?) will probably have to face if you want to keep your existing script file. Alternatively, I am confident the same results could be achieved using resources, but that will need a rewrite of the script.

Second, I am not worried about the wxPython issue, since applying a workaround from your side is easy and, as I wrote [here](https://github.com/albertosottile/py2exe/issues/7#issuecomment-492581491), it might not even be necessary in your case.

Thirdly, as you correctly noted, the fork is not listed on PyPI. This is also related to the small number of watchers/stars/forks. Honestly, I wanted the project to gain more traction before contacting the original maintainers and/or the PyPI team and get noticed. While it is true that the codebase seems to work now,  I strongly believe that maintaining a project like `py2exe` is not a job for one single person. (Side note: look at what is happening to PyInstaller now, they barely have the time to fix the most urgent issues, let alone implementing new features). Hence, I do agree with your worries about the future maintenance of `py2exe`, not because I am not willing, but because I am totally sure that any effort from me alone will not be enough. Now, the way I see it, the more developers adopt this new `py2exe`, the more traction the project will gain. So, it would be a great deal if NVDA could use it, thus my interest in this migration. Nevertheless, I will completely understand if you would decide to go for an alternative with wider adoption.

Wrapping things up, I think a good way to proceed for you will be to experiment with the forked `py2exe` and see how far you can go using that, without applying excessive efforts on your existing codebase. In this way, if things work out, you will have a huge return for a quite small investment. Eventually, if it is possible to iron out all the issues without writing many fixes, you could even release a new version based on this `py2exe`, helping me gaining some traction, hence increasing the chances of its survival. Otherwise, you will still have time to look into the alternatives without much to regret for having tried. 


Resolving patch:
diff --git a/source/logHandler.py b/source/logHandler.py
--- a/source/logHandler.py
+++ b/source/logHandler.py
@@ -96,17 +96,11 @@ def getCodePath(f):
 	return ".".join([x for x in path,className,funcName if x])
 
 # Function to strip the base path of our code from traceback text to improve readability.
-if getattr(sys, "frozen", None):
-	# We're running a py2exe build.
-	# The base path already seems to be stripped in this case, so do nothing.
-	def stripBasePathFromTracebackText(text):
-		return text
-else:
-	BASE_PATH = os.path.split(__file__)[0] + os.sep
-	TB_BASE_PATH_PREFIX = '  File "'
-	TB_BASE_PATH_MATCH = TB_BASE_PATH_PREFIX + BASE_PATH
-	def stripBasePathFromTracebackText(text):
-		return text.replace(TB_BASE_PATH_MATCH, TB_BASE_PATH_PREFIX)
+BASE_PATH = os.path.split(__file__)[0] + os.sep
+TB_BASE_PATH_PREFIX = '  File "'
+TB_BASE_PATH_MATCH = TB_BASE_PATH_PREFIX + BASE_PATH
+def stripBasePathFromTracebackText(text):
+	return text.replace(TB_BASE_PATH_MATCH, TB_BASE_PATH_PREFIX)
 
 class Logger(logging.Logger):
 	# Import standard levels for convenience.
diff --git a/source/setup.py b/source/setup.py
--- a/source/setup.py
+++ b/source/setup.py
@@ -8,117 +8,100 @@
 import os
 import copy
 import gettext
-gettext.install("nvda", unicode=True)
-from distutils.core import setup
+gettext.install("nvda")
+from setuptools import setup
 import py2exe as py2exeModule
 from glob import glob
 import fnmatch
 from versionInfo import *
-from py2exe import build_exe
+from py2exe import distutils_buildexe
+from py2exe.dllfinder import DllFinder
 import wx
-import imp
+import importlib.machinery
 
-MAIN_MANIFEST_EXTRA = r"""
-<file name="brailleDisplayDrivers\handyTech\HtBrailleDriverServer.dll">
-	<comClass
-		description="HtBrailleDriver Class"
-		clsid="{209445BA-92ED-4AB2-83EC-F24ACEE77EE0}"
-		threadingModel="Apartment"
-		progid="HtBrailleDriverServer.HtBrailleDriver"
-		tlbid="{33257EFB-336F-4680-B94E-F5013BA6B9B3}" />
-</file>
-<file name="brailleDisplayDrivers\handyTech\HtBrailleDriverServer.tlb">
-	<typelib tlbid="{33257EFB-336F-4680-B94E-F5013BA6B9B3}"
-		version="1.0"
-		helpdir="" />
-</file>
-<comInterfaceExternalProxyStub
-	name="IHtBrailleDriverSink"
-	iid="{EF551F82-1C7E-421F-963D-D9D03548785A}"
-	proxyStubClsid32="{00020420-0000-0000-C000-000000000046}"
-	baseInterface="{00000000-0000-0000-C000-000000000046}"
-	tlbid="{33257EFB-336F-4680-B94E-F5013BA6B9B3}" />
-<comInterfaceExternalProxyStub
-	name="IHtBrailleDriver"
-	iid="{43A71F9B-58EE-42D4-B58E-0F9FBA28D995}"
-	proxyStubClsid32="{00020424-0000-0000-C000-000000000046}"
-	baseInterface="{00000000-0000-0000-C000-000000000046}"
-	tlbid="{33257EFB-336F-4680-B94E-F5013BA6B9B3}" />
-<compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1">
-	<application>
-		<!-- Windows Vista -->
-		<supportedOS Id="{e2011457-1546-43c5-a5fe-008deee3d3f0}"/>
-		<!-- Windows 7 -->
-		<supportedOS Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}"/>
-		<!-- Windows 8 -->
-		<supportedOS Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}"/>
-		<!-- Windows 8.1 -->
-		<supportedOS Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}"/>
-		<!-- Windows 10 -->
-		<supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}"/>
-	</application> 
-</compatibility>
+RT_MANIFEST = 24
+manifest_template = """\
+<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
+<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
+	<trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
+		<security>
+			<requestedPrivileges>
+				<requestedExecutionLevel
+					level="asInvoker"
+					uiAccess="%(uiAccess)s"
+				/>
+			</requestedPrivileges>
+		</security>
+	</trustInfo>
+	<compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1">
+		<application>
+			<!-- Windows 7 -->
+			<supportedOS
+				Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}"
+			/>
+			<!-- Windows 8 -->
+			<supportedOS
+				Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}"
+			/>
+			<!-- Windows 8.1 -->
+			<supportedOS
+				Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}"
+			/>
+			<!-- Windows 10 -->
+			<supportedOS
+				Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}"
+			/>
+		</application> 
+	</compatibility>
+</assembly>
 """
 
-def getModuleExtention(thisModType):
-	for ext,mode,modType in imp.get_suffixes():
-		if modType==thisModType:
-			return ext
-	raise ValueError("unknown mod type %s"%thisModType)
-
 # py2exe's idea of whether a dll is a system dll appears to be wrong sometimes, so monkey patch it.
-origIsSystemDLL = build_exe.isSystemDLL
-def isSystemDLL(pathname):
-	dll = os.path.basename(pathname).lower()
-	if dll in ("msvcp71.dll", "msvcp90.dll", "gdiplus.dll","mfc71.dll", "mfc90.dll"):
-		# These dlls don't exist on many systems, so make sure they're included.
-		return 0
-	elif dll.startswith("api-ms-win-") or dll in ("powrprof.dll", "mpr.dll", "crypt32.dll"):
+orig_determine_dll_type = DllFinder.determine_dll_type
+def determine_dll_type(self, imagename):
+	dll = os.path.basename(imagename).lower()
+	if dll.startswith("api-ms-win-") or dll in ("powrprof.dll", "mpr.dll", "crypt32.dll"):
 		# These are definitely system dlls available on all systems and must be excluded.
 		# Including them can cause serious problems when a binary build is run on a different version of Windows.
-		return 1
-	return origIsSystemDLL(pathname)
-build_exe.isSystemDLL = isSystemDLL
+		return None
+	return orig_determine_dll_type(self, imagename)
+DllFinder.determine_dll_type = determine_dll_type
 
-class py2exe(build_exe.py2exe):
+class py2exe(distutils_buildexe.py2exe):
 	"""Overridden py2exe command to:
-		* Add a command line option --enable-uiAccess to enable uiAccess for the main executable
-		* Add extra info to the manifest
-		* Don't copy w9xpopen, as NVDA will never run on Win9x
+		* Add a command line option --enable-uiAccess to enable uiAccess for the main executable and EOA proxy
+		* Add a manifest to the executables
 	"""
 
-	user_options = build_exe.py2exe.user_options + [
+	user_options = distutils_buildexe.py2exe.user_options + [
 		("enable-uiAccess", "u", "enable uiAccess for the main executable"),
 	]
 
 	def initialize_options(self):
-		build_exe.py2exe.initialize_options(self)
+		super(py2exe, self).initialize_options()
 		self.enable_uiAccess = False
 
-	def copy_w9xpopen(self, modules, dlls):
-		pass
-
 	def run(self):
 		dist = self.distribution
 		if self.enable_uiAccess:
 			# Add a target for nvda_uiAccess, using nvda_noUIAccess as a base.
 			target = copy.deepcopy(dist.windows[0])
 			target["dest_base"] = "nvda_uiAccess"
-			target["uac_info"] = (target["uac_info"][0], True)
+			target['uiAccess'] = True
 			dist.windows.insert(1, target)
 			# nvda_eoaProxy should have uiAccess.
 			target = dist.windows[3]
-			target["uac_info"] = (target["uac_info"][0], True)
-
-		build_exe.py2exe.run(self)
-
-	def build_manifest(self, target, template):
-		mfest, rid = build_exe.py2exe.build_manifest(self, target, template)
-		if getattr(target, "script", "").endswith(".pyw"):
-			# This is one of the main application executables.
-			mfest = mfest[:mfest.rindex("</assembly>")]
-			mfest += MAIN_MANIFEST_EXTRA + "</assembly>"
-		return mfest, rid
+			target['uiAccess'] = True
+		# Add a manifest resource to every target at runtime.
+		for target in dist.windows:
+			target["other_resources"] = [
+				(
+					RT_MANIFEST,
+					1,
+					(manifest_template % dict(uiAccess=target['uiAccess'])).encode("utf-8")
+				),
+			]
+		super(py2exe, self).run()
 
 def getLocaleDataFiles():
 	wxDir=wx.__path__[0]
@@ -146,8 +129,6 @@ def getRecursiveDataFiles(dest,source,excludes=()):
 	[rulesList.extend(getRecursiveDataFiles(os.path.join(dest,dirName),os.path.join(source,dirName),excludes=excludes)) for dirName in os.listdir(source) if os.path.isdir(os.path.join(source,dirName)) and not dirName.startswith('.')]
 	return rulesList
 
-compiledModExtention = getModuleExtention(imp.PY_COMPILED)
-sourceModExtention = getModuleExtention(imp.PY_SOURCE)
 setup(
 	name = name,
 	version=version,
@@ -169,10 +150,12 @@ def getRecursiveDataFiles(dest,source,excludes=()):
 		{
 			"script":"nvda.pyw",
 			"dest_base":"nvda_noUIAccess",
-			"uac_info": ("asInvoker", False),
+			"uiAccess": False,
 			"icon_resources":[(1,"images/nvda.ico")],
+			"other_resources": [], # Populated at run time
 			"version":formatBuildVersionString(),
 			"description":"NVDA application",
+			"product_name":name,
 			"product_version":version,
 			"copyright":copyright,
 			"company_name":publisher,
@@ -180,9 +163,12 @@ def getRecursiveDataFiles(dest,source,excludes=()):
 		# The nvda_uiAccess target will be added at runtime if required.
 		{
 			"script": "nvda_slave.pyw",
+			"uiAccess": False,
 			"icon_resources": [(1,"images/nvda.ico")],
+			"other_resources": [], # Populated at run time
 			"version":formatBuildVersionString(),
 			"description": name,
+			"product_name":name,
 			"product_version": version,
 			"copyright": copyright,
 			"company_name": publisher,
@@ -190,10 +176,12 @@ def getRecursiveDataFiles(dest,source,excludes=()):
 		{
 			"script": "nvda_eoaProxy.pyw",
 			# uiAccess will be enabled at runtime if appropriate.
-			"uac_info": ("asInvoker", False),
+			"uiAccess": False,
 			"icon_resources": [(1,"images/nvda.ico")],
+			"other_resources": [], # Populated at run time
 			"version":formatBuildVersionString(),
 			"description": "NVDA Ease of Access proxy",
+			"product_name":name,
 			"product_version": version,
 			"copyright": copyright,
 			"company_name": publisher,
@@ -201,7 +189,7 @@ def getRecursiveDataFiles(dest,source,excludes=()):
 	],
 	options = {"py2exe": {
 		"bundle_files": 3,
-		"excludes": ["Tkinter",
+		"excludes": ["tkinter",
 			"serial.loopback_connection", "serial.rfc2217", "serial.serialcli", "serial.serialjava", "serial.serialposix", "serial.socket_connection"],
 		"packages": ["NVDAObjects","virtualBuffers","appModules","comInterfaces","brailleDisplayDrivers","synthDrivers"],
 		"includes": [
@@ -224,8 +212,22 @@ def getRecursiveDataFiles(dest,source,excludes=()):
 	] + (
 		getLocaleDataFiles()
 		+ getRecursiveDataFiles("synthDrivers", "synthDrivers",
-			excludes=("*%s" % sourceModExtention, "*%s" % compiledModExtention, "*.exp", "*.lib", "*.pdb"))
-		+ getRecursiveDataFiles("brailleDisplayDrivers", "brailleDisplayDrivers", excludes=("*%s"%sourceModExtention,"*%s"%compiledModExtention))
+			excludes=tuple(
+				"*%s" % ext
+				for ext in importlib.machinery.SOURCE_SUFFIXES + importlib.machinery.BYTECODE_SUFFIXES
+			) + (
+				"*.exp",
+				"*.lib",
+				"*.pdb",
+				"__pycache__"
+		))
+		+ getRecursiveDataFiles("brailleDisplayDrivers", "brailleDisplayDrivers",
+			excludes=tuple(
+				"*%s" % ext
+				for ext in importlib.machinery.SOURCE_SUFFIXES + importlib.machinery.BYTECODE_SUFFIXES
+			) + (
+				"__pycache__",
+		))
 		+ getRecursiveDataFiles('documentation', '../user_docs', excludes=('*.t2t', '*.t2tconf', '*/developerGuide.*'))
 	),
 )


Test patch: