Repository: SCons/scons

Issue:
Visual Studio Express 2017 not detected due to HOST_ARCH default (most likely)
# Visual Studio Express 2017 not detected

I discussed the issue with @bdbaddog on the #scons-help channel on discord and he asked me to add a bug report here.

## Table of Contents

* [Problem statement](#problem)
* [Environment](#environment)
* [Analysis](#analysis)
* [Workaround](#workaround)
* [Conclusion](#conclusion)
* [Reproducing the bug](#reproducing)

<a name="problem"></a>
## Problem statement

Consider the following SConstruct

~~~~python
Program('hello', 'hello.c')
~~~~

And the classical hello, world program

~~~~C
# include <stdio.h>

int main(int ac, char *av[]) {
    printf("Hello, world\n");
    return 0;
}
~~~~

When this is run, I get the following output:

~~~~text
(scons) c:\Users\ganss\appdata\local\Temp\scons>scons
scons: Reading SConscript files ...

scons: warning: No version of Visual Studio compiler found - C/C++ compilers most likely not set correctly
File "C:\Users\ganss\appdata\local\Temp\scons\SConstruct", line 1, in <module>
scons: done reading SConscript files.
scons: Building targets ...
cl /Fohello.obj /c hello.c /nologo
Der Befehl "cl" ist entweder falsch geschrieben oder
konnte nicht gefunden werden.
scons: *** [hello.obj] Error 1
scons: building terminated because of errors.

(scons) c:\Users\ganss\appdata\local\Temp\scons>
~~~~

<a name="environment"></a>
## Environment

~~~~text
(scons) c:\Users\ganss\appdata\local\Temp\scons>

(scons) c:\Users\ganss\appdata\local\Temp\scons>ver

Microsoft Windows [Version 10.0.18363.592]

(scons) c:\Users\ganss\appdata\local\Temp\scons>python --version
Python 3.8.1

(scons) c:\Users\ganss\appdata\local\Temp\scons>scons --version
SCons by Steven Knight et al.:
        script: v3.1.2.bee7caf9defd6e108fc2998a2520ddb36a967691, 2019-12-17 02:07:09, by bdeegan on octodog
        engine: v3.1.2.bee7caf9defd6e108fc2998a2520ddb36a967691, 2019-12-17 02:07:09, by bdeegan on octodog
        engine path: ['c:\\users\\ganss\\appdata\\local\\temp\\scons\\lib\\site-packages\\scons\\SCons']
Copyright (c) 2001 - 2019 The SCons Foundation
~~~~

The machine has a Visual Studio Express 2017 installed and no other versions of visual studio

~~~~text
(scons) c:\Users\ganss\appdata\local\Temp\scons>"c:\Program Files (x86)\Microsoft Visual Studio\Installer"\vswhere -products "*"
Visual Studio Locator version 2.5.2+gebb9f26a3d [query version 1.18.21.37008]
Copyright (C) Microsoft Corporation. All rights reserved.

instanceId: 4e83d1f0
installDate: 27.12.2019 23:46:12
installationName: VisualStudio/15.9.18+28307.960
installationPath: C:\Program Files (x86)\Microsoft Visual Studio\2017\WDExpress
installationVersion: 15.9.28307.960
productId: Microsoft.VisualStudio.Product.WDExpress
productPath: C:\Program Files (x86)\Microsoft Visual Studio\2017\WDExpress\Common7\IDE\WDExpress.exe
isPrerelease: 0
displayName: Visual Studio 2017 für Desktop Express
description: Eine IDE, die die Entwicklung von Desktopanwendungen ermöglicht. Testen Sie Visual Studio Community als erweiterbare Alternative mit vollständigem Funktionsumfang.
channelId: VisualStudio.15.Release
channelUri: https://aka.ms/vs/15/release/channel
enginePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\resources\app\ServiceHub\Services\Microsoft.VisualStudio.Setup.Service
releaseNotes: https://go.microsoft.com/fwlink/?LinkId=660692#15.9.18
thirdPartyNotices: https://go.microsoft.com/fwlink/?LinkId=660708
updateDate: 2019-12-27T22:46:12.4666321Z
catalog_buildBranch: d15.9
catalog_buildVersion: 15.9.28307.960
catalog_id: VisualStudio/15.9.18+28307.960
catalog_localBuild: build-lab
catalog_manifestName: VisualStudio
catalog_manifestType: installer
catalog_productDisplayVersion: 15.9.18
catalog_productLine: Dev15
catalog_productLineVersion: 2017
catalog_productMilestone: RTW
catalog_productMilestoneIsPreRelease: False
catalog_productName: Visual Studio
catalog_productPatchVersion: 18
catalog_productPreReleaseMilestoneSuffix: 1.0
catalog_productRelease: RTW
catalog_productSemanticVersion: 15.9.18+28307.960
catalog_requiredEngineVersion: 1.18.1049.33485
properties_campaignId:
properties_channelManifestId: VisualStudio.15.Release/15.9.18+28307.960
properties_nickname:
properties_setupEngineFilePath: C:\Program Files (x86)\Microsoft Visual Studio\Installer\vs_installershell.exe
~~~~

<a name="analysis"></a>
## Analysis

The issue obviously got something to do with tool detection. On @bdbaddog's proposal I let the following three scenarios run:

1. SConstruct is only a single statement: `DefaultEnvironment()`
   The environment variable _SCONS_MSCOMMON_DEBUG_ is set to _mstrace.log_.
   The resulting logfile is available as https://pastebin.com/e4kqhPDU

2. SConstruct is only a single statement: `DefaultEnvironment(MSVC_VERSION="14.1", TARGET_ARCH='x86')`
   The environment variable _SCONS_MSCOMMON_DEBUG_ is set to _mstrace-without-host_arch.log_.
   The resulting logfile is available as https://pastebin.com/eW56Pnud

3. SConstruct is only a single statement: `DefaultEnvironment(MSVC_VERSION="14.1", TARGET_ARCH='x86', HOST_ARCH='x86')`
   The environment variable _SCONS_MSCOMMON_DEBUG_ is set to _mstrace-without-host_arch.log_.
   The resulting logfile is available as https://pastebin.com/SZCE8Hp9

<a name="workaround"></a>
## Workaround
Setting the _HOST_ARCH_ variable to _x86_ makes the detection work, this SConstruct file
works on the machine:

~~~~python
DefaultEnvironment(HOST_ARCH='x86')
Program('hello', 'hello.c')
~~~~

<a name="conclusion"></a>
## Conclusion

As the visual studio express versions come without the AMD64 tools, the detection logic should not only consider the visual studio directories mandated by the detected host architecture but additionally should consider at least the _x86_ directories.

Not sure about _arm_ here - I don't even know if there are visual studio versions out that are running on more than one arm architecture.

<a name="reproducing"></a>
## Reproducing the bug

see [Problem statement](#problem)
SCons should use %ProgramFiles(x86)% to locate base dir for vswhere location
When the user has MSVS installed on any non C: drive SCons fails to find vswhere as the path we use to locate it is hardcoded to the C: drive.

According to:
https://github.com/microsoft/vswhere/wiki/Installing
and
https://devblogs.microsoft.com/setup/vswhere-is-now-installed-with-visual-studio-2017/

We should be using 

> vswhere.exe is installed in %ProgramFiles(x86)%\Microsoft Visual Studio\Installer (use %ProgramFiles% in a 32-bit program prior to Windows 10). 


This should fix GoDot Issue
https://github.com/godotengine/godot/issues/35446
SCons should use %ProgramFiles(x86)% to locate base dir for vswhere location
When the user has MSVS installed on any non C: drive SCons fails to find vswhere as the path we use to locate it is hardcoded to the C: drive.

According to:
https://github.com/microsoft/vswhere/wiki/Installing
and
https://devblogs.microsoft.com/setup/vswhere-is-now-installed-with-visual-studio-2017/

We should be using 

> vswhere.exe is installed in %ProgramFiles(x86)%\Microsoft Visual Studio\Installer (use %ProgramFiles% in a 32-bit program prior to Windows 10). 


This should fix GoDot Issue
https://github.com/godotengine/godot/issues/35446


Discussion:
Here are the logs mentioned as attachments
[mstrace.log](https://github.com/SCons/scons/files/4104643/mstrace.log)
[mstrace-with-host-arch.log](https://github.com/SCons/scons/files/4104644/mstrace-with-host-arch.log)
[mstrace-without-host-arch.log](https://github.com/SCons/scons/files/4104645/mstrace-without-host-arch.log)



See: https://visualstudio.microsoft.com/vs/express/ to download VS2017 express.
Yeah, this analsys looks basically correct, and I can reproduce the behavior.  We see multiple snips like this in the msvc trace file:
```
00076ms:pid19904:MSCommon/vc.py:trying to find VC 14.1
00138ms:pid19904:MSCommon/vc.py:find_vc_pdir_vswhere(): VC found: '14.1'
00138ms:pid19904:MSCommon/vc.py:found VC 14.1
00138ms:pid19904:MSCommon/vc.py:get_host_target()
00138ms:pid19904:MSCommon/vc.py:get_host_target() req_target_platform:None
00138ms:pid19904:MSCommon/vc.py:_check_cl_exists_in_vc_dir(): host platform amd64, target platform amd64 for version 14.1
00139ms:pid19904:MSCommon/vc.py:_check_cl_exists_in_vc_dir(): checking for cl.exe at C:\Program Files (x86)\Microsoft Visual Studio\2017\WDExpress\VC\Tools\MSVC\14.16.27023\bin\Hostx64\x64\cl.exe
00140ms:pid19904:MSCommon/vc.py:find_vc_pdir no compiler found 14.1
```
So - found it via `vswhere`, but then checking for the actual compiler, and it isn't there.  Note the "`checking for cl.exe`" message above.  On my system we do find `cl.exe`, and these are the paths found when starting from `c:/Program Files (x86)/Microsoft Visual Studio`"
```
./2017/WDExpress/VC/Tools/MSVC/14.16.27023/bin/Hostx86/arm/cl.exe
./2017/WDExpress/VC/Tools/MSVC/14.16.27023/bin/Hostx86/arm64/cl.exe
./2017/WDExpress/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe
./2017/WDExpress/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x86/cl.exe
```
Oh, this is cute.  You can update the check in `_check_cl_exists_in_vc_dir`, and get it to pass:
```
00125ms:pid04196:MSCommon/vc.py:_check_cl_exists_in_vc_dir(): host platform amd64, target platform amd64 for version 14.1
00125ms:pid04196:MSCommon/vc.py:_check_cl_exists_in_vc_dir(): checking for cl.exe at C:\Program Files (x86)\Microsoft Visual Studio\2017\WDExpress\VC\Tools\MSVC\14.16.27023\bin\Hostx64\x64\cl.exe
00125ms:pid04196:MSCommon/vc.py:_check_cl_exists_in_vc_dir(): checking for cl.exe at C:\Program Files (x86)\Microsoft Visual Studio\2017\WDExpress\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x64\cl.exe
00125ms:pid04196:MSCommon/vc.py:_check_cl_exists_in_vc_dir(): found cl.exe!
```
and it makes no difference, because this was just a yes/no question, and there's still somewhere else that relies on the same logic, and the net result is still no compiler found.
Tracing further, everything seems okay, then at the very end of `msvc_setup_env` there is a final sanity-check call:
```
    # final check to issue a warning if the compiler is not present
    msvc_cl = find_program_path(env, 'cl')
    if not msvc_cl:
        debug("msvc_setup_env() did not find 'cl'")
        SCons.Warnings.warn(SCons.Warnings.VisualCMissingWarning,
            "Could not find MSVC compiler 'cl', it may need to be installed separately with Visual Studio")
```
And the paths set in `env` don't include one where `cl.exe` is found.  I don't get this, because it has actually found and run `vcvarsall.bat` and that's what it gives back....
Okay, one more poke: gack. `vcvarsall.bat` doesn't fail if given a dodgy combination, it just doesn't set enough of the env up.  So we have a list of things to try:
```
00203ms:pid07832:MSCommon/vc.py:msvs_find_valid_batch_script(): host_platform: amd64 try_target_archs:['amd64', 'x86_amd64', 'x86']
```
The magic target we want is `x86_amd64` - this was put in to deal with other express versions, but it's still valid for 2017. If I manually run `vcvarsall.bat x86_amd64`, it does the right stuff - `cl` is in the PATH now. But first scons tries `vcvarsall.bat amd64` which by all indications succeeds, so there's no reason to go on and try the other targets, but there's no path to the compiler in PATH. Here's running it manually - showing no signs of a problem:
```
c:\Program Files (x86)\Microsoft Visual Studio\2017\WDExpress\VC\Auxiliary\Build>vcvarsall.bat amd64
**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.9.19
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'
```
So I don't know yet how to handle this case.



Resolving patch:
diff --git a/src/engine/SCons/Tool/MSCommon/vc.py b/src/engine/SCons/Tool/MSCommon/vc.py
--- a/src/engine/SCons/Tool/MSCommon/vc.py
+++ b/src/engine/SCons/Tool/MSCommon/vc.py
@@ -41,7 +41,10 @@
 import os
 import platform
 import sys
+from contextlib import suppress
 from string import digits as string_digits
+from subprocess import PIPE
+#TODO: Python 2 cleanup
 if sys.version_info[0] == 2:
     import collections
 
@@ -95,6 +98,10 @@ class BatchFileExecutionError(VisualCException):
     "aarch64"   : "arm64",
 }
 
+# Starting with 14.1 (aka VS2017), the tools are organized by host directory.
+# subdirs for each target. They are now in .../VC/Auxuiliary/Build.
+# Note 2017 Express uses Hostx86 even if it's on 64-bit Windows,
+# not reflected in this table.
 _HOST_TARGET_TO_CL_DIR_GREATER_THAN_14 = {
     ("amd64","amd64")  : ("Hostx64","x64"),
     ("amd64","x86")    : ("Hostx64","x86"),
@@ -106,8 +113,9 @@ class BatchFileExecutionError(VisualCException):
     ("x86","arm64")    : ("Hostx86","arm64"),
 }
 
-# get path to the cl.exe dir for older VS versions
-# based off a tuple of (host, target) platforms
+# before 14.1 (VS2017): the original x86 tools are in the tools dir,
+# any others are in a subdir named by the host/target pair,
+# or just a single word if host==target
 _HOST_TARGET_TO_CL_DIR = {
     ("amd64","amd64")  : "amd64",
     ("amd64","x86")    : "amd64_x86",
@@ -117,10 +125,38 @@ class BatchFileExecutionError(VisualCException):
     ("x86","x86")      : "",
     ("x86","arm")      : "x86_arm",
     ("x86","arm64")    : "x86_arm64",
+    ("arm","arm")      : "arm",
 }
 
-# Given a (host, target) tuple, return the argument for the bat file.
-# Both host and targets should be canonalized.
+# 14.1 (VS2017) and later:
+# Given a (host, target) tuple, return the batch file to look for.
+# We can't rely on returning an arg to use for vcvarsall.bat,
+# because that script will run even if given a pair that isn't installed.
+# Targets that already look like a pair are pseudo targets that
+# effectively mean to skip whatever the host was specified as.
+_HOST_TARGET_TO_BAT_ARCH_GT14 = {
+    ("amd64", "amd64"): "vcvars64.bat",
+    ("amd64", "x86"): "vcvarsamd64_x86.bat",
+    ("amd64", "x86_amd64"): "vcvarsx86_amd64.bat",
+    ("amd64", "x86_x86"): "vcvars32.bat",
+    ("amd64", "arm"): "vcvarsamd64_arm.bat",
+    ("amd64", "x86_arm"): "vcvarsx86_arm.bat",
+    ("amd64", "arm64"): "vcvarsamd64_arm64.bat",
+    ("amd64", "x86_arm64"): "vcvarsx86_arm64.bat",
+    ("x86", "x86"): "vcvars32.bat",
+    ("x86", "amd64"): "vcvarsx86_amd64.bat",
+    ("x86", "x86_amd64"): "vcvarsx86_amd64.bat",
+    ("x86", "arm"): "vcvarsx86_arm.bat",
+    ("x86", "x86_arm"): "vcvarsx86_arm.bat",
+    ("x86", "arm64"): "vcvarsx86_arm64.bat",
+    ("x86", "x86_arm64"): "vcvarsx86_arm64.bat",
+}
+
+# before 14.1 (VS2017):
+# Given a (host, target) tuple, return the argument for the bat file;
+# Both host and target should be canoncalized.
+# If the target already looks like a pair, return it - these are
+# pseudo targets (mainly used by Express versions)
 _HOST_TARGET_ARCH_TO_BAT_ARCH = {
     ("x86", "x86"): "x86",
     ("x86", "amd64"): "x86_amd64",
@@ -128,12 +164,16 @@ class BatchFileExecutionError(VisualCException):
     ("amd64", "x86_amd64"): "x86_amd64", # This is present in (at least) VS2012 express
     ("amd64", "amd64"): "amd64",
     ("amd64", "x86"): "x86",
+    ("amd64", "x86_x86"): "x86",
     ("x86", "ia64"): "x86_ia64",         # gone since 14.0
-    ("arm", "arm"): "arm",              # since 14.0, maybe gone 14.1?
     ("x86", "arm"): "x86_arm",          # since 14.0
     ("x86", "arm64"): "x86_arm64",      # since 14.1
     ("amd64", "arm"): "amd64_arm",      # since 14.0
     ("amd64", "arm64"): "amd64_arm64",  # since 14.1
+    ("x86", "x86_arm"): "x86_arm",      # since 14.0
+    ("x86", "x86_arm64"): "x86_arm64",  # since 14.1
+    ("amd64", "x86_arm"): "x86_arm",      # since 14.0
+    ("amd64", "x86_arm64"): "x86_arm64",  # since 14.1
 }
 
 _CL_EXE_NAME = 'cl.exe'
@@ -194,7 +234,7 @@ def get_host_target(env):
 
 # If you update this, update SupportedVSList in Tool/MSCommon/vs.py, and the
 # MSVC_VERSION documentation in Tool/msvc.xml.
-_VCVER = ["14.2", "14.1", "14.0", "14.0Exp", "12.0", "12.0Exp", "11.0", "11.0Exp", "10.0", "10.0Exp", "9.0", "9.0Exp","8.0", "8.0Exp","7.1", "7.0", "6.0"]
+_VCVER = ["14.2", "14.1", "14.1Exp", "14.0", "14.0Exp", "12.0", "12.0Exp", "11.0", "11.0Exp", "10.0", "10.0Exp", "9.0", "9.0Exp","8.0", "8.0Exp","7.1", "7.0", "6.0"]
 
 # if using vswhere, a further mapping is needed
 _VCVER_TO_VSWHERE_VER = {
@@ -204,9 +244,11 @@ def get_host_target(env):
 
 _VCVER_TO_PRODUCT_DIR = {
     '14.2' : [
-        (SCons.Util.HKEY_LOCAL_MACHINE, r'')], # VS 2019 doesn't set this key
+        (SCons.Util.HKEY_LOCAL_MACHINE, r'')], # not set by this version
     '14.1' : [
-        (SCons.Util.HKEY_LOCAL_MACHINE, r'')], # VS 2017 doesn't set this key
+        (SCons.Util.HKEY_LOCAL_MACHINE, r'')], # not set by this version
+    '14.1Exp' : [
+        (SCons.Util.HKEY_LOCAL_MACHINE, r'')], # not set by this version
     '14.0' : [
         (SCons.Util.HKEY_LOCAL_MACHINE, r'Microsoft\VisualStudio\14.0\Setup\VC\ProductDir')],
     '14.0Exp' : [
@@ -299,44 +341,46 @@ def find_vc_pdir_vswhere(msvc_version):
         debug("Unknown version of MSVC: %s" % msvc_version)
         raise UnsupportedVersion("Unknown version %s" % msvc_version)
 
-    # For bug 3333 - support default location of vswhere for both 64 and 32 bit windows
-    # installs.
-    for pf in ['Program Files (x86)', 'Program Files']:
-        vswhere_path = os.path.join(
-            'C:\\',
-            pf,
-            'Microsoft Visual Studio',
-            'Installer',
-            'vswhere.exe'
-        )
+    # For bug 3333: support default location of vswhere for both
+    # 64 and 32 bit windows installs.
+    # For bug 3542: also accommodate not being on C: drive.
+    # NB: this gets called from testsuite on non-Windows platforms.
+    # Whether that makes sense or not, don't break it for those.
+    # TODO: requested to add a user-specified path to vswhere
+    #       and have this routine set the same var if it finds it.
+    pfpaths = [
+        os.path.expandvars(r"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer"),
+        os.path.expandvars(r"%ProgramFiles%\Microsoft Visual Studio\Installer"),
+        os.path.expandvars(r"%ChocolateyInstall%\bin"),
+    ]
+    for pf in pfpaths:
+        vswhere_path = os.path.join(pf, "vswhere.exe")
         if os.path.exists(vswhere_path):
-            # If we found vswhere, then use it.
             break
     else:
-        # No vswhere on system, no install info available
+        # No vswhere on system, no install info available this way
         return None
 
-    vswhere_cmd = [vswhere_path,
-                   '-products', '*',
-                   '-version', vswhere_version,
-                   '-property', 'installationPath']
-
-    #TODO PY27 cannot use Popen as context manager
-    # try putting it back to the old way for now
-    sp = subprocess.Popen(vswhere_cmd,
-                          stdout=subprocess.PIPE,
-                          stderr=subprocess.PIPE)
-    vsdir, err = sp.communicate()
-    if vsdir:
-        vsdir = vsdir.decode("mbcs").splitlines()
-        # vswhere could easily return multiple lines
-        # we could define a way to pick the one we prefer, but since
+    vswhere_cmd = [
+        vswhere_path,
+        "-products", "*",
+        "-version", vswhere_version,
+        "-property", "installationPath",
+    ]
+
+    #cp = subprocess.run(vswhere_cmd, capture_output=True)  # 3.7+ only
+    cp = subprocess.run(vswhere_cmd, stdout=PIPE, stderr=PIPE)
+
+    if cp.stdout:
+        # vswhere could return multiple lines, e.g. if Build Tools
+        # and {Community,Professional,Enterprise} are both installed.
+        # We could define a way to pick the one we prefer, but since
         # this data is currently only used to make a check for existence,
-        # returning the first hit should be good enough for now.
-        vc_pdir = os.path.join(vsdir[0], 'VC')
-        return vc_pdir
+        # returning the first hit should be good enough.
+        lines = cp.stdout.decode("mbcs").splitlines()
+        return os.path.join(lines[0], 'VC')
     else:
-        # No vswhere on system, no install info available
+        # We found vswhere, but no install info available for this version
         return None
 
 
@@ -403,15 +447,20 @@ def find_batch_file(env,msvc_version,host_arch,target_arch):
     """
     Find the location of the batch script which should set up the compiler
     for any TARGET_ARCH whose compilers were installed by Visual Studio/VCExpress
+
+    In newer (2017+) compilers, make use of the fact there are vcvars
+    scripts named with a host_target pair that calls vcvarsall.bat properly,
+    so use that and return an indication we don't need the argument
+    we would have computed to run vcvarsall.bat.
     """
     pdir = find_vc_pdir(msvc_version)
     if pdir is None:
         raise NoVersionFound("No version of Visual Studio found")
-
     debug('find_batch_file() in {}'.format(pdir))
 
     # filter out e.g. "Exp" from the version name
     msvc_ver_numeric = get_msvc_version_numeric(msvc_version)
+    use_arg = True
     vernum = float(msvc_ver_numeric)
     if 7 <= vernum < 8:
         pdir = os.path.join(pdir, os.pardir, "Common7", "Tools")
@@ -422,7 +471,10 @@ def find_batch_file(env,msvc_version,host_arch,target_arch):
     elif 8 <= vernum <= 14:
         batfilename = os.path.join(pdir, "vcvarsall.bat")
     else:  # vernum >= 14.1  VS2017 and above
-        batfilename = os.path.join(pdir, "Auxiliary", "Build", "vcvarsall.bat")
+        batfiledir = os.path.join(pdir, "Auxiliary", "Build")
+        targ  = _HOST_TARGET_TO_BAT_ARCH_GT14[(host_arch, target_arch)]
+        batfilename = os.path.join(batfiledir, targ)
+        use_arg = False
 
     if not os.path.exists(batfilename):
         debug("Not found: %s" % batfilename)
@@ -437,8 +489,8 @@ def find_batch_file(env,msvc_version,host_arch,target_arch):
             sdk_bat_file_path = os.path.join(pdir,sdk_bat_file)
             if os.path.exists(sdk_bat_file_path):
                 debug('find_batch_file() sdk_bat_file_path:%s'%sdk_bat_file_path)
-                return (batfilename, sdk_bat_file_path)
-    return (batfilename, None)
+                return (batfilename, use_arg, sdk_bat_file_path)
+    return (batfilename, use_arg, None)
 
 
 __INSTALLED_VCS_RUN = None
@@ -446,10 +498,12 @@ def find_batch_file(env,msvc_version,host_arch,target_arch):
 _VC_TOOLS_VERSION_FILE = os.sep.join(_VC_TOOLS_VERSION_FILE_PATH)
 
 def _check_cl_exists_in_vc_dir(env, vc_dir, msvc_version):
-    """Find the cl.exe on the filesystem in the vc_dir depending on
-    TARGET_ARCH, HOST_ARCH and the msvc version. TARGET_ARCH and
-    HOST_ARCH can be extracted from the passed env, unless its None,
-    which then the native platform is assumed the host and target.
+    """Return status of finding a cl.exe to use.
+
+    Locates cl in the vc_dir depending on TARGET_ARCH, HOST_ARCH and the
+    msvc version. TARGET_ARCH and HOST_ARCH can be extracted from the
+    passed env, unless it is None, in which case the native platform is
+    assumed for both host and target.
 
     Args:
         env: Environment
@@ -484,7 +538,8 @@ def _check_cl_exists_in_vc_dir(env, vc_dir, msvc_version):
 
     # make sure the cl.exe exists meaning the tool is installed
     if ver_num > 14:
-        # 2017 and newer allowed multiple versions of the VC toolset to be installed at the same time.
+        # 2017 and newer allowed multiple versions of the VC toolset to be
+        # installed at the same time. This changes the layout.
         # Just get the default tool version for now
         #TODO: support setting a specific minor VC version
         default_toolset_file = os.path.join(vc_dir, _VC_TOOLS_VERSION_FILE)
@@ -509,10 +564,24 @@ def _check_cl_exists_in_vc_dir(env, vc_dir, msvc_version):
             debug('_check_cl_exists_in_vc_dir(): found ' + _CL_EXE_NAME + '!')
             return True
 
+        elif host_platform == "amd64" and host_trgt_dir[0] == "Hostx64":
+            # Special case: fallback to Hostx86 if Hostx64 was tried
+            # and failed.  This is because VS 2017 Express running on amd64
+            # will look to our probe like the host dir should be Hostx64,
+            # but Express uses Hostx86 anyway.
+            # We should key this off the "x86_amd64" and related pseudo
+            # targets, but we don't see those in this function.
+            host_trgt_dir = ("Hostx86", host_trgt_dir[1])
+            cl_path = os.path.join(vc_dir, 'Tools','MSVC', vc_specific_version, 'bin',  host_trgt_dir[0], host_trgt_dir[1], _CL_EXE_NAME)
+            debug('_check_cl_exists_in_vc_dir(): checking for ' + _CL_EXE_NAME + ' at ' + cl_path)
+            if os.path.exists(cl_path):
+                debug('_check_cl_exists_in_vc_dir(): found ' + _CL_EXE_NAME + '!')
+                return True
+
     elif 14 >= ver_num >= 8:
 
-        # Set default value to be -1 as "" which is the value for x86/x86 yields true when tested
-        # if not host_trgt_dir
+        # Set default value to be -1 as "" which is the value for x86/x86
+        # yields true when tested if not host_trgt_dir
         host_trgt_dir = _HOST_TARGET_TO_CL_DIR.get((host_platform, target_platform), None)
         if host_trgt_dir is None:
             debug('_check_cl_exists_in_vc_dir(): unsupported host/target platform combo')
@@ -604,7 +673,7 @@ def reset_installed_vcs():
 # within the same scons run. Windows builds on the CI system were split
 # into chunks to get around single-build time limits.
 # With VS2019 it got even slower and an optional persistent cache file
-# was introduced. The cache now also stores only the parsed vars, 
+# was introduced. The cache now also stores only the parsed vars,
 # not the entire output of running the batch file - saves a bit
 # of time not parsing every time.
 
@@ -646,7 +715,7 @@ def convert(data):
                     return data
 
             cache_data = convert(cache_data)
- 
+
     return cache_data
 
 def get_default_version(env):
@@ -696,33 +765,56 @@ def msvc_setup_env_once(env):
         env["MSVC_SETUP_RUN"] = True
 
 def msvc_find_valid_batch_script(env, version):
-    debug('msvc_find_valid_batch_script()')
-    # Find the host platform, target platform, and if present the requested
-    # target platform
+    """Find and execute appropriate batch script to set up build env.
+
+    The MSVC build environment depends heavily on having the shell
+    environment set.  SCons does not inherit that, and does not count
+    on that being set up correctly anyway, so it tries to find the right
+    MSVC batch script, or the right arguments to the generic batch script
+    vcvarsall.bat, and run that, so we have a valid environment to build in.
+    There are dragons here: the batch scripts don't fail (see comments
+    elsewhere), they just leave you with a bad setup, so try hard to
+    get it right.
+    """
+
+    # Find the host, target, and if present the requested target:
     platforms = get_host_target(env)
     debug(" msvs_find_valid_batch_script(): host_platform %s, target_platform %s req_target_platform:%s" % platforms)
-
     host_platform, target_platform, req_target_platform = platforms
-    try_target_archs = [target_platform]
 
-    # VS2012 has a "cross compile" environment to build 64 bit
-    # with x86_amd64 as the argument to the batch setup script
+    # Most combinations of host + target are straightforward.
+    # While all MSVC / Visual Studio tools are pysically 32-bit, they
+    # make it look like there are 64-bit tools if the host is 64-bit,
+    # so you can invoke the environment batch script to set up to build,
+    # say, amd64 host -> x86 target. Express versions are an exception:
+    # they always look 32-bit, so the batch scripts with 64-bit
+    # host parts are absent. We try to fix that up in a couple of ways.
+    # One is here: we make a table of "targets" to try, with the extra
+    # targets being tags that tell us to try a different "host" instead
+    # of the deduced host.
+    try_target_archs = [target_platform]
     if req_target_platform in ('amd64', 'x86_64'):
         try_target_archs.append('x86_amd64')
-    elif not req_target_platform and target_platform in ['amd64', 'x86_64']:
-        # There may not be "native" amd64, but maybe "cross" x86_amd64 tools
-        try_target_archs.append('x86_amd64')
-        # If the user hasn't specifically requested a TARGET_ARCH, and
-        # The TARGET_ARCH is amd64 then also try 32 bits if there are no viable
-        # 64 bit tools installed
-        try_target_archs.append('x86')
+    elif req_target_platform in ('x86',):
+        try_target_archs.append('x86_x86')
+    elif req_target_platform in ('arm',):
+        try_target_archs.append('x86_arm')
+    elif req_target_platform in ('arm64',):
+        try_target_archs.append('x86_arm64')
+    elif not req_target_platform:
+        if target_platform in ('amd64', 'x86_64'):
+            try_target_archs.append('x86_amd64')
+            # If the user hasn't specifically requested a TARGET_ARCH,
+            # and the TARGET_ARCH is amd64 then also try 32 bits
+            # if there are no viable 64 bit tools installed
+            try_target_archs.append('x86')
 
     debug("msvs_find_valid_batch_script(): host_platform: %s try_target_archs:%s"%(host_platform, try_target_archs))
 
     d = None
     for tp in try_target_archs:
         # Set to current arch.
-        env['TARGET_ARCH']=tp
+        env['TARGET_ARCH'] = tp
 
         debug("msvc_find_valid_batch_script() trying target_platform:%s"%tp)
         host_target = (host_platform, tp)
@@ -732,17 +824,9 @@ def msvc_find_valid_batch_script(env, version):
             SCons.Warnings.warn(SCons.Warnings.VisualCMissingWarning, warn_msg)
         arg = _HOST_TARGET_ARCH_TO_BAT_ARCH[host_target]
 
-        # Get just version numbers
-        maj, min = msvc_version_to_maj_min(version)
-        # VS2015+
-        if maj >= 14:
-            if env.get('MSVC_UWP_APP') == '1':
-                # Initialize environment variables with store/universal paths
-                arg += ' store'
-
         # Try to locate a batch file for this host/target platform combo
         try:
-            (vc_script, sdk_script) = find_batch_file(env, version, host_platform, tp)
+            (vc_script, use_arg, sdk_script) = find_batch_file(env, version, host_platform, tp)
             debug('msvc_find_valid_batch_script() vc_script:%s sdk_script:%s'%(vc_script,sdk_script))
         except VisualCException as e:
             msg = str(e)
@@ -758,6 +842,16 @@ def msvc_find_valid_batch_script(env, version):
         debug('msvc_find_valid_batch_script() use_script 2 %s, args:%s' % (repr(vc_script), arg))
         found = None
         if vc_script:
+            if not use_arg:
+                arg = ''  # bat file will supply platform type
+            # Get just version numbers
+            maj, min = msvc_version_to_maj_min(version)
+            # VS2015+
+            if maj >= 14:
+                if env.get('MSVC_UWP_APP') == '1':
+                    # Initialize environment variables with store/UWP paths
+                    arg = (arg + ' store').lstrip()
+
             try:
                 d = script_env(vc_script, args=arg)
                 found = vc_script
@@ -823,12 +917,13 @@ def msvc_setup_env(env):
         return None
 
     for k, v in d.items():
-        debug('msvc_setup_env() env:%s -> %s'%(k,v))
         env.PrependENVPath(k, v, delete_existing=True)
+        debug("msvc_setup_env() env['ENV']['%s'] = %s" % (k, env['ENV'][k]))
 
     # final check to issue a warning if the compiler is not present
     msvc_cl = find_program_path(env, 'cl')
     if not msvc_cl:
+        debug("msvc_setup_env() did not find 'cl'")
         SCons.Warnings.warn(SCons.Warnings.VisualCMissingWarning,
             "Could not find MSVC compiler 'cl', it may need to be installed separately with Visual Studio")
 
diff --git a/src/engine/SCons/Tool/MSCommon/vs.py b/src/engine/SCons/Tool/MSCommon/vs.py
--- a/src/engine/SCons/Tool/MSCommon/vs.py
+++ b/src/engine/SCons/Tool/MSCommon/vs.py
@@ -205,7 +205,8 @@ def reset(self):
                  hkeys=[],
                  common_tools_var='VS160COMNTOOLS',
                  executable_path=r'Common7\IDE\devenv.com',
-                 batch_file_path=r'VC\Auxiliary\Build\vsvars32.bat',
+                 # should be a fallback, prefer use vswhere installationPath
+                 batch_file_path=r'Common7\Tools\VsDevCmd.bat',
                  supported_arch=['x86', 'amd64', "arm"],
                  ),
 
@@ -216,10 +217,23 @@ def reset(self):
                  hkeys=[],
                  common_tools_var='VS150COMNTOOLS',
                  executable_path=r'Common7\IDE\devenv.com',
-                 batch_file_path=r'VC\Auxiliary\Build\vsvars32.bat',
+                 # should be a fallback, prefer use vswhere installationPath
+                 batch_file_path=r'Common7\Tools\VsDevCmd.bat',
                  supported_arch=['x86', 'amd64', "arm"],
                  ),
 
+    # Visual C++ 2017 Express Edition (for Desktop)
+    VisualStudio('14.1Exp',
+                 vc_version='14.1',
+                 sdk_version='10.0A',
+                 hkeys=[],
+                 common_tools_var='VS150COMNTOOLS',
+                 executable_path=r'Common7\IDE\WDExpress.exe',
+                 # should be a fallback, prefer use vswhere installationPath
+                 batch_file_path=r'Common7\Tools\VsDevCmd.bat',
+                 supported_arch=['x86', 'amd64', "arm"],
+    ),
+
     # Visual Studio 2015
     VisualStudio('14.0',
                  vc_version='14.0',


Test patch:
diff --git a/test/MSVC/MSVC_UWP_APP.py b/test/MSVC/MSVC_UWP_APP.py
--- a/test/MSVC/MSVC_UWP_APP.py
+++ b/test/MSVC/MSVC_UWP_APP.py
@@ -83,23 +83,23 @@ def AreVCStoreLibPathsInLIBPATH(output):
 test.skip_if_not_msvc()
 
 installed_msvc_versions = msvc.cached_get_installed_vcs()
-# MSVC guaranteed to be at least one version on the system or else skip_if_not_msvc() function
-# would have skipped the test
+# MSVC guaranteed to be at least one version on the system or else
+# skip_if_not_msvc() function would have skipped the test
 
 msvc_140 = '14.0' in installed_msvc_versions
 msvc_141 = '14.1' in installed_msvc_versions
+msvc_142 = '14.2' in installed_msvc_versions
 
-if not (msvc_140 or msvc_141):
-    test.skip_test("Available MSVC doesn't support App store")
+if not any((msvc_140, msvc_141, msvc_142)):
+    test.skip_test("Available MSVC doesn't support App store\n")
 
 if msvc_140:
-
-    test.write('SConstruct', """
+    test.write('SConstruct', """\
 if ARGUMENTS.get('MSVC_UWP_APP'):
     help_vars = Variables()
     help_vars.Add(EnumVariable(
                 'MSVC_UWP_APP',
-                'Build for a Universal Windows Platform (UWP) Application',
+                'Build a Universal Windows Platform (UWP) Application',
                 '0',
                 allowed_values=('0', '1')))
 else:
@@ -128,14 +128,31 @@ def AreVCStoreLibPathsInLIBPATH(output):
     test.fail_test((vclibstore_path_present is True) or (vclibstorerefs_path_present is True),
                 message='VC Store LIBPATHs present when MSVC_UWP_APP not set (msvc_version=%s)' % msvc_version)
 
-if msvc_141:
-
-    test.write('SConstruct', """
+if msvc_141 or msvc_142:
+    if msvc_142:
+        test.write('SConstruct', """\
+if ARGUMENTS.get('MSVC_UWP_APP'):
+    help_vars = Variables()
+    help_vars.Add(EnumVariable(
+                'MSVC_UWP_APP',
+                'Build a Universal Windows Platform (UWP) Application',
+                '0',
+                allowed_values=('0', '1')))
+else:
+    help_vars = None
+env = Environment(tools=['default', 'msvc'], variables=help_vars, MSVC_VERSION='14.2')
+# Print the ENV LIBPATH to stdout
+print('env[ENV][LIBPATH]=%s' % env.get('ENV').get('LIBPATH'))
+print('env[MSVC_VERSION]=%s' % env.get('MSVC_VERSION'))
+print('env[ENV][VSCMD_ARG_app_plat]=%s' % env.get('ENV').get('VSCMD_ARG_app_plat'))
+""")
+    elif msvc_141:
+        test.write('SConstruct', """\
 if ARGUMENTS.get('MSVC_UWP_APP'):
     help_vars = Variables()
     help_vars.Add(EnumVariable(
                 'MSVC_UWP_APP',
-                'Build for a Universal Windows Platform (UWP) Application',
+                'Build a Universal Windows Platform (UWP) Application',
                 '0',
                 allowed_values=('0', '1')))
 else: