Repository: nodejs/gyp-next

Issue:
Native addon compilation using node-gyp >= 10.2.0 (gyp-next >= v0.18.1)
I'm struggling since a large amount of time while trying to compile some Node.js native addons with recent node-gyp (which is internally using gyp-next) releases.

My environment:
Windows 11
Node.js: v22.15.1
Python: v3.13.3
node-gyp: v11.2.0 (gyp-next v0.20.0)
VS Build Tools 2022 (17.14.36109.1)
Windows SDK 10.0.26100.0

It seems like node-gyp is trying to invoke the VS Build Tools compiler (cl.exe) with irrelevant args (GCC/Clang styled):
```
Traceback (most recent call last):
[build]     File "C:\ProgramData\nvm\v22.15.1\node_modules\node-gyp\gyp\gyp_main.py", line 45, in <module>
[build]       sys.exit(gyp.script_main())
[build]                ~~~~~~~~~~~~~~~^^
[build]     File "C:\ProgramData\nvm\v22.15.1\node_modules\node-gyp\gyp\pylib\gyp\__init__.py", line 700, in script_main
[build]       return main(sys.argv[1:])
[build]     File "C:\ProgramData\nvm\v22.15.1\node_modules\node-gyp\gyp\pylib\gyp\__init__.py", line 692, in main
[build]       return gyp_main(args)
[build]     File "C:\ProgramData\nvm\v22.15.1\node_modules\node-gyp\gyp\pylib\gyp\__init__.py", line 659, in gyp_main
[build]       [generator, flat_list, targets, data] = Load(
[build]                                               ~~~~^
[build]           build_files,
[build]           ^^^^^^^^^^^^
[build]       ...<6 lines>...
[build]           options.circular_check,
[build]           ^^^^^^^^^^^^^^^^^^^^^^^
[build]       )
[build]       ^
[build]     File "C:\ProgramData\nvm\v22.15.1\node_modules\node-gyp\gyp\pylib\gyp\__init__.py", line 138, in Load
[build]       generator.CalculateVariables(default_variables, params)
[build]       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
[build]     File "C:\ProgramData\nvm\v22.15.1\node_modules\node-gyp\gyp\pylib\gyp\generator\msvs.py", line 2036, in CalculateVariables
[build]       if gyp.common.GetFlavor(params) == "ninja":
[build]          ~~~~~~~~~~~~~~~~~~~~^^^^^^^^
[build]     File "C:\ProgramData\nvm\v22.15.1\node_modules\node-gyp\gyp\pylib\gyp\common.py", line 502, in GetFlavor
[build]       defines = GetCrossCompilerPredefines()
[build]     File "C:\ProgramData\nvm\v22.15.1\node_modules\node-gyp\gyp\pylib\gyp\common.py", line 449, in GetCrossCompilerPredefines
[build]       stdout = subprocess.run(
[build]                ~~~~~~~~~~~~~~^
[build]           real_cmd, shell=True,
[build]           ^^^^^^^^^^^^^^^^^^^^^
[build]           capture_output=True, check=True
[build]           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[build]       ).stdout
[build]       ^
[build]     File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.13_3.13.1008.0_x64__qbz5n2kfra8p0\Lib\subprocess.py", line 577, in run
[build]       raise CalledProcessError(retcode, process.args,
[build]                                output=stdout, stderr=stderr)
[build]   subprocess.CalledProcessError: Command '['cl.exe', '-dM', '-E', '-x', 'c', 'C:\\Users\\benja\\AppData\\Local\\Temp\\tmpz5je1x25.c']' returned non-zero exit status 2.
```

These changes have been introduced with https://github.com/nodejs/gyp-next/commit/de0e1c9a5791d1bf4bc3103f878ab74814864ab4 on release v0.17.0.

When I'm using node-gyp v10.1.0 (internally using v0.16.1), the addons compilation just goes fine.
Am I missing some flag or some configuration step ?

Thanks a lot!

Discussion:
@Ben08300: Before https://github.com/nodejs/gyp-next/commit/de0e1c9a5791d1bf4bc3103f878ab74814864ab4 the `GetFlavor(params)` call directly returns `"win"`.
After that commit, since the parameter `params` does not include a `"flavor"` we're executing `defines = GetCrossCompilerPredefines()` that leads to an incorrect `cl.exe` invokation.
What I don't understand is that I'm not dealing with cross-compiling at all so what's the point of calling such a function.

Solution patch:
diff --git a/.github/workflows/node-gyp.yml b/.github/workflows/node-gyp.yml
index 56cdb40..c13b28e 100644
--- a/.github/workflows/node-gyp.yml
+++ b/.github/workflows/node-gyp.yml
@@ -14,10 +14,13 @@ jobs:
         python-version: ["3.9", "3.11", "3.13"]
         include:
           - node-version: "22"
-            os: macos-13
+            os: macos-13  # macOS on Intel
             python-version: "3.13"
           - node-version: "22"
-            os: windows-2025
+            os: ubuntu-24.04-arm  # Ubuntu on ARM
+            python-version: "3.13"
+          - node-version: "22"
+            os: windows-11-arm  # Windows on ARM
             python-version: "3.13"
     runs-on: ${{ matrix.os }}
     steps:
diff --git a/.github/workflows/nodejs.yml b/.github/workflows/nodejs.yml
index 213fcbd..416dfe4 100644
--- a/.github/workflows/nodejs.yml
+++ b/.github/workflows/nodejs.yml
@@ -12,9 +12,11 @@ jobs:
         os: [macos-latest, ubuntu-latest, windows-latest]
         python: ["3.9", "3.11", "3.13"]
         include:
-          - os: macos-13
+          - os: macos-13  # macOS on Intel
             python-version: "3.13"
-          - os: windows-2025
+          - os: ubuntu-24.04-arm  # Ubuntu on ARM
+            python-version: "3.13"
+          - os: windows-11-arm  # Windows on ARM
             python-version: "3.13"
 
     runs-on: ${{ matrix.os }}
diff --git a/pylib/gyp/common.py b/pylib/gyp/common.py
index fbf1024..35a64b1 100644
--- a/pylib/gyp/common.py
+++ b/pylib/gyp/common.py
@@ -421,8 +421,9 @@ def EnsureDirExists(path):
     except OSError:
         pass
 
-def GetCrossCompilerPredefines():  # -> dict
+def GetCompilerPredefines():  # -> dict
     cmd = []
+    defines = {}
 
     # shlex.split() will eat '\' in posix mode, but
     # setting posix=False will preserve extra '"' cause CreateProcess fail on Windows
@@ -439,7 +440,7 @@ def GetCrossCompilerPredefines():  # -> dict
         if CXXFLAGS := os.environ.get("CXXFLAGS"):
             cmd += shlex.split(replace_sep(CXXFLAGS))
     else:
-        return {}
+        return defines
 
     if sys.platform == "win32":
         fd, input = tempfile.mkstemp(suffix=".c")
@@ -450,17 +451,33 @@ def GetCrossCompilerPredefines():  # -> dict
                 real_cmd, shell=True,
                 capture_output=True, check=True
             ).stdout
+        except subprocess.CalledProcessError as e:
+            print(
+                "Warning: failed to get compiler predefines\n"
+                "cmd: %s\n"
+                "status: %d" % (e.cmd, e.returncode),
+                file=sys.stderr
+            )
+            return defines
         finally:
             os.unlink(input)
     else:
         input = "/dev/null"
         real_cmd = [*cmd, "-dM", "-E", "-x", "c", input]
-        stdout = subprocess.run(
-            real_cmd, shell=False,
-            capture_output=True, check=True
-        ).stdout
+        try:
+            stdout = subprocess.run(
+                real_cmd, shell=False,
+                capture_output=True, check=True
+            ).stdout
+        except subprocess.CalledProcessError as e:
+            print(
+                "Warning: failed to get compiler predefines\n"
+                "cmd: %s\n"
+                "status: %d" % (e.cmd, e.returncode),
+                file=sys.stderr
+            )
+            return defines
 
-    defines = {}
     lines = stdout.decode("utf-8").replace("\r\n", "\n").split("\n")
     for line in lines:
         if (line or "").startswith("#define "):
@@ -499,7 +516,7 @@ def GetFlavor(params):
     if "flavor" in params:
         return params["flavor"]
 
-    defines = GetCrossCompilerPredefines()
+    defines = GetCompilerPredefines()
     if "__EMSCRIPTEN__" in defines:
         return "emscripten"
     if "__wasm__" in defines:
diff --git a/pylib/gyp/generator/make.py b/pylib/gyp/generator/make.py
index 201fb08..4d4f049 100644
--- a/pylib/gyp/generator/make.py
+++ b/pylib/gyp/generator/make.py
@@ -2381,7 +2381,7 @@ def WriteAutoRegenerationRule(params, root_makefile, makefile_name, build_files)
         % {
             "makefile_name": makefile_name,
             "deps": replace_sep(
-                " ".join(SourceifyAndQuoteSpaces(bf) for bf in build_files)
+                " ".join(sorted(SourceifyAndQuoteSpaces(bf) for bf in build_files))
             ),
             "cmd": replace_sep(gyp.common.EncodePOSIXShellList(
                 [gyp_binary, "-fmake"] + gyp.RegenerateFlags(options) + build_files_args


Test patch:
diff --git a/pylib/gyp/common_test.py b/pylib/gyp/common_test.py
index bd7172a..d25e621 100755
--- a/pylib/gyp/common_test.py
+++ b/pylib/gyp/common_test.py
@@ -7,6 +7,7 @@
 """Unit tests for the common.py file."""
 
 import os
+import subprocess
 import sys
 import unittest
 from unittest.mock import MagicMock, patch
@@ -85,22 +86,34 @@ class TestGetFlavor(unittest.TestCase):
     @patch("os.close")
     @patch("os.unlink")
     @patch("tempfile.mkstemp")
-    def test_GetCrossCompilerPredefines(self, mock_mkstemp, mock_unlink, mock_close):
+    def test_GetCompilerPredefines(self, mock_mkstemp, mock_unlink, mock_close):
         mock_close.return_value = None
         mock_unlink.return_value = None
         mock_mkstemp.return_value = (0, "temp.c")
 
-        def mock_run(env, defines_stdout, expected_cmd):
+        def mock_run(env, defines_stdout, expected_cmd, throws=False):
             with patch("subprocess.run") as mock_run:
-                mock_process = MagicMock()
-                mock_process.returncode = 0
-                mock_process.stdout = TestGetFlavor.MockCommunicate(defines_stdout)
-                mock_run.return_value = mock_process
                 expected_input = "temp.c" if sys.platform == "win32" else "/dev/null"
+                if throws:
+                    mock_run.side_effect = subprocess.CalledProcessError(
+                        returncode=1,
+                        cmd=[
+                            *expected_cmd,
+                            "-dM", "-E", "-x", "c", expected_input
+                        ]
+                    )
+                else:
+                    mock_process = MagicMock()
+                    mock_process.returncode = 0
+                    mock_process.stdout = TestGetFlavor.MockCommunicate(defines_stdout)
+                    mock_run.return_value = mock_process
                 with patch.dict(os.environ, env):
-                    defines = gyp.common.GetCrossCompilerPredefines()
+                    try:
+                        defines = gyp.common.GetCompilerPredefines()
+                    except Exception as e:
+                        self.fail(f"GetCompilerPredefines raised an exception: {e}")
                     flavor = gyp.common.GetFlavor({})
-                if env.get("CC_target"):
+                if env.get("CC_target") or env.get("CC"):
                     mock_run.assert_called_with(
                         [
                             *expected_cmd,
@@ -110,6 +123,9 @@ class TestGetFlavor(unittest.TestCase):
                         capture_output=True, check=True)
                 return [defines, flavor]
 
+        [defines0, _] = mock_run({ "CC": "cl.exe" }, "", ["cl.exe"], True)
+        assert defines0 == {}
+
         [defines1, _] = mock_run({}, "", [])
         assert defines1 == {}