Repository: mesonbuild/meson

Issue:
Arguments in MESONINTROSPECT are escaped incosistently
**Describe the bug**
The arguments in the `MESONINTROSPECT` environment variable are escaped with the `join_args` function in `mesonbuild/utils/universal.py` for postconf scripts while they are escaped with `shlex.quote()` in the other situations.

`join_args` and `shlex.quote()` have different behaviors on Windows. `join_args` returns a string that is compatible with Win32 `CreateProcess` function, but `shlex.quote()` returns a string that is compatible with Python `shlex.split()` function. However, neither is compatible with both functions (e.g., `shlex.quote()` is not compatible with Win32 `CreateProcess`.)

The incompatibility of `join_args` and `shlex.split()` broke `test cases/common/139 mesonintrospect from scripts` and the breakage was detected with #12115. 

Here are possible ways to fix the inconsistency:
1. Use `shlex.quote()` instead of `join_args` to escape the arguments in `MESONINTROSPECT` for postconf scripts. This breaks post scripts that pass `MESONINTROSPECT` to Windows APIs.
2. ~Always use `join_args` to escape the arguments in `MESONINTROSPECT` and make `join_args` emit a string compatible with `shlex.split()`. More concretely, make it replace `\` with `\\`. This adds extra verbosity to `MESONINTROSPECT`.~ No, Windows treats `\\` literally.
3. Always use `join_args` to escape the arguments in `MESONINTROSPECT` and make `check_introspection.py` pass it to cmd on Windows. This abandons compatibility with `shlex.split()` and makes it harder to use `MESONINTROSPECT` in Python scripts.

@xclaesse suggested always using `join_args` and checking other `shlex.quote()` usage. I found the other usage is fine:

- `mesonbuild/coredata.py` uses the function to get `meson.build_options()` but the documentation says not to parse it:
https://github.com/mesonbuild/meson/blob/bde690b06e930020a0ec6ccaea7a76babf77dff5/docs/yaml/builtins/meson.yaml#L174
- `mesonbuild/mtest.py` uses the function if it's *not* on Windows. It returns a string as is on Windows and has "TODO" comment. The output is used only for logging:
https://github.com/mesonbuild/meson/blob/543e9ca0cf0c00d752bd723ec403e91b839bf9b4/mesonbuild/mtest.py#L229
- `mesonbuild/backend/ninjabackend.py` uses the function for GCC response file. GCC response file has its own parser so the compatibility with Windows APIs does not matter.
- `mesonbuild/modules/external_project.py` uses the function for environment variables like `CFLAGS`. Such variables should always have POSIX shell syntax since they are used by configure scripts and Makefile that use POSIX shell.

@eli-schwartz on the other hand suggests always using `shlex.quote()` since the usage of `join_args` is a breaking change introduced with commit 522392e7553823e6b3ac38cadc4fbee72eae9540.

I'd like to point out the documentation also says `MESONINTROSPECT` is compatible with `shlex.split()`:
https://github.com/mesonbuild/meson/blob/bde690b06e930020a0ec6ccaea7a76babf77dff5/docs/yaml/builtins/meson.yaml#L89

@xclaesse @eli-schwartz Taking all of the above into account, what do you think we should use for `MESONINTROSPECT`, `shlex.split()` or `join_args`?

**To Reproduce**
Run tests with #12115.

**Expected behavior**
The tests should pass.

**system parameters**
* Is this a [cross build](https://mesonbuild.com/Cross-compilation.html) or just a plain native build (for the same computer)? Native.
* what operating system (e.g. MacOS Catalina, Windows 10, CentOS 8.0, Ubuntu 18.04, etc.) Arch Linux ARM
* what Python version are you using e.g. 3.8.0 3.11.3
* what `meson --version` 1.2.1
* what `ninja --version` if it's a Ninja build 1.11.1



Discussion:

Resolving patch:
diff --git a/mesonbuild/backend/backends.py b/mesonbuild/backend/backends.py
--- a/mesonbuild/backend/backends.py
+++ b/mesonbuild/backend/backends.py
@@ -14,6 +14,7 @@
 import os
 import pickle
 import re
+import shlex
 import shutil
 import typing as T
 import hashlib
@@ -26,8 +27,7 @@
 from ..compilers import LANGUAGES_USING_LDFLAGS, detect
 from ..mesonlib import (
     File, MachineChoice, MesonException, OrderedSet,
-    classify_unity_sources, OptionKey, join_args,
-    ExecutableSerialisation
+    ExecutableSerialisation, classify_unity_sources, OptionKey
 )
 
 if T.TYPE_CHECKING:
@@ -1598,22 +1598,23 @@ def eval_custom_target_command(
         cmd = [i.replace('\\', '/') for i in cmd]
         return inputs, outputs, cmd
 
+    def get_introspect_command(self) -> str:
+        return ' '.join(shlex.quote(x) for x in self.environment.get_build_command() + ['introspect'])
+
     def get_run_target_env(self, target: build.RunTarget) -> mesonlib.EnvironmentVariables:
         env = target.env if target.env else mesonlib.EnvironmentVariables()
         if target.default_env:
-            introspect_cmd = join_args(self.environment.get_build_command() + ['introspect'])
             env.set('MESON_SOURCE_ROOT', [self.environment.get_source_dir()])
             env.set('MESON_BUILD_ROOT', [self.environment.get_build_dir()])
             env.set('MESON_SUBDIR', [target.subdir])
-            env.set('MESONINTROSPECT', [introspect_cmd])
+            env.set('MESONINTROSPECT', [self.get_introspect_command()])
         return env
 
     def run_postconf_scripts(self) -> None:
         from ..scripts.meson_exe import run_exe
-        introspect_cmd = join_args(self.environment.get_build_command() + ['introspect'])
         env = {'MESON_SOURCE_ROOT': self.environment.get_source_dir(),
                'MESON_BUILD_ROOT': self.environment.get_build_dir(),
-               'MESONINTROSPECT': introspect_cmd,
+               'MESONINTROSPECT': self.get_introspect_command(),
                }
 
         for s in self.build.postconf_scripts:


Test patch: