Repository: mesonbuild/meson

Issue:
'meson test' fails to run on Python 3.12
**Describe the bug**
Meson fails to run on Python 3.12 due to reliance on the implicit instantiation of the asyncio mainloop that was deprecated in Python 3.10. It has now been removed.

Additional details in [this Fedora bug](https://bugzilla.redhat.com/show_bug.cgi?id=2155490) and [this Python upstream issue](https://github.com/python/cpython/issues/93453).


**To Reproduce**

* Install python 3.12 alpha.
* Run `meson test` in any project that supports it


**Expected behavior**
Tests should run

**system parameters**
* Is this a [cross build](https://mesonbuild.com/Cross-compilation.html) or just a plain native build (for the same computer)?
  * Native
* what operating system (e.g. MacOS Catalina, Windows 10, CentOS 8.0, Ubuntu 18.04, etc.)
  * Experienced on Fedora Rawhide (38), but likely present on any Linux with Python 3.12
* what Python version are you using e.g. 3.8.0
  * Python 3.12
* what `meson --version`
  * Meson 0.64.1
* what `ninja --version` if it's a Ninja build
  * Ninja 1.5.8.1



Discussion:
CPython has tagged this as a release blocker for 3.10 and 3.11 stable (due to adding a deprecation warning after the initial .0 release) and is reverting the change for 3.12 since it wasn't properly deprecated. The new plan is to remove it in 3.14 instead.

This isn't to say that it would be bad to move away from that behavior, but, it's a bit less of a priority I guess.
@scivision any code that manages to *segfault* in pure python code is almost certainly a CPython bug, which really should be fixed by CPython during the alpha phase so that projects written in python don't need to solve interpreter segfaults on their end.
Hi Eli, my apologies I tested with an old Meson version. I'm updating my Python 3.12.0a3 results now with current Meson. So my first post I deleted was invalid.
I reran with Meson master branch and Python 3.12.0a3 I built myself on CentOS.
It appears likely many or all of these errors are due to "meson test" issues noted above and below with Python 3.12 development -- a future Python 3.12 alpha release is likely to fix.

```sh
$ /local/bin/python3 run_unittests.py

Meson build system 1.0.0.rc2 Unit Tests
=========================================== test session starts
===========================================platform linux -- Python 3.12.0a3, pytest-7.2.0, pluggy-1.0.0
rootdir: /code/meson, configfile: setup.cfg
collected 479 items
```

```
========================================= short test summary info 
FAILED run_unittests.py::AllPlatformTests::test_scripts_loaded_modules - AssertionError: Lists differ: ['mesonbuild', 'mesonbuild._pathlib', 'meso[173 chars]ore'] != ['__edita...
FAILED run_unittests.py::AllPlatformTests::test_prebuilt_object - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_long_output - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_prebuilt_shared_lib_rpath - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_testsetup_default - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_force_fallback_for_nofallback - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_lto_threads - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_force_fallback_for - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_reconfigure - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_dist_git_script - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_static_library_lto - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_prebuilt_shared_lib - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_testsetup_selection - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_prebuilt_static_lib - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_forcefallback - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_suite_selection - AssertionError: 2 != 4
FAILED run_unittests.py::AllPlatformTests::test_dist_git - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_testsetups - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_testrepeat - subprocess.CalledProcessError: Command '['/code/cmake-python-build/build/local/bin/python3',...
FAILED run_unittests.py::CrossFileTests::test_cross_exe_passed_no_wrapper - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_pkgconfig_static - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::AllPlatformTests::test_wrap_git - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::CrossFileTests::test_needs_exe_wrapper_true_wrapper - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::LinuxlikeTests::test_freezing - AssertionError: 'Traceback' unexpectedly found in 'ninja explain: output meson-benchmark-prereq of pho...
FAILED run_unittests.py::CrossFileTests::test_needs_exe_wrapper_false - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::LinuxlikeTests::test_symlink_builddir - subprocess.CalledProcessError: Command '['/code/cmake-python-build/build/local/bin/python3',...
FAILED run_unittests.py::SubprojectsCommandTests::test_purge - subprocess.CalledProcessError: Command '['/code/cmake-python-build/build/local/bin/python3',...
FAILED run_unittests.py::CrossFileTests::test_needs_exe_wrapper_true - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::SubprojectsCommandTests::test_foreach - subprocess.CalledProcessError: Command '['/code/cmake-python-build/build/local/bin/python3',...
FAILED run_unittests.py::LinuxlikeTests::test_usage_external_library - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::LinuxlikeTests::test_pch_with_address_sanitizer - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::LinuxlikeTests::test_link_arg_fullname - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::LinuxlikeTests::test_static_link - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::LinuxlikeTests::test_pkgconfig_uninstalled - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
FAILED run_unittests.py::SubprojectsCommandTests::test_git_update - subprocess.CalledProcessError: Command '['/code/cmake-python-build/build/local/bin/python3',...
FAILED run_unittests.py::AllPlatformTests::test_verbose - subprocess.CalledProcessError: Command '['/code/cmake-python-build/build/local/bin/python3',...
FAILED run_unittests.py::LinuxlikeTests::test_wrap_with_file_url - subprocess.CalledProcessError: Command '['/ninja-1.11.1/ninja', '-w', 'dupbuild=err', '-d', ...
=================== 37 failed, 370 passed, 72 skipped, 451 warnings in 77.25s (0:01:17) ===================Total time: 77.540 seconds
```

Here's what I get on a trivial Meson project test. 

Note that whether or not specifying `meson test --num-processes 1` the same error occurs.

```
ninja: no work to do.
ninja: Entering directory `/code/trivial/build'
ninja: no work to do.
Traceback (most recent call last):
  File "/code/meson/mesonbuild/mesonmain.py", line 193, in run
    return options.run_func(options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/meson/mesonbuild/mtest.py", line 2107, in run
    return th.doit()
           ^^^^^^^^^
  File "/code/meson/mesonbuild/mtest.py", line 1788, in doit
    self.run_tests(runners)
  File "/code/meson/mesonbuild/mtest.py", line 1926, in run_tests
    loop = asyncio.get_event_loop()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/cmake-python-build/build/local/lib/python3.12/asyncio/events.py", line 676, in get_event_loop
    raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'MainThread'.

ERROR: Unhandled python exception

    This is a Meson bug and should be reported!
```

Resolving patch:
diff --git a/mesonbuild/mesonmain.py b/mesonbuild/mesonmain.py
--- a/mesonbuild/mesonmain.py
+++ b/mesonbuild/mesonmain.py
@@ -235,6 +235,13 @@ def set_meson_command(mainfile):
     mesonlib.set_meson_command(mainfile)
 
 def run(original_args, mainfile):
+    if os.environ.get('MESON_SHOW_DEPRECATIONS'):
+        # workaround for https://bugs.python.org/issue34624
+        import warnings
+        for typ in [DeprecationWarning, SyntaxWarning, FutureWarning, PendingDeprecationWarning]:
+            warnings.filterwarnings('error', category=typ, module='mesonbuild')
+        warnings.filterwarnings('ignore', message=".*importlib-resources.*")
+
     if sys.version_info >= (3, 10) and os.environ.get('MESON_RUNNING_IN_PROJECT_TESTS'):
         # workaround for https://bugs.python.org/issue34624
         import warnings
diff --git a/mesonbuild/msubprojects.py b/mesonbuild/msubprojects.py
--- a/mesonbuild/msubprojects.py
+++ b/mesonbuild/msubprojects.py
@@ -699,7 +699,8 @@ def run(options: 'Arguments') -> int:
             raise MesonException(f'Unknown subproject type {t!r}, supported types are: {ALL_TYPES_STRING}')
     tasks: T.List[T.Awaitable[bool]] = []
     task_names: T.List[str] = []
-    loop = asyncio.get_event_loop()
+    loop = asyncio.new_event_loop()
+    asyncio.set_event_loop(loop)
     executor = ThreadPoolExecutor(options.num_processes)
     if types:
         wraps = [wrap for wrap in wraps if wrap.type in types]
diff --git a/mesonbuild/mtest.py b/mesonbuild/mtest.py
--- a/mesonbuild/mtest.py
+++ b/mesonbuild/mtest.py
@@ -502,7 +502,9 @@ def __init__(self) -> None:
         self.progress_task = None          # type: T.Optional[asyncio.Future]
         self.max_left_width = 0            # type: int
         self.stop = False
-        self.update = asyncio.Event()
+        # TODO: before 3.10 this cannot be created immediately, because
+        # it will create a new event loop
+        self.update: asyncio.Event
         self.should_erase_line = ''
         self.test_count = 0
         self.started_tests = 0
@@ -572,7 +574,7 @@ def emit_progress(self, harness: 'TestHarness') -> None:
 
     def start(self, harness: 'TestHarness') -> None:
         async def report_progress() -> None:
-            loop = asyncio.get_event_loop()
+            loop = asyncio.get_running_loop()
             next_update = 0.0
             self.request_update()
             while not self.stop:
@@ -600,6 +602,7 @@ async def report_progress() -> None:
                 self.emit_progress(harness)
             self.flush()
 
+        self.update = asyncio.Event()
         self.test_count = harness.test_count
         self.cols = max(self.cols, harness.max_left_width + 30)
 
@@ -1229,13 +1232,14 @@ def check_futures(futures: T.Iterable[asyncio.Future]) -> None:
 
     # Python is silly and does not have a variant of asyncio.wait with an
     # absolute time as deadline.
-    deadline = None if timeout is None else asyncio.get_event_loop().time() + timeout
+    loop = asyncio.get_running_loop()
+    deadline = None if timeout is None else loop.time() + timeout
     while futures and (timeout is None or timeout > 0):
         done, futures = await asyncio.wait(futures, timeout=timeout,
                                            return_when=asyncio.FIRST_EXCEPTION)
         check_futures(done)
         if deadline:
-            timeout = deadline - asyncio.get_event_loop().time()
+            timeout = deadline - loop.time()
 
     check_futures(futures)
 
@@ -1922,9 +1926,12 @@ def get_pretty_suite(self, test: TestSerialisation) -> str:
     def run_tests(self, runners: T.List[SingleTestRunner]) -> None:
         try:
             self.open_logfiles()
-            # Replace with asyncio.run once we can require Python 3.7
-            loop = asyncio.get_event_loop()
-            loop.run_until_complete(self._run_tests(runners))
+
+            # TODO: this is the default for python 3.8
+            if sys.platform == 'win32':
+                asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy())
+
+            asyncio.run(self._run_tests(runners))
         finally:
             self.close_logfiles()
 
@@ -1942,6 +1949,7 @@ async def _run_tests(self, runners: T.List[SingleTestRunner]) -> None:
         running_tests = {}  # type: T.Dict[asyncio.Future, str]
         interrupted = False
         ctrlc_times = deque(maxlen=MAX_CTRLC)  # type: T.Deque[float]
+        loop = asyncio.get_running_loop()
 
         async def run_test(test: SingleTestRunner) -> None:
             async with semaphore:
@@ -1990,7 +1998,7 @@ def sigint_handler() -> None:
             nonlocal interrupted
             if interrupted:
                 return
-            ctrlc_times.append(asyncio.get_event_loop().time())
+            ctrlc_times.append(loop.time())
             if len(ctrlc_times) == MAX_CTRLC and ctrlc_times[-1] - ctrlc_times[0] < 1:
                 self.flush_logfiles()
                 mlog.warning('CTRL-C detected, exiting')
@@ -2007,10 +2015,10 @@ def sigint_handler() -> None:
 
         if sys.platform != 'win32':
             if os.getpgid(0) == os.getpid():
-                asyncio.get_event_loop().add_signal_handler(signal.SIGINT, sigint_handler)
+                loop.add_signal_handler(signal.SIGINT, sigint_handler)
             else:
-                asyncio.get_event_loop().add_signal_handler(signal.SIGINT, sigterm_handler)
-            asyncio.get_event_loop().add_signal_handler(signal.SIGTERM, sigterm_handler)
+                loop.add_signal_handler(signal.SIGINT, sigterm_handler)
+            loop.add_signal_handler(signal.SIGTERM, sigterm_handler)
         try:
             for runner in runners:
                 if not runner.is_parallel:
@@ -2027,8 +2035,8 @@ def sigint_handler() -> None:
             await complete_all(futures)
         finally:
             if sys.platform != 'win32':
-                asyncio.get_event_loop().remove_signal_handler(signal.SIGINT)
-                asyncio.get_event_loop().remove_signal_handler(signal.SIGTERM)
+                loop.remove_signal_handler(signal.SIGINT)
+                loop.remove_signal_handler(signal.SIGTERM)
             for l in self.loggers:
                 await l.finish(self)
 
@@ -2087,10 +2095,6 @@ def run(options: argparse.Namespace) -> int:
     if options.wrapper:
         check_bin = options.wrapper[0]
 
-    if sys.platform == 'win32':
-        loop = asyncio.ProactorEventLoop()
-        asyncio.set_event_loop(loop)
-
     if check_bin is not None:
         exe = ExternalProgram(check_bin, silent=True)
         if not exe.found():


Test patch: