Repository: mesonbuild/meson

Issue:
deja-dup 43.2 tests under xbps-src fail to run with a TypeError in meson
**Describe the bug**
Trying to run deja-dup 43.2 tests under xbps-src (Void Linux build system) results in an error:
```
[0/1] Running all tests.
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/mesonbuild/mesonmain.py", line 149, in run
    return options.run_func(options)
  File "/usr/lib/python3.10/site-packages/mesonbuild/mtest.py", line 2017, in run
    return th.doit()
  File "/usr/lib/python3.10/site-packages/mesonbuild/mtest.py", line 1685, in doit
    runners.extend(self.get_test_runner(test) for test in tests)
  File "/usr/lib/python3.10/site-packages/mesonbuild/mtest.py", line 1685, in <genexpr>
    runners.extend(self.get_test_runner(test) for test in tests)
  File "/usr/lib/python3.10/site-packages/mesonbuild/mtest.py", line 1586, in get_test_runner
    return SingleTestRunner(test, env, name, options)
  File "/usr/lib/python3.10/site-packages/mesonbuild/mtest.py", line 1308, in __init__
    self.cmd = self._get_cmd()
  File "/usr/lib/python3.10/site-packages/mesonbuild/mtest.py", line 1374, in _get_cmd
    test_cmd = self._get_test_cmd()
  File "/usr/lib/python3.10/site-packages/mesonbuild/mtest.py", line 1352, in _get_test_cmd
    if self.options.no_rebuild and not os.path.isfile(testentry):
  File "/usr/lib/python3.10/genericpath.py", line 30, in isfile
    st = os.stat(path)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

ERROR: Unhandled python exception

    This is a Meson bug and should be reported!
FAILED: meson-test 
/usr/bin/meson test --no-rebuild --print-errorlogs
ninja: build stopped: subcommand failed.
```

**To Reproduce**
This seems to only happen under xbps-src that runs builds in a limited chroot which might be why I can't reproduce this on my own computer.

**Expected behavior**
Tests should run and meson shouldn't exit with a TypeError.

**system parameters**
* native build
* Void Linux
* Python 3.10
* meson 0.61.99 (789f440f7c2bfda2e17f05ecdcb1a19245f892df), meson 0.61.2 and meson 0.60.3
* ninja 1.10.2



Discussion:
It seems odd that `_get_test_cmd` claims the test command is a NoneType.

The specific code that's being hit here was added in c85f7a39a80ccbadb9ffb76102da30c05e3db66f but before that we checked `if self.test.fname[0].endswith('.jar')` and now we first check if `os.path.isfile(self.test.fname[0])`.

We always assumed that a test has to have a test command, and I don't get how that would ever not be the case. I wonder what `meson-private/meson_test_setup.dat` contains in it...
Okay, confirmed issue:

```python
>>> ret = pickle.load(open('meson-private/meson_test_setup.dat', 'rb'))
>>> for i in ret:
...     print(f'{i.name=}, {i.fname=}')
... 
i.name='validate-gschema', i.fname=['/usr/bin/glib-compile-schemas']
i.name='validate-monitor-desktop', i.fname=[None]
i.name='validate-desktop', i.fname=[None]
i.name='validate-appstream', i.fname=[None]
i.name='help-linguas', i.fname=['/builddir/deja-dup-43.2/help/./validate-linguas']
i.name='script-bad-hostname', i.fname=[None]
i.name='script-bad-volume', i.fname=[None]
i.name='script-cancel-noop', i.fname=[None]
i.name='script-cancel', i.fname=[None]
i.name='script-clean-cache-if-full', i.fname=[None]
i.name='script-clean-cache', i.fname=[None]
i.name='script-clean-incomplete', i.fname=[None]
i.name='script-clean-tempdir', i.fname=[None]
i.name='script-custom-tool-setup-bad', i.fname=[None]
i.name='script-custom-tool-setup-fails', i.fname=[None]
i.name='script-custom-tool-setup-teardown', i.fname=[None]
i.name='script-custom-tool-wrapper', i.fname=[None]
i.name='script-custom-tool-wrapper-bad', i.fname=[None]
i.name='script-delete-just-right', i.fname=[None]
i.name='script-delete-never', i.fname=[None]
i.name='script-delete-too-few', i.fname=[None]
i.name='script-delete-too-old', i.fname=[None]
i.name='script-disk-full2', i.fname=[None]
i.name='script-disk-full', i.fname=[None]
i.name='script-disk-small', i.fname=[None]
i.name='script-encrypt-ask', i.fname=[None]
i.name='script-encrypt-detect', i.fname=[None]
i.name='script-excludes', i.fname=[None]
i.name='script-instance-error', i.fname=[None]
i.name='script-mkdir', i.fname=[None]
i.name='script-nag', i.fname=[None]
i.name='script-no-space', i.fname=[None]
i.name='script-old-version', i.fname=[None]
i.name='script-permission', i.fname=[None]
i.name='script-read-error', i.fname=[None]
i.name='script-restore-full', i.fname=[None]
i.name='script-restore-specific', i.fname=[None]
i.name='script-special-chars', i.fname=[None]
i.name='script-stop', i.fname=[None]
i.name='script-symlink-direct', i.fname=[None]
i.name='script-symlink-exclude2', i.fname=[None]
i.name='script-symlink-exclude', i.fname=[None]
i.name='script-symlink-follow2', i.fname=[None]
i.name='script-symlink-follow', i.fname=[None]
i.name='script-symlink-loop', i.fname=[None]
i.name='script-symlink-recursive', i.fname=[None]
i.name='script-symlink-subdir', i.fname=[None]
i.name='script-symlink-trickshot', i.fname=[None]
i.name='script-threshold-full', i.fname=[None]
i.name='script-threshold-inc', i.fname=[None]
i.name='script-verify', i.fname=[None]
i.name='script-write-error', i.fname=[None]
i.name='libdeja', i.fname=[None]
i.name='scheduler', i.fname=['/builddir/deja-dup-43.2/build/monitor/tests/scheduler-test']
i.name='po-linguas', i.fname=['/builddir/deja-dup-43.2/po/./validate-linguas']
i.name='potfiles', i.fname=['/builddir/deja-dup-43.2/po/./validate-potfiles']
```

Looking up the tests in question, they are all cases where tests have been defined using a `find_program(..., required: false)` which was not found.

As expected, it fails with Meson 0.59 too, like this:

```python
    if self.test.fname[0].endswith('.jar'):
AttributeError: 'NoneType' object has no attribute 'endswith'
```

I think we should not allow adding a test setup with a not-found program... probably we should not allow that anywhere else at all. In fact for say custom_target we already fail with `ERROR: Tried to use not-found external program in "command"`.

Resolving patch:
diff --git a/mesonbuild/interpreter/interpreter.py b/mesonbuild/interpreter/interpreter.py
--- a/mesonbuild/interpreter/interpreter.py
+++ b/mesonbuild/interpreter/interpreter.py
@@ -1984,7 +1984,10 @@ def make_test(self, node: mparser.BaseNode,
                              location=node)
             name = name.replace(':', '_')
         exe = args[1]
-        if isinstance(exe, mesonlib.File):
+        if isinstance(exe, ExternalProgram):
+            if not exe.found():
+                raise InvalidArguments('Tried to use not-found external program as test exe')
+        elif isinstance(exe, mesonlib.File):
             exe = self.find_program_impl([exe])
 
         env = self.unpack_env_kwarg(kwargs)


Test patch: