Repository: mesonbuild/meson

Issue:
Build RPATHs should not be overrideable with LD_LIBRARY_PATH, affects OSes using DT_RUNPATH such as FreeBSD and Debian
Due to #1519. Should we revert or is there a fix?


Discussion:
FWICT, the problem is in the order in which the runtime linker searches for libraries (from the manpage):

> If a shared object dependency does not contain a slash, then it is searched for in the following order:
>
> * (ELF only) Using the directories specified in the `DT_RPATH` dynamic section attribute of the binary if present and `DT_RUNPATH` attribute does not exist. Use of `DT_RPATH` is deprecated.
>
> * Using the environment variable `LD_LIBRARY_PATH` (unless the executable is being run in secure-execution mode; see below).  in which case it is ignored.
>
> * (ELF only) Using the directories specified in the `DT_RUNPATH` dynamic section attribute of the binary if present.

On some systems, including Debian Sid, setting `-Wl,-rpath` sets `DT_RUNPATH` instead of `DT_RPATH`, so `LD_LIBRARY_PATH` is searched first.

The only reliable fix is for `mesontest.py` to append the relevant test directories at the beginning of the `LD_LIBRARY_PATH` env variable. It is going to be a very rare use-case for someone to want to not use the libraries they just built when running the test program that they just built.

However, we can decide that this is too invasive a change right now, and just skip the test for this release.
Continued from https://github.com/mesonbuild/meson/issues/2881:

> FreeBSD configures its clang to pass --enable-new-dtags to the linker by default, so RPATH gets converted to RUNPATH. If Debian is doing the same thing, this issue should also affect Debian users.

It does, and that's what the bug is about. We could disable it, but the way we rewrite rpaths on install means we will also set the install rpath with the old dtags, which breaks Debian and FreeBSD policy.

So the best solution would be to use the old dtags for build rpaths, and the new dtags (or whatever the default is for the linker) for install rpaths. Should be doable.

The other solution is to always use wrapper scripts for custom targets too, which will slow down builds by quite a bit because we have to spawn the python interpreter.
Now we have `meson devenv` (https://github.com/mesonbuild/meson/pull/8393), so users can manually workaround the issue by building and testing in the `meson devenv` environment. That is, users can run `meson devenv ninja` instead of `ninja`, and run `meson devenv meson test` instead of `meson test`.
> Now we have meson devenv (https://github.com/mesonbuild/meson/pull/8393), so users can manually workaround the issue by building and testing in the meson devenv environment. That is, users can run meson devenv ninja instead of ninja, and run meson devenv meson test instead of meson test.

We can, but `meson devenv -C build-x86-64 meson test` is deeply unsatisfying UX ... it's surfacing a really arcane implementation detail (`LD_LIBRARY_PATH` vs. `RUNPATH` vs. `RPATH`) as a first-class thing that users should have to care about. It's not predictable either: sometimes `meson test -C build-x86-64` will work, and sometimes you'll get inexplicable errors.

Can we please just do this, as suggested by @nirbheek?
> The only reliable fix is for mesontest.py to append the relevant test directories at the beginning of the LD_LIBRARY_PATH env variable. It is going to be a very rare use-case for someone to want to not use the libraries they just built when running the test program that they just built.

FWIW, my usecase is that I build and install Wayland (the library) and libweston+Weston into `~/prefix/wayland`. I need to have `~/prefix/wayland/lib` in `$LD_LIBRARY_PATH`, because otherwise I can't resolve the newer symbols it has (or test the newer behaviour it exposes). But if I do, the tests all break, because they link to `libweston.so.$n` from there instead of from the build directory.

Resolving patch:
diff --git a/mesonbuild/backend/backends.py b/mesonbuild/backend/backends.py
--- a/mesonbuild/backend/backends.py
+++ b/mesonbuild/backend/backends.py
@@ -18,6 +18,7 @@
 from functools import lru_cache
 from itertools import chain
 from pathlib import Path
+import copy
 import enum
 import json
 import os
@@ -1141,9 +1142,21 @@ def create_test_serialisation(self, tests: T.List['Test']) -> T.List[TestSeriali
                     cmd_args.extend(self.construct_target_rel_paths(a, t.workdir))
                 else:
                     raise MesonException('Bad object in test command.')
+
+            t_env = copy.deepcopy(t.env)
+            if not machine.is_windows() and not machine.is_cygwin() and not machine.is_darwin():
+                ld_lib_path: T.Set[str] = set()
+                for d in depends:
+                    if isinstance(d, build.BuildTarget):
+                        for l in d.get_all_link_deps():
+                            if isinstance(l, build.SharedLibrary):
+                                ld_lib_path.add(os.path.join(self.environment.get_build_dir(), l.get_subdir()))
+                if ld_lib_path:
+                    t_env.prepend('LD_LIBRARY_PATH', list(ld_lib_path), ':')
+
             ts = TestSerialisation(t.get_name(), t.project_name, t.suite, cmd, is_cross,
                                    exe_wrapper, self.environment.need_exe_wrapper(),
-                                   t.is_parallel, cmd_args, t.env,
+                                   t.is_parallel, cmd_args, t_env,
                                    t.should_fail, t.timeout, t.workdir,
                                    extra_paths, t.protocol, t.priority,
                                    isinstance(exe, build.Target),


Test patch: