Repository: mesonbuild/meson

Issue:
Boost rewrite breaks build
Consider the following meson.build file
```
project('test_link', 'cpp', default_options : ['cpp_std=c++17'])

boost_dep = dependency('boost', modules : ['system'])

sources = files(
        'main.cpp'
)

a_out = executable('a.out', sources, dependencies : [boost_dep])
```
(the file `main.cpp` can just contain `int main(){}`.)

This no longer builds.  The behavior was introduced with 08224dafcba1b694fb624553e7d84deb565aae22.

The problem is that instead of `-lboost_system`, the file `/usr/lib/libboost_system.so.1.72.0` is passed as such to the linker:
```
[2/2] c++  -o a.out 'a.out@exe/main.cpp.o' -Wl,--as-needed -Wl,--no-undefined -Wl,--start-group /usr/lib/libboost_system.so.1.72.0 -Wl,--end-group
FAILED: a.out 
c++  -o a.out 'a.out@exe/main.cpp.o' -Wl,--as-needed -Wl,--no-undefined -Wl,--start-group /usr/lib/libboost_system.so.1.72.0 -Wl,--end-group
/usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/../../../../x86_64-pc-linux-gnu/bin/ld: error: /usr/lib/libboost_system.so.1.72.0: incompatible target
collect2: error: ld returned 1 exit status
```


Discussion:
I should add that this *only* happens on our Gentoo system, not the Ubuntu one.  It happens with gcc-9, gcc-10, and clang-10.  Please let me know what I should provide to make it more reproducible, ideally in a CI test.

One difference is that the Gentoo system features Boost 1.72, the Ubuntu one Boost 1.71.
The two `meson-log.txt` files do not show any significant differences except for library paths.
What is the output of `ldd <your exe>` when built with `-lboost_system` and can you reproduce the bug when compiling your exe manually (without all the `-Wl,` stuff)?

Also can you post the output of `find / -name "*boost_system*" -type f`?

This looks like a really weird bug since the `/usr/lib/libboost_system.so.1.72.0` seems to be the correct library...
(To keep the linker from dropping the shared library altogether, I changed `main.cpp` to
```cpp
#include <boost/filesystem.hpp>

int main()
{
        return boost::filesystem::path("/path/to/something").filename().size() != 0;
}
```
and added `filesystem` as a boost dependency to `meson.build`.)

```sh
fiesh@wsbox /tmp/working % ldd a.out
        linux-vdso.so.1 (0x00007fffbf9f7000)
        libboost_filesystem.so.1.72.0 => /usr/lib64/libboost_filesystem.so.1.72.0 (0x00007f5305d7e000)
        libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/libstdc++.so.6 (0x00007f5305b05000)
        libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/libgcc_s.so.1 (0x00007f5305aec000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f5305933000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f5305929000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5305909000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f53057d1000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f5305df8000)
```

I suppose by "compiling by hand" you mean the following?
```
fiesh@wsbox /tmp/notworking % c++  -o a.out 'a.out@exe/main.cpp.o' /usr/lib/libboost_system.so.1.72.0 /usr/lib/libboost_filesystem.so.1.72.0
/usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/../../../../x86_64-pc-linux-gnu/bin/ld: error: /usr/lib/libboost_system.so.1.72.0: incompatible target
/usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/../../../../x86_64-pc-linux-gnu/bin/ld: error: /usr/lib/libboost_filesystem.so.1.72.0: incompatible target
../t/main.cpp:5: error: undefined reference to 'boost::filesystem::path::filename() const'
collect2: error: ld returned 1 exit status
```

Unlike with static libraries, I don't think it is possible to specify a shared library by its name, one has to use the `-llibname` mechanism.

Running `find / -name "*boost_system*" -type f` is a bit difficult since it's a rather big server.  But I suppose this should suffice:
```sh
% find /usr -name "*boost_system*" -type f 2>/dev/null
/usr/lib/libboost_system.so.1.72.0
/usr/lib/debug/usr/lib64/libboost_system.so.1.72.0.debug
/usr/lib/debug/usr/lib/libboost_system.so.1.72.0.debug
/usr/lib/libboost_system.a
/usr/share/aclocal/ax_boost_system.m4
/usr/lib64/libboost_system.so.1.72.0
/usr/lib64/libboost_system.a
```
Ah, seems like meson uses `lib` instead of `lib64`. Thanks, I now know where the problem is.
> Ah, seems like meson uses `lib` instead of `lib64`. Thanks, I now know where the problem is.

Oh wow awesome!  I was just about to prepare a `Dockerfile` to reproduce it.  But I guess I'll put that on halt for now.
>  The two `meson-log.txt` files do not show any significant differences except for library paths.

Library paths seem to be *extremely* important! Since meson uses full paths to shared libraries *always*, one must assume this works. However, you're getting an error message from `ld.gold` stating that it tried to link a file which doesn't share the same architecture/os ABI. Hence @mensinda's conclusion that it's incorrectly using /usr/lib on your system.

I can trivially reproduce the error message on archlinux by trying to relink a random program using meson's command line but switching one lib to use /usr/lib32 and adding -fuse-ld=gold.

```
$ cc -fuse-ld=gold  -o libalpm.so.12.0.1 'common@sta/src_common_ini.c.o' 'common@sta/src_common_util-common.c.o' 'alpm_objlib@sta/lib_libalpm_add.c.o' 'alpm_objlib@sta/lib_libalpm_alpm.c.o' 'alpm_objlib@sta/lib_libalpm_alpm_list.c.o' 'alpm_objlib@sta/lib_libalpm_backup.c.o' 'alpm_objlib@sta/lib_libalpm_base64.c.o' 'alpm_objlib@sta/lib_libalpm_be_local.c.o' 'alpm_objlib@sta/lib_libalpm_be_package.c.o' 'alpm_objlib@sta/lib_libalpm_be_sync.c.o' 'alpm_objlib@sta/lib_libalpm_conflict.c.o' 'alpm_objlib@sta/lib_libalpm_db.c.o' 'alpm_objlib@sta/lib_libalpm_deps.c.o' 'alpm_objlib@sta/lib_libalpm_diskspace.c.o' 'alpm_objlib@sta/lib_libalpm_dload.c.o' 'alpm_objlib@sta/lib_libalpm_error.c.o' 'alpm_objlib@sta/lib_libalpm_filelist.c.o' 'alpm_objlib@sta/lib_libalpm_graph.c.o' 'alpm_objlib@sta/lib_libalpm_group.c.o' 'alpm_objlib@sta/lib_libalpm_handle.c.o' 'alpm_objlib@sta/lib_libalpm_hook.c.o' 'alpm_objlib@sta/lib_libalpm_log.c.o' 'alpm_objlib@sta/lib_libalpm_package.c.o' 'alpm_objlib@sta/lib_libalpm_pkghash.c.o' 'alpm_objlib@sta/lib_libalpm_rawstr.c.o' 'alpm_objlib@sta/lib_libalpm_remove.c.o' 'alpm_objlib@sta/lib_libalpm_signing.c.o' 'alpm_objlib@sta/lib_libalpm_sync.c.o' 'alpm_objlib@sta/lib_libalpm_trans.c.o' 'alpm_objlib@sta/lib_libalpm_util.c.o' 'alpm_objlib@sta/lib_libalpm_version.c.o' -Wl,--as-needed -Wl,--no-undefined -shared -fPIC -Wl,--start-group -Wl,-soname,libalpm.so.12 /usr/lib/libcrypto.so /usr/lib/libarchive.so /usr/lib32/libcurl.so -lgpgme -lassuan -lgpg-error -Wl,--end-group
/usr/bin/ld.gold: error: /usr/lib32/libcurl.so: incompatible target
```

Using the boring old non-gold linker  I get the error message:
```
/usr/bin/ld: /usr/lib32/libcurl.so: error adding symbols: file in wrong format
````
This happens when boost is installed for multiple ABIs (eg. 32-bit and 64-bit). The 32-bit boost libraries end up in either /usr/lib or /usr/lib32, and the 64-bit libraries are in /usr/lib64.

The new boost dependency code has a very naive algorithm for finding valid libdirs; it basically just uses anything that starts with "lib". There is probably some better code elsewhere in meson to enumerate valid libdirs for the host toolchain.

Resolving patch:
diff --git a/mesonbuild/dependencies/boost.py b/mesonbuild/dependencies/boost.py
--- a/mesonbuild/dependencies/boost.py
+++ b/mesonbuild/dependencies/boost.py
@@ -412,10 +412,19 @@ def __init__(self, environment: Environment, kwargs):
                 break
 
     def run_check(self, inc_dirs: T.List[BoostIncludeDir], lib_dirs: T.List[Path]) -> bool:
+        mlog.debug('  - potential library dirs: {}'.format([x.as_posix() for x in lib_dirs]))
+        mlog.debug('  - potential include dirs: {}'.format([x.path.as_posix() for x in inc_dirs]))
+
         #   2. Find all boost libraries
         libs = []  # type: T.List[BoostLibraryFile]
         for i in lib_dirs:
-            libs += self.detect_libraries(i)
+            libs = self.detect_libraries(i)
+            if libs:
+                mlog.debug('  - found boost library dir: {}'.format(i))
+                # mlog.debug('  - raw library list:')
+                # for j in libs:
+                #     mlog.debug('    - {}'.format(j))
+                break
         libs = sorted(set(libs))
 
         modules = ['boost_' + x for x in self.modules]
@@ -423,9 +432,6 @@ def run_check(self, inc_dirs: T.List[BoostIncludeDir], lib_dirs: T.List[Path]) -
             mlog.debug('  - found boost {} include dir: {}'.format(inc.version, inc.path))
             f_libs = self.filter_libraries(libs, inc.version_lib)
 
-            # mlog.debug('  - raw library list:')
-            # for j in libs:
-            #     mlog.debug('    - {}'.format(j))
             mlog.debug('  - filtered library list:')
             for j in f_libs:
                 mlog.debug('    - {}'.format(j))
@@ -500,6 +506,19 @@ def detect_inc_dirs(self, root: Path) -> T.List[BoostIncludeDir]:
         return [self._include_dir_from_version_header(x) for x in candidates]
 
     def detect_lib_dirs(self, root: Path) -> T.List[Path]:
+        # First check the system include paths. Only consider those within the
+        # given root path
+        system_dirs_t = self.clib_compiler.get_library_dirs(self.env)
+        system_dirs = [Path(x) for x in system_dirs_t]
+        system_dirs = [x.resolve() for x in system_dirs if x.exists()]
+        system_dirs = [x for x in system_dirs if mesonlib.path_is_in_root(x, root)]
+        system_dirs = list(mesonlib.OrderedSet(system_dirs))
+
+        if system_dirs:
+            return system_dirs
+
+        # No system include paths were found --> fall back to manually looking
+        # for library dirs in root
         dirs = []     # type: T.List[Path]
         subdirs = []  # type: T.List[Path]
         for i in root.iterdir():
@@ -511,7 +530,25 @@ def detect_lib_dirs(self, root: Path) -> T.List[Path]:
             for j in i.iterdir():
                 if j.is_dir() and j.name.endswith('-linux-gnu'):
                     subdirs += [j]
-        return dirs + subdirs
+
+        # Filter out paths that don't match the target arch to avoid finding
+        # the wrong libraries. See https://github.com/mesonbuild/meson/issues/7110
+        if not self.arch:
+            return dirs + subdirs
+
+        arch_list_32 = ['32', 'i386']
+        arch_list_64 = ['64']
+
+        raw_list = dirs + subdirs
+        no_arch = [x for x in raw_list if not any([y in x.name for y in arch_list_32 + arch_list_64])]
+
+        matching_arch = []  # type: T.List[Path]
+        if '32' in self.arch:
+            matching_arch = [x for x in raw_list if any([y in x.name for y in arch_list_32])]
+        elif '64' in self.arch:
+            matching_arch = [x for x in raw_list if any([y in x.name for y in arch_list_64])]
+
+        return sorted(matching_arch) + sorted(no_arch)
 
     def filter_libraries(self, libs: T.List[BoostLibraryFile], lib_vers: str) -> T.List[BoostLibraryFile]:
         # MSVC is very picky with the library tags
diff --git a/mesonbuild/mesonlib.py b/mesonbuild/mesonlib.py
--- a/mesonbuild/mesonlib.py
+++ b/mesonbuild/mesonlib.py
@@ -1533,6 +1533,16 @@ def relpath(path: str, start: str) -> str:
     except (TypeError, ValueError):
         return path
 
+def path_is_in_root(path: Path, root: Path, resolve: bool = False) -> bool:
+    # Check wheter a path is within the root directory root
+    try:
+        if resolve:
+            path.resolve().relative_to(root.resolve())
+        else:
+            path.relative_to(root)
+    except ValueError:
+        return False
+    return True
 
 class LibType(Enum):
 


Test patch: