Repository: easybuilders/easybuild-framework

Issue:
CUDA stubs directory must not be added to rpath/runpath
This is a follow up on https://github.com/easybuilders/easybuild-easyblocks/pull/1464 

 adding stubs to LIBRARY_PATH is causing us problems, because it ends up in our binaries RUNPATH.

If this happens and you don't use LD_LIBRARY_PATH (because you are using RUNPATH), you will have problems at runtime.

This is probably something you need to check when you are using runpath/rpath with easybuild (for Compute Canada, @bartoldeman had to adjust our linker wrapper to exclude these paths)

```
A stub object is a shared object, built entirely from mapfiles, that supplies the same linking interface as the real object, while containing no code or data. Stub objects cannot be used at runtime.
``` 




Discussion:
@mboisson How are these paths excluded in your linker wrapper exactly? Just based on directory name?
@bartoldeman would be best to answer that question. 
yes, as follows:
```
        # also exclude stub libraries like for CUDA
        if [ "${1%%/stubs}" != "$1" ]; then
            return 0
        fi
```
it's not beautiful, but it's unlikely that  a directory named "stubs" contains non-stub libraries I would think.
Are there any other cases known besides `CUDA`? If not, we can consider making the check a bit more strict to also check on `CUDA` in the path name...
I don't know of any others... in fact if I google `/usr/lib64/stubs` I only find references to cuda.
How does #2725 look by means of a fix?
fixed via #2725
Note that this issue has resurfaced since https://github.com/easybuilders/easybuild-easyblocks/pull/2373 since the stubs libs are now in `$EBROOTCUDA/stubs/lib64`.
@bartoldeman So we need to extend the filtering done in #2725 to `CUDA/.*/stubs/lib64` or something?
@boegel exactly


Resolving patch:
diff --git a/easybuild/tools/toolchain/toolchain.py b/easybuild/tools/toolchain/toolchain.py
--- a/easybuild/tools/toolchain/toolchain.py
+++ b/easybuild/tools/toolchain/toolchain.py
@@ -960,9 +960,11 @@ def prepare_rpath_wrappers(self, rpath_filter_dirs=None, rpath_include_dirs=None
 
         # always include filter for 'stubs' library directory,
         # cfr. https://github.com/easybuilders/easybuild-framework/issues/2683
-        lib_stubs_pattern = '.*/lib(64)?/stubs/?'
-        if lib_stubs_pattern not in rpath_filter_dirs:
-            rpath_filter_dirs.append(lib_stubs_pattern)
+        # (since CUDA 11.something the stubs are in $EBROOTCUDA/stubs/lib64)
+        lib_stubs_patterns = ['.*/lib(64)?/stubs/?', '.*/stubs/lib(64)?/?']
+        for lib_stubs_pattern in lib_stubs_patterns:
+            if lib_stubs_pattern not in rpath_filter_dirs:
+                rpath_filter_dirs.append(lib_stubs_pattern)
 
         # directory where all wrappers will be placed
         wrappers_dir = os.path.join(tempfile.mkdtemp(), RPATH_WRAPPERS_SUBDIR)


Test patch:
diff --git a/test/framework/toolchain.py b/test/framework/toolchain.py
--- a/test/framework/toolchain.py
+++ b/test/framework/toolchain.py
@@ -2437,12 +2437,16 @@ def test_toolchain_prepare_rpath(self):
         # check whether 'stubs' library directory are correctly filtered out
         paths = [
             'prefix/software/CUDA/1.2.3/lib/stubs/',  # should be filtered (no -rpath)
+            'prefix/software/CUDA/1.2.3/stubs/lib/',  # should be filtered (no -rpath)
             'tmp/foo/',
             'prefix/software/stubs/1.2.3/lib',  # should NOT be filtered
             'prefix/software/CUDA/1.2.3/lib/stubs',  # should be filtered (no -rpath)
+            'prefix/software/CUDA/1.2.3/stubs/lib',  # should be filtered (no -rpath)
             'prefix/software/CUDA/1.2.3/lib64/stubs/',  # should be filtered (no -rpath)
+            'prefix/software/CUDA/1.2.3/stubs/lib64/',  # should be filtered (no -rpath)
             'prefix/software/foobar/4.5/notreallystubs',  # should NOT be filtered
             'prefix/software/CUDA/1.2.3/lib64/stubs',  # should be filtered (no -rpath)
+            'prefix/software/CUDA/1.2.3/stubs/lib64',  # should be filtered (no -rpath)
             'prefix/software/zlib/1.2.11/lib',
             'prefix/software/bleh/0/lib/stubs',  # should be filtered (no -rpath)
             'prefix/software/foobar/4.5/stubsbutnotreally',  # should NOT be filtered
@@ -2465,12 +2469,16 @@ def test_toolchain_prepare_rpath(self):
             '-Wl,-rpath=%s/prefix/software/foobar/4.5/stubsbutnotreally' % self.test_prefix,
             '%(user)s.c',
             '-L%s/prefix/software/CUDA/1.2.3/lib/stubs/' % self.test_prefix,
+            '-L%s/prefix/software/CUDA/1.2.3/stubs/lib/' % self.test_prefix,
             '-L%s/tmp/foo/' % self.test_prefix,
             '-L%s/prefix/software/stubs/1.2.3/lib' % self.test_prefix,
             '-L%s/prefix/software/CUDA/1.2.3/lib/stubs' % self.test_prefix,
+            '-L%s/prefix/software/CUDA/1.2.3/stubs/lib' % self.test_prefix,
             '-L%s/prefix/software/CUDA/1.2.3/lib64/stubs/' % self.test_prefix,
+            '-L%s/prefix/software/CUDA/1.2.3/stubs/lib64/' % self.test_prefix,
             '-L%s/prefix/software/foobar/4.5/notreallystubs' % self.test_prefix,
             '-L%s/prefix/software/CUDA/1.2.3/lib64/stubs' % self.test_prefix,
+            '-L%s/prefix/software/CUDA/1.2.3/stubs/lib64' % self.test_prefix,
             '-L%s/prefix/software/zlib/1.2.11/lib' % self.test_prefix,
             '-L%s/prefix/software/bleh/0/lib/stubs' % self.test_prefix,
             '-L%s/prefix/software/foobar/4.5/stubsbutnotreally' % self.test_prefix,