Libtorch.so not found in a shared library when compiling with g++ and a custom compilation command

tmaric: <p>I’m combining libtorch with <a href="https://www.openfoam.com/documentation/guides/latest/doc/" rel="noopener nofollow ugc">OpenFOAM</a> and when I install libtorch as a package on Linux (Arch or Ubuntu), the libtorch.so is found as are the related headers, but if I download the C++ API archive and unzip it in a separate directory, then the “-I” option of “g++” makes it possible to find the headers, but the “-L” option doesn’t seem to find “libtorch.so”.</p>
<p>I’m using <a href="https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-1.7.1%2Bcpu.zip" rel="noopener nofollow ugc">libtorch-cxx11-abi-shared-with-deps-1.7.1+cpu.zip</a>, and g++ (GCC) 10.2.0.</p>
<p>Here is the g++ command for the library, it is rather long, so I snipped the above part, let me know if I should upload everything:</p>
<pre><code class="lang-auto">g++ -std=c++2a -m64 -pthread -DOPENFOAM=2012 -DWM_DP -DWM_LABEL_SIZE=32 -Wall -Wextra -Wold-style-cast -Wnon-virtual-dtor -Wno-unused-parameter -Wno-invalid-offsetof -Wno-attributes -Wno-unknown-pragmas  -O3  -DNoRepository -ftemplate-depth-100 -std=c++2a -DAI_DEBUG -Wno-deprecated-declarations -Wno-old-style-cast -Wno-redundant-move -I./lnInclude -I/home/tm/OpenFOAM/openfoam/src/fileFormats/lnInclude -I/home/tm/OpenFOAM/openfoam/src/surfMesh/lnInclude -I/home/tm/OpenFOAM/openfoam/src/meshTools/lnInclude -I/home/tm/OpenFOAM/openfoam/src/finiteVolume/lnInclude -I/home/tm/Projects/research/openfoam/openfoam-ml/libtorch/include -I/home/tm/Projects/research/openfoam/openfoam-ml/libtorch/include/torch/csrc/api/include  -iquote. -IlnInclude -I/home/tm/OpenFOAM/openfoam/src/OpenFOAM/lnInclude -I/home/tm/OpenFOAM/openfoam/src/OSspecific/POSIX/lnInclude   -fPIC -shared -Xlinker --add-needed -Xlinker --no-as-needed  Make/linux64GccDPInt32Opt/torchFunctions.o Make/linux64GccDPInt32Opt/mlpFieldApproximator/mlpFieldApproximator.o Make/linux64GccDPInt32Opt/isoMlpFieldApproximator/isoMlpFieldApproximator.o Make/linux64GccDPInt32Opt/pcMLPsurface/pcMLPsurface.o -L/home/tm/OpenFOAM/openfoam/platforms/linux64GccDPInt32Opt/lib \
    -L/home/tm/Projects/research/openfoam/openfoam-ml/libtorch/lib -lOpenFOAM -lfileFormats -lsurfMesh -lmeshTools -lfiniteVolume -ltorch  -o /home/tm/OpenFOAM/tm-v2012/platforms/linux64GccDPInt32Opt/lib/libmlFieldApproximation.so
</code></pre>
<p>To show that the -L folder does contain the libraries:</p>
<pre><code class="lang-auto">?&gt; ls /home/tm/Projects/research/openfoam/openfoam-ml/libtorch/lib
libasmjit.a                       libCaffe2_perfkernels_avx.a  libfoxi_loader.a       libnnpack_reference_layers.a  libshm.so
libbenchmark.a                    libcaffe2_protos.a           libgloo.a              libonnx.a                     libtensorpipe.a
libbenchmark_main.a               libclog.a                    libgmock.a             libonnx_proto.a               libtensorpipe_agent.so
libc10d.a                         libcpuinfo.a                 libgmock_main.a        libprocess_group_agent.so     libtensorpipe_uv.a
libc10.so                         libcpuinfo_internals.a       libgomp-75eea7e8.so.1  libprotobuf.a                 libtorchbind_test.so
libcaffe2_detectron_ops.so        libdnnl.a                    libgtest.a             libprotobuf-lite.a            libtorch_cpu.so
libcaffe2_module_test_dynamic.so  libfbgemm.a                  libgtest_main.a        libprotoc.a                   libtorch_global_deps.so
libcaffe2_observers.so            libfmt.so                    libjitbackend_test.so  libpthreadpool.a              libtorch_python.so
libCaffe2_perfkernels_avx2.a      libfmt.so.7                  libmkldnn.a            libpytorch_qnnpack.a          libtorch.so
libCaffe2_perfkernels_avx512.a    libfmt.so.7.0.3              libnnpack.a            libqnnpack.a                  libXNNPACK.a
</code></pre>
<p>but when I inspect the library with ldd:</p>
<pre><code class="lang-auto">ldd $FOAM_USER_LIBBIN/libmlFieldApproximation.so 
	linux-vdso.so.1 (0x00007ffdddb71000)
	libOpenFOAM.so =&gt; /home/tm/OpenFOAM/openfoam/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so (0x00007f6e6bf8a000)
	libfileFormats.so =&gt; /home/tm/OpenFOAM/openfoam/platforms/linux64GccDPInt32Opt/lib/libfileFormats.so (0x00007f6e6bde9000)
	libsurfMesh.so =&gt; /home/tm/OpenFOAM/openfoam/platforms/linux64GccDPInt32Opt/lib/libsurfMesh.so (0x00007f6e6bbab000)
	libmeshTools.so =&gt; /home/tm/OpenFOAM/openfoam/platforms/linux64GccDPInt32Opt/lib/libmeshTools.so (0x00007f6e6b42d000)
	libfiniteVolume.so =&gt; /home/tm/OpenFOAM/openfoam/platforms/linux64GccDPInt32Opt/lib/libfiniteVolume.so (0x00007f6e68dfd000)
	libtorch.so =&gt; not found
	libstdc++.so.6 =&gt; /usr/lib/libstdc++.so.6 (0x00007f6e68bf2000)
	libm.so.6 =&gt; /usr/lib/libm.so.6 (0x00007f6e68aad000)
	libgcc_s.so.1 =&gt; /usr/lib/libgcc_s.so.1 (0x00007f6e68a93000)
	libpthread.so.0 =&gt; /usr/lib/libpthread.so.0 (0x00007f6e68a72000)
	libc.so.6 =&gt; /usr/lib/libc.so.6 (0x00007f6e688a3000)
	/usr/lib64/ld-linux-x86-64.so.2 (0x00007f6e6cb9c000)
	libPstream.so =&gt; /home/tm/OpenFOAM/openfoam/platforms/linux64GccDPInt32Opt/lib/sys-openmpi/libPstream.so (0x00007f6e68887000)
	libtorch_cpu.so =&gt; not found
	libc10.so =&gt; not found
	libz.so.1 =&gt; /usr/lib/libz.so.1 (0x00007f6e6886d000)
	libmpi.so.40 =&gt; /usr/lib/openmpi/libmpi.so.40 (0x00007f6e68744000)
	libopen-rte.so.40 =&gt; /usr/lib/openmpi/libopen-rte.so.40 (0x00007f6e6868c000)
	libopen-pal.so.40 =&gt; /usr/lib/openmpi/libopen-pal.so.40 (0x00007f6e685cc000)
	libdl.so.2 =&gt; /usr/lib/libdl.so.2 (0x00007f6e685c5000)
	librt.so.1 =&gt; /usr/lib/librt.so.1 (0x00007f6e685ba000)
	libutil.so.1 =&gt; /usr/lib/libutil.so.1 (0x00007f6e685b5000)
	libhwloc.so.15 =&gt; /usr/lib/libhwloc.so.15 (0x00007f6e6855a000)
	libudev.so.1 =&gt; /usr/lib/libudev.so.1 (0x00007f6e68532000)
</code></pre>
<p>g++ seems to find all the OpenFOAM libraries somewhere in the sub-folders of $HOME, and it works if I install libtorch as a package on the (Arch Linux) system. OF is not using CMake, but this should not play any role, as CMake basically defines the rules for g++, same as OF does it with its built in build system.</p>
<p>I’m not using CUDA (yet).</p>
<p>Not sure what else could play a role… any ideas why this is happening?</p>
<p>I can extend LD_LIBRARY_PATH, but is this a good idea?</p>