How does one use LibTorch (C++) with the Arch64 build of PyTorch for the Nvidia Jetson?

username: Alio
cooked: <p>I’ve successfully installed PyTorch v1.10.0 on my Nvidia Jetson but I would like to use the C++ / Libtorch portion of the install.<br>
The “wheels” specifically for the Nvidia Jetsons are <a href="https://forums.developer.nvidia.com/t/pytorch-for-jetson-version-1-10-now-available/72048" rel="noopener nofollow ugc">here</a></p>
<p>On my Jetson (AGX Xavier) I believe Libtorch is installed in this path<br>
~/.local/lib/python3.6/site-packages/torch/lib</p>
<p>But I don’t know how to make use of anything in this path. I would like to use it to run a Libtorch script in C++. This is a list of all the files in that path.<br>
<code>libbackend_with_compiler.so, libcaffe2_detectron_ops_gpu.so, libcaffe2_observers.so, libtorchbind_test.so, libtorch_global_deps.so, libc10_cuda.so, libcaffe2_module_test_dynamic.so, libjitbackend_test.so, libtorch_cpu.so       libtorch_python.so, libc10.so, libcaffe2_nvrtc.so, libshm.so, libtorch_cuda.so, libtorch.so</code></p>
<p>Could I specifically use the <code>libtorch.so</code> to compile my LibTorch script somehow?</p>
<p>Below: I’m simply trying to compile the example script for Libtorch from the <a href="https://pytorch.org/cppdocs/installing.html" rel="noopener nofollow ugc">official site</a><br>
I am following the instructions on the page exactly and for /absolute/path/to/libtorch I put the path I mentioned above, ~/.local/lib/python3.6/site-packages/torch/lib.</p>
<p>I need to build the Libtorch script with cmake and I’m getting the error…</p>
<pre><code class="lang-auto">
$ cmake -DCMAKE_PREFIX_PATH=/home/jet/.local/lib/python3.6/site-packages/torch/lib ..
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:4 (find_package):
  By not providing "FindTorch.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Torch", but
  CMake did not find one.


  Could not find a package configuration file provided by "Torch" with any of
  the following names:

    TorchConfig.cmake
    torch-config.cmake

  Add the installation prefix of "Torch" to CMAKE_PREFIX_PATH or set
  "Torch_DIR" to a directory containing one of the above files.  If "Torch"
  provides a separate development package or SDK, be sure it has been
  installed.


-- Configuring incomplete, errors occurred!
</code></pre>
<p>This is my CMakeLists.txt</p>
<pre><code class="lang-auto">
cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
project(example-app)

find_package(Torch REQUIRED)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")

add_executable(example-app example-app.cpp)
target_link_libraries(example-app "${TORCH_LIBRARIES}")
set_property(TARGET example-app PROPERTY CXX_STANDARD 14)

# The following code block is suggested to be used on Windows.
# According to https://github.com/pytorch/pytorch/issues/25457,
# the DLLs need to be copied to avoid memory errors.
if (MSVC)
  file(GLOB TORCH_DLLS "${TORCH_INSTALL_PREFIX}/lib/*.dll")
  add_custom_command(TARGET example-app
                     POST_BUILD
                     COMMAND ${CMAKE_COMMAND} -E copy_if_different
                     ${TORCH_DLLS}
                     $&lt;TARGET_FILE_DIR:example-app&gt;)
endif (MSVC)
</code></pre>

username: ptrblck
cooked: <p>I haven’t checked it, but maybe try to use the built-in <code>cmake_prefix_path</code> instead of providing it manually via:</p>
<pre><code class="lang-python">-DCMAKE_PREFIX_PATH=`python -c 'import torch;print(torch.utils.cmake_prefix_path)'`
</code></pre>

username: Alio
cooked: <p>I tried to build that way. I get errors and no build. <a href="https://pytorch.org/cppdocs/installing.html" rel="noopener nofollow ugc">I am trying to build the official LibTorch example</a>.</p>
<p>I wrote ‘python3’ instead of just python because I am able to import torch when I open python3.</p>
<p>The cmake command</p>
<pre><code class="lang-auto">cmake -DCMAKE_PREFIX_PATH=`python3 -c 'import torch;print(torch.utils.cmake_prefix_path)'` ..
-- The C compiler identification is Clang 6.0.0
-- The CXX compiler identification is Clang 6.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/lib/ccache/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/lib/ccache/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found CUDA: /usr/local/cuda-10.2 (found version "10.2")
-- Caffe2: CUDA detected: 10.2
-- Caffe2: CUDA nvcc is: /usr/local/cuda-10.2/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda-10.2
-- Caffe2: Header version is: 10.2
-- Found CUDNN: /usr/lib/aarch64-linux-gnu/libcudnn.so
-- Found cuDNN: v8.2.1  (include: /usr/include, library: /usr/lib/aarch64-linux-gnu/libcudnn.so)
-- /usr/local/cuda-10.2/lib64/libnvrtc.so shorthash is 7d272a04
CMake Warning at /home/jet/.local/lib/python3.6/site-packages/torch/share/cmake/Caffe2/public/utils.cmake:365 (message):
  In the future we will require one to explicitly pass TORCH_CUDA_ARCH_LIST
  to cmake instead of implicitly setting it as an env variable.  This will
  become a FATAL_ERROR in future version of pytorch.
Call Stack (most recent call first):
  /home/jet/.local/lib/python3.6/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:437 (torch_cuda_get_nvcc_gencode_flag)
  /home/jet/.local/lib/python3.6/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:88 (include)
  /home/jet/.local/lib/python3.6/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  CMakeLists.txt:4 (find_package)
-- Added CUDA NVCC flags for: -gencode;arch=compute_53,code=sm_53;-gencode;arch=compute_62,code=sm_62;-gencode;arch=compute_72,code=sm_72
CMake Warning at /home/jet/.local/lib/python3.6/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
  static library kineto_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
  /home/jet/.local/lib/python3.6/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found)
  CMakeLists.txt:4 (find_package)
-- Found Torch: /home/jet/.local/lib/python3.6/site-packages/torch/lib/libtorch.so
-- Configuring done
-- Generating done
-- Build files have been written to: /home/jet/example-app/build
</code></pre>
<p>The cmake build command</p>
<pre><code class="lang-auto">$cmake --build . --config Release
[ 50%] Building CXX object CMakeFiles/example-app.dir/example-app.cpp.o
[100%] Linking CXX executable example-app
/usr/bin/ld: cannot find -lCUDA_cublas_LIBRARY-NOTFOUND
clang: error: linker command failed with exit code 1 (use -v to see invocation)
CMakeFiles/example-app.dir/build.make:110: recipe for target 'example-app' failed
make[2]: *** [example-app] Error 1
CMakeFiles/Makefile2:82: recipe for target 'CMakeFiles/example-app.dir/all' failed
make[1]: *** [CMakeFiles/example-app.dir/all] Error 2
Makefile:90: recipe for target 'all' failed
make: *** [all] Error 2
</code></pre>

username: edgarriba
cooked: <p>Use this reference <a href="https://github.com/pytorch/pytorch/blob/master/docs/libtorch.rst#building-libtorch-using-cmake" rel="noopener nofollow ugc">https://github.com/pytorch/pytorch/blob/master/docs/libtorch.rst#building-libtorch-using-cmake</a></p>

username: lei_li2
cooked: <p>There is a way to use Libtorch on the nvidia jetson platform.</p>
<p>The Pytorch official has not provided an aarch64 Libtorch package for downloading, It only provides an x86-64 Libtorch package。</p>
<p>In the Nvidia Jetson platform, Nvidia provides the PyTorch python package, which contains Libtorch in it.<br>
So, We can use Libtorch by installing the Pytorch Python package.</p>
<h2>
<a name="firstly-you-should-download-pytorch-for-jetson-on-below-link-1" class="anchor" href="#firstly-you-should-download-pytorch-for-jetson-on-below-link-1"></a><strong>Firstly, you should download PyTorch for Jetson on below link</strong>
</h2>
<p><a href="https://forums.developer.nvidia.com/t/pytorch-for-jetson/72048" rel="noopener nofollow ugc">https://forums.developer.nvidia.com/t/pytorch-for-jetson/72048</a></p>
<h2>
<a name="secondly-install-the-whl-file-following-this-guide-2" class="anchor" href="#secondly-install-the-whl-file-following-this-guide-2"></a><strong>Secondly, install the “whl” file following this guide</strong>
</h2>
<p>[<a href="https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html" class="inline-onebox" rel="noopener nofollow ugc">Installing PyTorch for Jetson Platform - NVIDIA Docs</a>]</p>
<p>After you finished the above steps. The LibTorch has been installed on your device.</p>
<h2>
<a name="how-to-use-3" class="anchor" href="#how-to-use-3"></a><strong>How to Use?</strong>
</h2>
<h3>
<a name="where-is-libtorch-installed-4" class="anchor" href="#where-is-libtorch-installed-4"></a><strong>where is LibTorch installed?</strong>
</h3>
<p>Commonly, it installs at ~/.local/lib/python3.6/site-packages/torch/<br>
In this dir, you can see bin, lib, and include dir, It is organized just like other C++ libraries.</p>
<h3>
<a name="how-to-use-libtorch-with-cmake-5" class="anchor" href="#how-to-use-libtorch-with-cmake-5"></a><strong>How to use LibTorch with CMake?</strong>
</h3>
<p>for example, add these two lines to your CMakeLists.txt</p>
<pre><code class="lang-auto">// set PyTorch path
set(Torch_DIR /home/nx/.local/lib/python3.6/site-packages/torch/share/cmake/Torch)
// use find_package to use libtorch
find_package(Torch REQUIRED)
</code></pre>