Issues about torch::deploy demo

JiayiFeng: <p>I’m trying to run the demo in <a href="https://pytorch.org/docs/stable/deploy.html" class="inline-onebox" rel="noopener nofollow ugc">torch::deploy — PyTorch 1.11.0 documentation</a>, but I have some problems during compiling and running the cpp code in the section “Loading and running the model in C++”</p>
<p>During compiling with the <code>CMakeLists.txt</code> provided in the section “Building and running the application”, I encountered this error message:</p>
<pre><code class="lang-auto">(torch-deploy) root@0e5fa48bd947:~/torch-deploy_test/example-app/build# cmake --build . --config Release
[ 50%] Building CXX object CMakeFiles/example-app.dir/example-app.cpp.o
[100%] Linking CXX executable example-app
/usr/bin/ld: CMakeFiles/example-app.dir/example-app.cpp.o: in function `main':
example-app.cpp:(.text+0x2eb): undefined reference to `torch::deploy::InterpreterManager::InterpreterManager(unsigned long, std::shared_ptr&lt;torch::deploy::Environment&gt;)'
/usr/bin/ld: example-app.cpp:(.text+0x359): undefined reference to `torch::deploy::InterpreterManager::loadPackage(std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;)'
/usr/bin/ld: CMakeFiles/example-app.dir/example-app.cpp.o: in function `torch::deploy::InterpreterManager::acquireOne()':
example-app.cpp:(.text._ZN5torch6deploy18InterpreterManager10acquireOneEv[_ZN5torch6deploy18InterpreterManager10acquireOneEv]+0x3e): undefined reference to `torch::deploy::LoadBalancer::acquire()'
/usr/bin/ld: example-app.cpp:(.text._ZN5torch6deploy18InterpreterManager10acquireOneEv[_ZN5torch6deploy18InterpreterManager10acquireOneEv]+0xab): undefined reference to `torch::deploy::InterpreterSession::~InterpreterSession()'
/usr/bin/ld: CMakeFiles/example-app.dir/example-app.cpp.o: in function `torch::deploy::Package::loadPickle(std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;)':
example-app.cpp:(.text._ZN5torch6deploy7Package10loadPickleERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES9_[_ZN5torch6deploy7Package10loadPickleERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES9_]+0x198): undefined reference to `torch::deploy::InterpreterSession::createMovable(torch::deploy::Obj)'
/usr/bin/ld: example-app.cpp:(.text._ZN5torch6deploy7Package10loadPickleERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES9_[_ZN5torch6deploy7Package10loadPickleERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES9_]+0x1a8): undefined reference to `torch::deploy::InterpreterSession::~InterpreterSession()'
/usr/bin/ld: example-app.cpp:(.text._ZN5torch6deploy7Package10loadPickleERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES9_[_ZN5torch6deploy7Package10loadPickleERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES9_]+0x23e): undefined reference to `torch::deploy::InterpreterSession::~InterpreterSession()'
/usr/bin/ld: CMakeFiles/example-app.dir/example-app.cpp.o: in function `torch::deploy::Package::acquireSession()':
example-app.cpp:(.text._ZN5torch6deploy7Package14acquireSessionEv[_ZN5torch6deploy7Package14acquireSessionEv]+0xad): undefined reference to `torch::deploy::InterpreterSession::~InterpreterSession()'
/usr/bin/ld: example-app.cpp:(.text._ZN5torch6deploy7Package14acquireSessionEv[_ZN5torch6deploy7Package14acquireSessionEv]+0xdf): undefined reference to `torch::deploy::InterpreterSession::~InterpreterSession()'
/usr/bin/ld: CMakeFiles/example-app.dir/example-app.cpp.o: in function `void std::_Destroy&lt;torch::deploy::Interpreter&gt;(torch::deploy::Interpreter*)':
example-app.cpp:(.text._ZSt8_DestroyIN5torch6deploy11InterpreterEEvPT_[_ZSt8_DestroyIN5torch6deploy11InterpreterEEvPT_]+0x18): undefined reference to `torch::deploy::Interpreter::~Interpreter()'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/example-app.dir/build.make:112: example-app] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/example-app.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
</code></pre>
<p>The linker cannot find some pytorch::deploy related symbols, and seems that they are not contained by any .so file in the <code>${conda_env}/lib/python3.9/site-packages/torch</code> dir. Then I search the pytorch’s <code>build/lib</code> dir and find these symbols in the <code>libtorch_deploy_internal.a</code>. So I add this line into the <code>CMakeLists.txt</code>:</p>
<pre><code class="lang-auto">target_link_libraries(example-app ${my_pytorch_code_dir}/build/lib/libtorch_deploy_internal.a dl)
</code></pre>
<p>Then it compiles successfully. But when I try to run it and load a model package, it crashes and throws this error message:</p>
<pre><code class="lang-auto">terminate called after throwing an instance of 'std::runtime_error'
  what():  Exception Caught inside torch::deploy embedded library:
Check failed: (libStart != nullptr &amp;&amp; libEnd != nullptr), function writeDeployInterpreter, file /pytorch/torch/csrc/deploy/deploy.cpp, line 80.
torch::deploy requires a build-time dependency on embedded_interpreter or embedded_interpreter_cuda, neither of which were found.  torch::cuda::is_available()=1

Aborted (core dumped)
</code></pre>
<p>I do all above tests in <code>nvidia/cuda:11.6.0-devel-ubuntu20.04</code> docker container. The torch::deploy is installed follow the instruction in <a href="https://pytorch.org/docs/stable/deploy.html#installing-torch-deploy" class="inline-onebox" rel="noopener nofollow ugc">torch::deploy — PyTorch 1.11.0 documentation</a>. The PyTorch code version is v1.12.0-rc7</p>
<p>I have no idea about what is the root cause and how to solve it. Can anyone help? Many thanks!</p>

s4ayub: <p>Follow: <a href="https://github.com/pytorch/pytorch/blob/master/docs/source/deploy.rst" class="inline-onebox" rel="noopener nofollow ugc">pytorch/deploy.rst at master · pytorch/pytorch · GitHub</a><br>
the v1.11 pytorch website docs are outdated</p>
<p>the v1.12+ pytorch website docs have it: <a href="https://pytorch.org/docs/1.12/deploy.html?highlight=deploy" class="inline-onebox" rel="noopener nofollow ugc">torch::deploy — PyTorch 1.12 documentation</a>,</p>