ImportError: undefined symbol

username: Picus
cooked: <p>Hi! I’m trying to make my first Pytorch extension!<br>
I use setuptools, pybind and cmake to do it.</p>
<p>The compilation with <code>python setup.py install</code> works fine but at execution time, I get this error that I’ve never seen before: <code>ImportError: &lt;path_to_the_lib_so_file&gt;: undefined symbol: _ZN8pybind116detail11type_casterIN2at6TensorEvE4loadENS_6handleEb</code></p>
<p>Does anyone have a clue of what it even means?<br>
Here is the code:</p>
<p>my_extension.cpp</p>
<pre data-code-wrap="cpp"><code class="lang-cpp">#include &lt;torch/extension.h&gt;
#include &lt;pybind11/pybind11.h&gt;

torch::Tensor multiply_by_two(torch::Tensor input) {
    return input * 2;
}

PYBIND11_MODULE(my_extension, m) {
    m.def("multiply_by_two", &amp;multiply_by_two, "A function that multiplies each element in a tensor by 2");
}
</code></pre>
<p>CMakelists.txt</p>
<pre data-code-wrap="cmake"><code class="lang-cmake">cmake_minimum_required(VERSION 3.5)
project(my_extension LANGUAGES CXX)

// Doesn't work without this for some reason
set(CMAKE_PREFIX_PATH "/home/x00/anaconda3/envs/retro/lib/python3.10/site-packages/torch")

find_package(pybind11 REQUIRED)
find_package(Torch REQUIRED)

pybind11_add_module(my_extension my_extension.cpp)

target_link_libraries(my_extension PRIVATE ${TORCH_LIBRARIES} ${CMAKE_DL_LIBS})
target_include_directories(my_extension PRIVATE ${TORCH_INCLUDE_DIRS})
</code></pre>

username: gauenk
cooked: <p>The easiest thing is to not use CMake, but rather let setuptools do the compiling. So your command will be <code>python -m pip install -e .</code> (like you are already doing), but you’ll need to create a setup.py file by following <a href="https://pytorch.org/tutorials/advanced/cpp_extension.html" rel="noopener nofollow ugc">the docs</a>. <a href="https://github.com/gauenk/spix_paper/blob/main/setup.py" rel="noopener nofollow ugc">Here is an example of mine for reference.</a></p>
<p>However, this is dreadfully slow so I’ve been looking for a way to use CMake instead. Setuptools seems to re-compile the entire project each time (which is slow), and it is much easier to use ccache to speed up the compiling process when using CMake. The speed-up of ccache is significant, as it brings a compile time of several minutes down to only 2 - 4 seconds.</p>
<p>I stumbled on this thread since I am having linking errors like you with pybind11. So if I solve the issue and remember, I will post my findings back here.</p>

username: gauenk
cooked: <p>Okay, so I’ve figured out how to speed up compiling using setuptools and not cmake. At the top of your setup.py file add the following few lines:</p>
<pre><code class="lang-auto">import os
os.environ['PYTORCH_NVCC'] = "ccache nvcc"
os.environ['TORCH_EXTENSION_SKIP_NVCC_GEN_DEPENDENCIES'] = '1'
</code></pre>
<p>To make this work, you may have to compile ccache from source since the “apt-get” version seems to be out of date. Then I only link the built ccache to the c++ compilers:</p>
<pre><code class="lang-auto">ln -s ccache /usr/local/bin/gcc
ln -s ccache /usr/local/bin/g++
ln -s ccache /usr/local/bin/c++
</code></pre>
<p>Hopefully it goes smoothly for you as well.</p>

username: Picus
cooked: <p>Thanks, but I’m not sure I can use that.</p>
<p>See, this is my first try but, in the future, I plan to manually access to the content of tensors on the GPU with ROCm. For that, I need to compile my code using hipcc. With CMake if would just need to use <code>project(my_extension LANGUAGES HIP)</code> but I don’t think I can change the compiler used by setuptools, can I?</p>

username: Picus
cooked: <p>Update: it doesn’t work.</p>
<p>I have to choose between <code>undefined symbol</code> or Pytorch crying because I’m not using gcc or g++. I tried to bypass this check but the error that occures after that it’s impossible to debug for me.</p>

username: gauenk
cooked: <p>I am probably not much help since I haven’t worked with rocm. The following link suggests it works on at least one setup:</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://rocm.blogs.amd.com/artificial-intelligence/cpp-extn/readme.html">
  <header class="source">
      <img src="https://www.amd.com/themes/custom/amd/favicon.ico" class="site-icon" width="" height="">

      <a href="https://rocm.blogs.amd.com/artificial-intelligence/cpp-extn/readme.html" target="_blank" rel="noopener nofollow ugc">rocm.blogs.amd.com</a>
  </header>

  <article class="onebox-body">
    

<h3><a href="https://rocm.blogs.amd.com/artificial-intelligence/cpp-extn/readme.html" target="_blank" rel="noopener nofollow ugc">PyTorch C++ Extension on AMD GPU — ROCm Blogs</a></h3>

  <p>PyTorch C++ Extension on AMD hardware</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>There is also an open issue on Pytorch:</p>
<aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/tutorials/issues/2342">
  <header class="source">

      <a href="https://github.com/pytorch/tutorials/issues/2342" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/tutorials</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue" data-github-private-repo="false">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/tutorials/issues/2342" target="_blank" rel="noopener nofollow ugc">Custom rocm hip and c++ extensions</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2022-12-31" data-time="17:12:20" data-timezone="UTC">05:12PM - 31 Dec 22 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/salykova" target="_blank" rel="noopener nofollow ugc">
          <img alt="salykova" src="https://avatars.githubusercontent.com/u/63703454?v=4" class="onebox-avatar-inline" width="20" height="20">
          salykova
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          triaged
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: rocm
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          advanced
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          amd
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          docathon-h2-2023
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          ciflow/rocm
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">### 🚀 The feature, motivation and pitch

Dear PyTorch developers and community<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">,

We have nice tutorial [cpp_extension](https://pytorch.org/tutorials/advanced/cpp_extension.html) on custom cuda extensions written by Peter Goldsborough. I’m wondering if the same can be done but on AMD GPUs with kernels written using rocm HIP. I mean the following: call custom forward+backward hip kernel from pytorch and include it in deep learning pipeline. Is it currently supported and are there any limitations?

Does somebody have experience of writing custom hip/c++ kernels and using them in pytorch?


cc @sekyondaMeta @svekars @carljparker @NicolasHug @kit1980 @subramen @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Best wishes</p>