Importing CUDA extension, undefined symbol: nvrtcGetProgramLogSize

username: kasparas-k
cooked: <p>Hello,</p>
<p>I’ve been modifying a CUDA extension from the official LatticeNet repo (my fork link is coming, from which you can also find the original), so I could use it without installing all the other extra infrastructure packages I don’t need. I’ve managed to get it to the stage, where I can compile the extension and attempt to import it.</p>
<p>When I do import it after torch, I get the following error:</p>
<blockquote>
<p>/usr/local/lib/python3.9/dist-packages/latticenet.cpython-39-x86_64-linux-gnu.so: undefined symbol: nvrtcGetProgramLogSize</p>
</blockquote>
<p>From my searches, I found that this is a CUDA symbol, but I do not know how to interpret it being not found. CUDA is available on both machines I tried this on.</p>
<p>Here’s a link to my fork:</p><aside class="onebox githubfolder" data-onebox-src="https://github.com/kasparas-k/lattice_net_standalone/tree/only_cuda">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/kasparas-k/lattice_net_standalone/tree/only_cuda" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h3><a href="https://github.com/kasparas-k/lattice_net_standalone/tree/only_cuda" target="_blank" rel="noopener nofollow ugc">GitHub - kasparas-k/lattice_net_standalone at only_cuda</a></h3>

  <p><a href="https://github.com/kasparas-k/lattice_net_standalone/tree/only_cuda" target="_blank" rel="noopener nofollow ugc">only_cuda</a></p>

  <p><span class="label1">Fast Point Cloud Segmentation Using Permutohedral Lattices</span></p>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Here’s my setup.py script</p>
<pre><code class="lang-auto">import pathlib
from setuptools import setup, Extension
from setuptools import setup
from distutils.sysconfig import get_python_inc

from torch.utils.cpp_extension import BuildExtension, CUDAExtension

PROJ_DIR = pathlib.Path(__file__).parent.resolve()
EIGEN_INCLUDE = '/usr/include/eigen3'
OPENCV_INCLUDE = '/usr/include/opencv4'

setup(
    ...
    zip_safe=False,
    ext_modules=[
        CUDAExtension(
            name='latticenet',
            sources=['src/Lattice.cu',
                     'src/HashTable.cu',
                     'src/PyBridge.cxx'],
            include_dirs=[str(PROJ_DIR / 'include'),
                          str(PROJ_DIR / 'deps/include'),
                          str(PROJ_DIR / 'deps'),
                          OPENCV_INCLUDE,
                          EIGEN_INCLUDE],
            extra_compile_args=[f'-D PROJECT_SOURCE_DIR="{PROJ_DIR}"',
                                f'-D CMAKE_SOURCE_DIR="{PROJ_DIR}"'],
        )
    ],
    cmdclass={
        'build_ext': BuildExtension
    }
)
</code></pre>
<p>Tested in environments:<br>
Local:<br>
python version:       3.10.9<br>
pytorch                   1.13.1          py3.10_cuda11.7_cudnn8.5.0_0    pytorch<br>
pytorch-cuda              11.7                 h67b0de4_1    pytorch<br>
GCC version:          11.3.0</p>
<p>Google Colab:<br>
python version:       3.9.16<br>
pytorch                    1.13.1+cu116<br>
CUDA version: 12.0<br>
GCC version:  9.4.0</p>
<p><a href="https://drive.google.com/file/d/1KzqIFwlYgq34bWzc4F4gPQ2kOiAJ2sRK/view?usp=sharing" rel="noopener nofollow ugc">Here’s a notebook for compiling this extension in Colab</a></p>
<p>The original work provides a Docker file, in which PyTorch is downloaded and compiled from source, and a bunch of other actions. I haven’t been successful in compiling the latest (or the older one used in the Dockerfile) version from source locally, or on Colab, so I tried to use pre-compiled binaries. Could this be the main problem?</p>

username: ptrblck
cooked: <p>I don’t see any calls into <code>nvrtc</code> in your repository and thus don’t know why your lib would want to load this symbol. I might miss something as the GitHub search did not find anything useful.<br>
In any case, PyTorch itself ships with <code>libnvrtc</code> and I would assume importing <code>torch</code> before your extension could solve the issue.</p>

username: kasparas-k
cooked: <p>Thanks for checking my problem out!</p>
<p>It seems that nvrtcGetProgramLogSize is called in the <a href="https://github.com/kasparas-k/lattice_net_standalone/blob/9889171f4caf21ac0954486ed013d69e942c7199/deps/jitify/jitify.hpp#L1897" rel="noopener nofollow ugc">jitify.hpp header file</a>, but just like you said, I can find libnvrtc in my PyTorch folder. Maybe I need to somehow specify its location directly, for it to be linked?</p>
<p>But it doesn’t seem to be the first nvrtc reference in that file, so it should probably stop earlier. The <code>nvrtcGetProgramLogSize</code> call itself is inside a function of return type <code>nvrtcResult</code>, I think it should trip up on this before even getting to <code>nvrtcGetProgramLogSize</code> if the library wasn’t linked in the first place. But that’s just a guess, I don’t have the most detailed understanding.</p>
<p>I do import torch first, because otherwise I get</p>
<blockquote>
<p>ImportError: libc10.so: cannot open shared object file: No such file or directory</p>
</blockquote>