CppExtension loading complains undefined symbol

jinserk: <p>Hi,</p>
<p>I made a CppExtension, and the building had no problem, but when I import it from python, I’ve got an undefined symbol error:</p>
<pre><code class="lang-auto">Python 3.7.1 (default, Oct 23 2018, 17:15:52) 
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
&gt;&gt;&gt; import torch_asr._latgen_lib
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
ImportError: /home/jinserk/.pyenv/versions/3.7.1/lib/python3.7/site-packages/torch_asr/_latgen_lib.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN2at19UndefinedTensorImpl10_singletonE
&gt;&gt;&gt; 
</code></pre>
<p>I’m using gcc 8.2.0 on Ubuntu 18.10. Interestingly, I had no problem on the system of gcc 4.8.5 on CentOS7 with the exact same extension and setup.py.</p>
<p>I wonder whether the ATen shared libraries are automatically loaded by PyTorch or not. Do I need to add PyTorch directory to LD_LIBRARY_PATH manually? Obviously I didn’t do it on CentOS7. If I has to, why Ubuntu requires to do so while CentOS7 doesn’t?</p>

albanD: <p>Hi,</p>
<p>That feels like some ABI incompatibility between the compiler that was used for compiling pytorch and the one for compiling the extension.<br>
I’m sure <a class="mention" href="/u/goldsborough">@goldsborough</a> can tell you why !</p>

jinserk: <p>Thanks for reply, <a class="mention" href="/u/alband">@albanD</a> !<br>
Oh, I’m using <code>torch_nightly</code> on Ubuntu, but a custom built pytorch on CentOS. How to check the ABI compatibility between the pytorch binaries and the extension I made? If they don’t have the compatibility, how to fix it with the official build of <code>torch_nightly</code>?</p>

albanD: <p>So the ABI compatibility should be checked automatically and a warning raised if there is a risk of non-compatibility. I guess this check is faulty? <a class="mention" href="/u/goldsborough">@goldsborough</a> will let you know for sure.<br>
In the meantime, a temporary solution would be to try with a different gcc version. I am not sure which one is the best as it says 4.9+ in the warning but 8.2 might be too much?</p>

goldsborough: <p>You need to <code>import torch</code> before you import your library.</p>

jinserk: <p>Hi <a class="mention" href="/u/goldsborough">@goldsborough</a>,</p>
<p>I did it but still I have some problems:</p>
<pre><code class="lang-auto">$ python
Python 3.7.1 (default, Oct 23 2018, 17:15:52) 
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
&gt;&gt;&gt; import torch
&gt;&gt;&gt; import torch.utils.cpp_extension
&gt;&gt;&gt; torch.utils.cpp_extension.check_compiler_abi_compatibility("g++")
True
&gt;&gt;&gt; torch.utils.cpp_extension.check_compiler_abi_compatibility("gcc")
True
&gt;&gt;&gt; import torch_asr._latgen_lib
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
ImportError: /home/jinserk/.pyenv/versions/3.7.1/lib/python3.7/site-packages/torch_asr/_latgen_lib.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN3fst12ReadFstKaldiESs
&gt;&gt;&gt; 
</code></pre>

jinserk: <p>I’ve tried gcc 7.3.0 but the error of undefined symbol is the same.</p>

albanD: <p>Hi,</p>
<p>This missing symbol is not from pytorch <code>_ZN3fst12ReadFstKaldiESs</code> fst::ReadFst. Do you use another shared library that you don’t link / load before loading your binary?</p>

jinserk: <p>Yes. I use Kaldi, a ASR framework, and link the shared libs from the project. However, when I put the lib directory into LD_LIBRARY_PATH, the error about the Kaldi has gone, but still it complains about the ATen library.</p>
<p>Here is my project setup.py:<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/jinserk/pytorch-asr/blob/master/asr/kaldi/setup.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/jinserk/pytorch-asr/blob/master/asr/kaldi/setup.py" target="_blank" rel="nofollow noopener">jinserk/pytorch-asr/blob/master/asr/kaldi/setup.py</a></h4>
<pre><code class="lang-py">#!python

from pathlib import Path
import subprocess as sp
from setuptools import setup

from torch.utils.cpp_extension import BuildExtension, CppExtension

from _path import KALDI_ROOT


# making graph
print("## download &amp; prepare model...")
sp.run([f"scripts/mkgraph.sh {KALDI_ROOT}"], shell=True, check=True)

# building latget_lib
sources = ['src/latgen_lib.cc']

include_dirs = [
    KALDI_ROOT + "/src",
</code></pre>

  This file has been truncated. <a href="https://github.com/jinserk/pytorch-asr/blob/master/asr/kaldi/setup.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

Max_xu: <p>Hi, have you figured out the reason?</p>

eduardo4jesus: <aside class="quote no-group" data-username="albanD" data-post="2" data-topic="27908">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar"> albanD:</div>
<blockquote>
<p>eels like some ABI incompatibility between</p>
</blockquote>
</aside>
<p>Have you solved this?</p>
<p>I might have a similar issue, in my case I am using</p>
<pre><code class="lang-auto">#include &lt;ATen/native/ConvUtils.h&gt; // for cudnn_conv_suggest_memory_format
[...]
auto dataType = at::native::getCudnnDataType(input);
[...]
</code></pre>
<p>The extension builds fine (either with <code>setup.py</code> and <code>load</code>/JIT), however, when importing I get an error</p>
<pre><code class="lang-auto">ImportError: /home/eduardoj/miniconda3/lib/python3.9/site-packages/jordan_g-0.0.0-py3.9-linux-x86_64.egg/cudnn_convolution.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZN2at6native16getCudnnDataTypeERKNS_6TensorE
</code></pre>
<p>Do I need to add any path to <code>$LD_LIBRARY_PATH </code>?</p>