Undefined symbol: _ZNK2at6Tensor4dataIcEEPT_v when build pytorch cuda extension

singleroc: <p>HI, Dear guys,</p>
<p>I’m build a cuda extention for my pytorch script. However, the following error occurs.<br>
<code>File "&lt;frozen importlib._bootstrap&gt;", line 684, in _load File "&lt;frozen importlib._bootstrap&gt;", line 658, in _load_unlocked File "&lt;frozen importlib._bootstrap&gt;", line 571, in module_from_spec File "&lt;frozen importlib._bootstrap_external&gt;", line 922, in create_module File "&lt;frozen importlib._bootstrap&gt;", line 219, in _call_with_frames_removed ImportError: /tmp/torch_extensions/tools/tools.so: undefined symbol: _ZNK2at6Tensor4dataIcEEPT_v</code></p>
<p>I wonder if anyone could give me some light.</p>
<p>The code is very simple.</p>
<pre><code class="lang-auto">from torch.utils.cpp_extension import load
tools = load(name='tools', sources=['test.cpp', 'test.cu'])
print(tools.dispatch, type(tools.dispatch), str(tools.dispatch))
</code></pre>
<p>in file test.cpp:</p>
<pre><code class="lang-auto">#include &lt;torch/extension.h&gt;
#include &lt;vector&gt;

#define CHECK_CUDA(x) AT_ASSERTM(x.type().is_cuda(), #x " must be a CUDA tensor")
#define CHECK_CONTIGUOUS(x) AT_ASSERTM(x.is_contiguous(), #x " must be contiguous")
#define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)

// declarations
int cuda_dispatch(torch::Tensor input, torch::Tensor codec);

std::vector&lt;torch::Tensor&gt; dispatch(torch::Tensor input, torch::Tensor codec) {
  CHECK_INPUT(input);
  CHECK_INPUT(codec);
  cuda_dispatch(input, codec);
  return { codec };
}

PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {
  m.def("dispatch", &amp;dispatch, "dispatch input according quantization levels (CUDA only)");
}
</code></pre>
<p>in file test.cu:</p>
<pre><code class="lang-auto">#include &lt;torch/extension.h&gt;

#include &lt;cuda.h&gt;
#include &lt;cuda_runtime.h&gt;

template &lt;typename Dtype&gt;
__global__ void kernel_dispatch(
    const Dtype * __restrict__ input,
    char * __restrict__ codec) {
}

int cuda_dispatch(torch::Tensor input, torch::Tensor codec) {
  const int gsize = 10240;
  const int lsize = 128;

  AT_DISPATCH_FLOATING_TYPES_AND_HALF(input.type(), "kernel_dispatch",
        ( [&amp;] { 
          kernel_dispatch &lt;scalar_t&gt; &lt;&lt;&lt; gsize, lsize&gt;&gt;&gt; (
              input.data&lt;scalar_t&gt;(),
              codec.data&lt;char&gt;());
          }
        )
      );

  return 0;
}
</code></pre>
<p>if the <code>codec</code> tensor is changed to a float tensor and replace codec.data &lt;char&gt;() to codec.data&lt;scalar_t&gt;()) in the test.cu, the error was gone.</p>
<p>Thus I suspect that there’s some limitation I unaware for the cuda template around AT_DISPATCH_FLOATING_TYPES_AND_HALF.</p>

smth: <p>Do you have multiple PyTorch installs? That is often the main issue, in such errors.</p>

singleroc: <p><a class="mention" href="/u/smth">@smth</a>  Hi, Thanks for your attention.</p>
<p>I install the pytorch via pip (in a pyenv managed enviroment) and only one instance is installed.<br>
I tried on different machines, such as<br>
1080ti + CUDA9.0<br>
2080ti + CUDA10.0</p>
<p>I set up a quick demo in repo: <a href="https://github.com/blueardour/extension-demo.git" rel="nofollow noopener">https://github.com/blueardour/extension-demo.git</a><br>
There are two extensions inside. The only differece is the  datatype of the <code>codec</code> vairable. As I mentioned in the original post, if all the tensors are the same type, no error occur. Otherwise I met the undefined error. (It indeed could be worked around to force all tensor as the same datatype, however I just want to try to reduce the data precision for runtime memory saving in my real applications.)</p>
<p>to run: python demo.py<br>
gives:</p>
<pre><code class="lang-auto">&lt;built-in method dispatch of PyCapsule object at 0x7fe5b02e4510&gt;
demo2 build failed
</code></pre>

singleroc: <p>Hi,  I latter found the error is not related with cuda extension. It seemed to be the data type convert error when accessing the tensor.</p>
<pre><code class="lang-auto">#include &lt;ATen/ATen.h&gt;
int cuda_dispatch(at::Tensor input, at::Tensor codec) {
  //void *ptr0 = codec.data&lt;char&gt;();  //this would casue the undefined error
  void *ptr1 = codec.data&lt;unsigned char&gt;();  // this is OK
  void *ptr2 = codec.data&lt;short&gt;();  // this is OK
  void *ptr3 = codec.data&lt;int&gt;();  // this is OK
  return 0;
}
</code></pre>
<p>Seems only type of char is not support~</p>
<p>Never mind when the official guide says:<br>
<code>at::Tensor::C10_DEPRECATED_MESSAGE("Tensor.data&lt;T&gt;() is deprecated. Please use Tensor.data_ptr &lt;T&gt;() instead.") const</code>  in <a href="https://pytorch.org/cppdocs/api/classat_1_1_tensor.html#_CPPv4NK2at6Tensor8data_ptrEv" rel="nofollow noopener">tensor-ops</a></p>