Cuda extension builds but won't import (or run, depending on the situation)

username: Chris_Anderson
cooked: <p>I posted about this in the relevant GitHub repo (this post is largely copied from <a href="https://github.com/erikwijmans/Pointnet2_PyTorch/issues/93" rel="nofollow noopener">https://github.com/erikwijmans/Pointnet2_PyTorch/issues/93</a>), was hoping I could get some help here:</p>
<p>I can build the extension with <code>python setup.py build_ext --inplace</code>:</p>
<pre><code class="lang-bash">running build_ext
building 'pointnet2._ext' extension
creating build/lib.linux-x86_64-3.7/pointnet2
g++ -pthread -shared -Wl,-z,relro -g -L/usr/local/cuda/lib64 -L/usr/lib64 -lcudart -lpython3.7m -o build/lib.linux-x86_64-3.7/pointnet2/_ext.cpython-37m-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-3.7/pointnet2/_ext.cpython-37m-x86_64-linux-gnu.so -&gt; pointnet2
</code></pre>
<p>But when I try to import it, I run into problems:</p>
<pre><code class="lang-python">python -c "import pointnet2._ext"
Traceback (most recent call last):
  File "&lt;string&gt;", line 1, in &lt;module&gt;
ImportError: dynamic module does not define module export function (PyInit__ext)
</code></pre>
<p>Cuda version: 10.2<br>
Nvidia driver version: 440.33.01<br>
GPU: Nvidia Tesla T4 (compute capability 7.5)<br>
Pytorch version: 1.4<br>
OS: Amazon Linux 2</p>
<p>I’m able to get it working on a similar setup, but using Ubuntu and a GTX 1080. It just started failing as I’ve tried to get it working on EC2. Things I’ve tried:</p>
<ul>
<li>building on the EC2 instance</li>
<li>building a working docker image on one machine (does not work on the EC2 instance)</li>
<li>ensuring any python2 interpreters are unreachable</li>
<li>renaming the cuda extension to <code>_ext</code>, and matching the name of the package</li>
</ul>
<p>If I try to run the code from a docker image that works on my local machine, I think it imports successfully, but I get this:</p>
<pre><code class="lang-auto">CUDA kernel failed : no kernel image is available for execution on the device                  
void furthest_point_sampling_kernel_wrapper(int, int, int, const float*, float*, int*) at L:228
 in pointnet2/_ext-src/src/sampling_gpu.cu                                                     
</code></pre>
<p>Any ideas?</p>

username: ptrblck
cooked: <p>I’m not sure how to solve the first error.<br>
However, the “no kernel image” error is usually raised, if you’ve build your extension for a specific compute architecture (e.g. 7.0 for Volta), while you are now trying to run it on another one.<br>
You could use <code>TORCH_CUDA_ARCH_LIST="6.0 7.0" python setu.py build_ext</code> to use the specified compute capabilities.</p>

username: antmillar
cooked: <p>Did you have any luck resolving this issue? I have exactly the same problem the model runs fine on my docker image on local. But if I try to run in google compute VM with a Tesla P4 I get</p>
<pre><code class="lang-auto">CUDA kernel failed : no kernel image is available for execution on the device
</code></pre>
<p>I tried specifying compute capabilities with</p>
<pre><code class="lang-auto">TORCH_CUDA_ARCH_LIST="6.0 6.1 7.0" python setup.py install
</code></pre>
<p>But still get the same error</p>