Pytorch CUDA problems with Pybind11

georgeh: <p>Good Afternoon,</p>
<p>I am wondering if anyone has experienced the following issue. I am trying to embed a python interpreter in c++ using pybind11, and then inside the interpreter do some operations on tensors on the GPU. The problem is, the second I send a tensor to the GPU through a command such as .cuda or .to(device) (with the device being cuda), it causes my main c++ code to either hang or to aborted core dumped.</p>
<p>My c++ code is as follows:</p>
<pre><code class="lang-auto">#include &lt;pybind11/pybind11.h&gt;
#include &lt;pybind11/embed.h&gt;

namespace py = pybind11;

int main() {
    py::scoped_interpreter guard{};
    py::module m = py::module::import("cuda_test");
    py::function test_func = m.attr("test_func");
    auto test = test_func().cast&lt;py::int_&gt;();
    return 0;
}
</code></pre>
<p>and my python code is as follows</p>
<pre><code class="lang-auto">import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

def test_func():
    a = torch.tensor([0]).to("device")
    print(a.is_cuda)
    return 0
</code></pre>
<p>The code compiles fine, and it works so long as I set the device to “cpu”. If the device is “cuda” then the program either hangs indefinitely, or produces the message “Aborted (core dumped)”. For reference, I am running this on Ubuntu 20.04. Oddly enough, I can see the process in nvidia-smi, so something must be happening with the GPU. Any ideas on what is happening here? Has anyone experienced this?</p>
<p>Thank you again for any ideas or suggestions. I am a complete novice, so I may be doing something incorrect which is very simple.</p>

georgeh: <p>Ok, just as a bit of extra information, I have determined that I was including arrayfire headers and libraries in my makefile (though I didn’t include a directive in my code above), and this seems to be the linchpin of this whole thing - if I remove the arrayfire headers and libraries then everything works fine! Unfortunately, I need my code to work alongside arrayfire, so now my question is: Is it known that arrayfire and pytorch conflict with each other somehow in this context?</p>

ptrblck: <p>I don’t know what might be causing the issue, but you might need to debug it by checking what the actual error is via <code>gdb</code> and the corresponding <code>backtrace</code>.</p>

georgeh: <p>Hi ptrblck,</p>
<p>Thank you for your response! I am pasting the output of running gdb and bt below. Unfortunately I am a complete novice and have no clue what the following message might mean, of course this may be useful for anyone else who is attempting the same thing.</p>
<blockquote>
<p>[Thread debugging using libthread_db enabled]<br>
Using host libthread_db library “/lib/x86_64-linux-gnu/libthread_db.so.1”.<br>
[New Thread 0x7fffe9140700 (LWP 832421)]<br>
[New Thread 0x7fffe893f700 (LWP 832422)]<br>
[New Thread 0x7fffe1dff700 (LWP 832423)]<br>
[New Thread 0x7fffe15fe700 (LWP 832424)]<br>
[New Thread 0x7fffe0dfd700 (LWP 832425)]<br>
[New Thread 0x7fffc5fff700 (LWP 832426)]<br>
[New Thread 0x7fffc57fe700 (LWP 832427)]<br>
[New Thread 0x7fff3f801700 (LWP 832431)]</p>
<p><strong>Thread 1 “main” received signal SIGSEGV, Segmentation fault.</strong><br>
0x00005555566d2fb0 in ?? ()<br>
(gdb) bt<br>
<span class="hashtag-raw">#0</span>  0x00005555566d2fb0 in ?? ()<br>
<span class="hashtag-raw">#1</span>  0x00007fff3e7a8dc0 in ?? ()<br>
<span class="hashtag-raw">#2</span>  0x00007fff3e99a81c in ?? ()<br>
<span class="hashtag-raw">#3</span>  0x00007fff3e99a810 in ?? ()<br>
<span class="hashtag-raw">#4</span>  0x00007fff3e9daf58 in ?? ()<br>
<span class="hashtag-raw">#5</span>  0x00007ffff7d5a4eb in _Py_Dealloc (<br>
op=) at /usr/local/src/conda/python-3.10.9/Objects/object.c:2295<br>
<span class="hashtag-raw">#6</span>  _Py_DECREF (op=0x555556cb1040)<br>
at /usr/local/src/conda/python-3.10.9/Include/object.h:500<br>
<span class="hashtag-raw">#7</span>  _Py_XDECREF (op=0x555556cb1040)<br>
at /usr/local/src/conda/python-3.10.9/Include/object.h:567<br>
<span class="hashtag-raw">#8</span>  _PyEval_EvalFrameDefault (tstate=, f=,<br>
throwflag=)<br>
at /usr/local/src/conda/python-3.10.9/Python/ceval.c:4280<br>
<span class="hashtag-raw">#9</span>  0x00007ffff7d663a7 in PyDict_GetItemWithError (key=0x7fff31550280, op=0x0)<br>
at /usr/local/src/conda/python-3.10.9/Objects/object.c:1306<br>
<span class="hashtag-raw">#10</span> _PyObject_GenericGetAttrWithDict (obj=,<br>
name=0x7fff31550280, dict=0x0, suppress=0)<br>
at /usr/local/src/conda/python-3.10.9/Objects/object.c:1285<br>
<span class="hashtag-raw">#11</span> 0x0000555555573420 in ?? ()<br>
<span class="hashtag-raw">#12</span> 0x00007fff3e7b02b0 in ?? ()<br>
<span class="hashtag-raw">#13</span> 0x00007fff31542610 in ?? ()<br>
–Type  for more, q to quit, c to continue without paging–<br>
<span class="hashtag-raw">#14</span> 0x00007fff3e7b02b0 in ?? ()<br>
<span class="hashtag-raw">#15</span> 0x00005555566d2fb0 in ?? ()<br>
<span class="hashtag-raw">#16</span> 0x00007fff3ea33070 in ?? ()<br>
<span class="hashtag-raw">#17</span> 0x00007ffff7d754c4 in PyObject_GenericGetAttr (<br>
name=, obj=0x7fff316e7880)<br>
at /usr/local/src/conda/python-3.10.9/Objects/object.c:1333<br>
<span class="hashtag-raw">#18</span> PyObject_GetAttr (name=0x7fff3e7b02b0, v=0x7fff316e7880)<br>
at /usr/local/src/conda/python-3.10.9/Objects/object.c:932<br>
<span class="hashtag-raw">#19</span> _PyObject_GetAttrId (name=0x7ffff7fa2f30 &lt;PyId__initializing.3&gt;,<br>
v=)<br>
at /usr/local/src/conda/python-3.10.9/Objects/object.c:872<br>
<span class="hashtag-raw">#20</span> <em>PyModuleSpec_IsInitializing (<br>
spec=)<br>
at /usr/local/src/conda/python-3.10.9/Objects/moduleobject.c:709<br>
<span class="hashtag-raw">#21</span> import_ensure_initialized (name=0x555555573ae0 &lt;<em>Py_FalseStruct&gt;,<br>
mod=0x5555566d2fb0, interp=)<br>
at /usr/local/src/conda/python-3.10.9/Python/import.c:358<br>
<span class="hashtag-raw">#22</span> PyImport_ImportModuleLevelObject (name=0x7fff3ea0af30,<br>
globals=, locals=, fromlist=,<br>
level=)<br>
at /usr/local/src/conda/python-3.10.9/Python/import.c:1617<br>
–Type  for more, q to quit, c to continue without paging–<br>
<span class="hashtag-raw">#23</span> 0x00007ffff7d75987 in PyImport_ImportModuleLevelObject (<br>
name=0x7fff3e79c800, globals=, locals=,<br>
fromlist=, level=)<br>
at /usr/local/src/conda/python-3.10.9/Include/object.h:500<br>
<span class="hashtag-raw">#24</span> 0x00007ffff7d818f8 in builtin___import</em></em> (self=,<br>
args=, kwds=)<br>
at /usr/local/src/conda/python-3.10.9/Python/bltinmodule.c:272<br>
<span class="hashtag-raw">#25</span> 0x00007fff3e79d620 in ?? ()<br>
<span class="hashtag-raw">#26</span> 0x00005555566d2fb0 in ?? ()<br>
<span class="hashtag-raw">#27</span> 0x00007fff3ea2c9a0 in ?? ()<br>
<span class="hashtag-raw">#28</span> 0x0000000000000000 in ?? ()</p>
</blockquote>

ptrblck: <p>Unfortunately, I don’t see the root cause of the issue in the stacktrace besides a <code>Dealloc</code> call on an object which seems to be freed already.</p>

ten_ko: <p>Hello, I am also encountering this issue when using Cuda in Pybind embedded interpreter. Have you figured it out?</p>