Kernel dies when moving model to cuda

username: Dwight_Foster
cooked: <p>When I attempt to move my model to cuda the kernel says the kernel has died. here is my code:<br>
import torch.nn as nn<br>
import torch.nn.functional as F</p>
<p>class Net(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(Net, self).<strong>init</strong>()<br>
self.conv1 = nn.Conv2d(3, 6, 5)<br>
self.pool = nn.MaxPool2d(2, 2)<br>
self.conv2 = nn.Conv2d(6, 16, 5)<br>
self.fc1 = nn.Linear(16 * 5 * 5, 120)<br>
self.fc2 = nn.Linear(120, 84)<br>
self.fc3 = nn.Linear(84, 10)</p>
<pre><code>def forward(self, x):
    x = self.pool(F.relu(self.conv1(x)))
    x = self.pool(F.relu(self.conv2(x)))
    x = x.view(-1, 16 * 5 * 5)
    x = F.relu(self.fc1(x))
    x = F.relu(self.fc2(x))
    x = self.fc3(x)
    return x
</code></pre>
<p>net = Net()<br>
use_cuda = torch.cuda.is_available()<br>
net.cuda()</p>
<p>When I print is use_cuda it says true which means it detects the gpu.<br>
I am using a 2080 with cuda 10.1 and pytorch version 1.3.1<br>
Any advice?</p>

username: ptrblck
cooked: <p>Could you rerun the script in the terminal and post the stack trace here, please?</p>

username: Dwight_Foster
cooked: <p>I ran the script and I got this error message Segmentation fault (core dumped).</p>

username: ptrblck
cooked: <p>Could you run your code with <code>gdb</code> and try to get the stack trace as explained in <a href="https://discuss.pytorch.org/t/issue-kernel-dies-when-fitting-pytorch-model-linux-mint/22961/16">this post</a>?</p>

username: Dwight_Foster
cooked: <p>Ok I did that here are the results:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/3/6377095cbed3168fc978e779b20979a512b572bd.png" data-download-href="https://discuss.pytorch.org/uploads/default/6377095cbed3168fc978e779b20979a512b572bd" title="Screenshot%20from%202019-12-04%2019-21-21"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/3/6377095cbed3168fc978e779b20979a512b572bd_2_690x458.png" alt="Screenshot%20from%202019-12-04%2019-21-21" data-base62-sha1="ebUiRxTO9teOokT0AZLi3Yqvjat" width="690" height="458" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/3/6377095cbed3168fc978e779b20979a512b572bd_2_690x458.png, https://discuss.pytorch.org/uploads/default/original/3X/6/3/6377095cbed3168fc978e779b20979a512b572bd.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/3/6377095cbed3168fc978e779b20979a512b572bd.png 2x" data-dominant-color="412137"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot%20from%202019-12-04%2019-21-21</span><span class="informations">734×488 102 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Dwight_Foster
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/9/39b86a06a3abfbe76e591c86b4e85d3241432713.png" data-download-href="https://discuss.pytorch.org/uploads/default/39b86a06a3abfbe76e591c86b4e85d3241432713" title="Screenshot%20from%202019-12-04%2019-21-32"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/9/39b86a06a3abfbe76e591c86b4e85d3241432713_2_690x458.png" alt="Screenshot%20from%202019-12-04%2019-21-32" data-base62-sha1="8eCjVeab90gkzHuu5eAZ3A9e50v" width="690" height="458" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/9/39b86a06a3abfbe76e591c86b4e85d3241432713_2_690x458.png, https://discuss.pytorch.org/uploads/default/original/3X/3/9/39b86a06a3abfbe76e591c86b4e85d3241432713.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/9/39b86a06a3abfbe76e591c86b4e85d3241432713.png 2x" data-dominant-color="412137"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot%20from%202019-12-04%2019-21-32</span><span class="informations">734×488 109 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Dwight_Foster
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/a/3a39a0f9d8d03dd25dc50d56b05b94cceb250131.png" data-download-href="https://discuss.pytorch.org/uploads/default/3a39a0f9d8d03dd25dc50d56b05b94cceb250131" title="Screenshot%20from%202019-12-04%2019-21-38"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/a/3a39a0f9d8d03dd25dc50d56b05b94cceb250131_2_690x458.png" alt="Screenshot%20from%202019-12-04%2019-21-38" data-base62-sha1="8j5a4rGSa4LZ3sZRQZyiaw35mGB" width="690" height="458" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/a/3a39a0f9d8d03dd25dc50d56b05b94cceb250131_2_690x458.png, https://discuss.pytorch.org/uploads/default/original/3X/3/a/3a39a0f9d8d03dd25dc50d56b05b94cceb250131.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/a/3a39a0f9d8d03dd25dc50d56b05b94cceb250131.png 2x" data-dominant-color="412137"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot%20from%202019-12-04%2019-21-38</span><span class="informations">734×488 94.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Dwight_Foster
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/9/79246aaa1a2b204fea65afb1b0ef2c9ace7de385.png" data-download-href="https://discuss.pytorch.org/uploads/default/79246aaa1a2b204fea65afb1b0ef2c9ace7de385" title="Screenshot%20from%202019-12-04%2019-22-05"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/9/79246aaa1a2b204fea65afb1b0ef2c9ace7de385_2_690x458.png" alt="Screenshot%20from%202019-12-04%2019-22-05" data-base62-sha1="hhFNmNIgPPifc0S9Mz2BdJUvN4x" width="690" height="458" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/9/79246aaa1a2b204fea65afb1b0ef2c9ace7de385_2_690x458.png, https://discuss.pytorch.org/uploads/default/original/3X/7/9/79246aaa1a2b204fea65afb1b0ef2c9ace7de385.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/9/79246aaa1a2b204fea65afb1b0ef2c9ace7de385.png 2x" data-dominant-color="412137"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot%20from%202019-12-04%2019-22-05</span><span class="informations">734×488 94.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Dwight_Foster
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/7/0734548207cc1b70962dbb30a9f56b5968b64290.png" data-download-href="https://discuss.pytorch.org/uploads/default/0734548207cc1b70962dbb30a9f56b5968b64290" title="Screenshot%20from%202019-12-04%2019-22-12"><img src="https://discuss.pytorch.org/uploads/default/original/3X/0/7/0734548207cc1b70962dbb30a9f56b5968b64290.png" alt="Screenshot%20from%202019-12-04%2019-22-12" data-base62-sha1="11JsswQRQCnX7CIYzp8ZUlSBayk" width="690" height="458" data-dominant-color="48293E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot%20from%202019-12-04%2019-22-12</span><span class="informations">734×488 117 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Dwight_Foster
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/5/65bce415e83eb77091c2e4fa9ba6995c077f3cd8.png" data-download-href="https://discuss.pytorch.org/uploads/default/65bce415e83eb77091c2e4fa9ba6995c077f3cd8" title="Screenshot%20from%202019-12-04%2019-22-22"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/5/65bce415e83eb77091c2e4fa9ba6995c077f3cd8_2_690x458.png" alt="Screenshot%20from%202019-12-04%2019-22-22" data-base62-sha1="ew0V8mcyut2vbIm5hxa9OXyV9Ec" width="690" height="458" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/5/65bce415e83eb77091c2e4fa9ba6995c077f3cd8_2_690x458.png, https://discuss.pytorch.org/uploads/default/original/3X/6/5/65bce415e83eb77091c2e4fa9ba6995c077f3cd8.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/5/65bce415e83eb77091c2e4fa9ba6995c077f3cd8.png 2x" data-dominant-color="44253A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot%20from%202019-12-04%2019-22-22</span><span class="informations">734×488 102 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Dwight_Foster
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/1/91b98a82d45c72783af77dc6335cf24996c5e8b9.png" data-download-href="https://discuss.pytorch.org/uploads/default/91b98a82d45c72783af77dc6335cf24996c5e8b9" title="Screenshot%20from%202019-12-04%2019-22-51"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/1/91b98a82d45c72783af77dc6335cf24996c5e8b9_2_690x458.png" alt="Screenshot%20from%202019-12-04%2019-22-51" data-base62-sha1="kN8KGUSl2i4DQGn7oYNxvX1OeSd" width="690" height="458" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/1/91b98a82d45c72783af77dc6335cf24996c5e8b9_2_690x458.png, https://discuss.pytorch.org/uploads/default/original/3X/9/1/91b98a82d45c72783af77dc6335cf24996c5e8b9.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/9/1/91b98a82d45c72783af77dc6335cf24996c5e8b9.png 2x" data-dominant-color="412137"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot%20from%202019-12-04%2019-22-51</span><span class="informations">734×488 92.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Dwight_Foster
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/e/ae100b5950b265c797cf87a1503851eb66b736f1.png" data-download-href="https://discuss.pytorch.org/uploads/default/ae100b5950b265c797cf87a1503851eb66b736f1" title="Screenshot%20from%202019-12-04%2019-23-03"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/e/ae100b5950b265c797cf87a1503851eb66b736f1_2_690x458.png" alt="Screenshot%20from%202019-12-04%2019-23-03" data-base62-sha1="oPPsbMYZn5fNPo7wRcjUortRVpT" width="690" height="458" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/e/ae100b5950b265c797cf87a1503851eb66b736f1_2_690x458.png, https://discuss.pytorch.org/uploads/default/original/3X/a/e/ae100b5950b265c797cf87a1503851eb66b736f1.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/a/e/ae100b5950b265c797cf87a1503851eb66b736f1.png 2x" data-dominant-color="402036"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot%20from%202019-12-04%2019-23-03</span><span class="informations">734×488 91.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: Dwight_Foster
cooked: <p>Sorry I had to put them in different posts it wouldn’t let me put them in the same post because I am a new user. And when I enter the disas thing it says: No function contains program counter for selected frame.</p>

username: ptrblck
cooked: <p>Could you post the output as text and wrapping it into three backticks ```?<br>
This would format the text and would make it easier to follow, what’s going on.</p>

username: Dwight_Foster
cooked: <pre><code class="lang-auto">Starting program: /usr/bin/python3 cifar10_tutorial.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff373a700 (LWP 3201)]
[New Thread 0x7ffff2f39700 (LWP 3202)]
[New Thread 0x7fffee738700 (LWP 3203)]
[New Thread 0x7fffedf37700 (LWP 3204)]
[New Thread 0x7fffe9736700 (LWP 3205)]
[New Thread 0x7fffe8f35700 (LWP 3206)]
[New Thread 0x7fffe6734700 (LWP 3207)]
[Thread 0x7fffe6734700 (LWP 3207) exited]
[Thread 0x7fffe8f35700 (LWP 3206) exited]
[Thread 0x7fffe9736700 (LWP 3205) exited]
[Thread 0x7fffedf37700 (LWP 3204) exited]
[Thread 0x7fffee738700 (LWP 3203) exited]
[Thread 0x7ffff2f39700 (LWP 3202) exited]
[Thread 0x7ffff373a700 (LWP 3201) exited]
Files already downloaded and verified
Files already downloaded and verified
[New Thread 0x7fffe6734700 (LWP 4763)]
[New Thread 0x7fffe8f35700 (LWP 4764)]
[New Thread 0x7fffe9736700 (LWP 4765)]
[New Thread 0x7fffedf37700 (LWP 4766)]
[New Thread 0x7fff7f99a700 (LWP 4767)]
[New Thread 0x7fff7f199700 (LWP 4768)]
[New Thread 0x7fff7e998700 (LWP 4769)]
[New Thread 0x7fff7e017700 (LWP 4771)]
[New Thread 0x7fff7d816700 (LWP 4772)]
horse   cat plane truck
[New Thread 0x7fff7cdc7700 (LWP 4781)]
[New Thread 0x7fff76e8f700 (LWP 4782)]
[New Thread 0x7fff7668e700 (LWP 4783)]

Thread 1 "python3" received signal SIGSEGV, Segmentation fault.
0x00007fff77127bf0 in ?? () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
(gdb) backtrace
#0  0x00007fff77127bf0 in ?? () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#1  0x00007fff77127c52 in ?? () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#2  0x00007fff7712851b in ?? () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#3  0x00007fff77062af9 in ?? () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#4  0x00007fff77087bc3 in ?? () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#5  0x00007fff76fc4d18 in ?? () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#6  0x00007fff76fc52ac in ?? () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#7  0x00007fffee089137 in ?? ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libcudart-1b201d85.so.10.1
#8  0x00007fffee0815f0 in ?? ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libcudart-1b201d85.so.10.1
#9  0x00007fffee08c012 in ?? ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libcudart-1b201d85.so.10.1
#10 0x00007fffee091581 in ?? ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libcudart-1b201d85.so.10.1
#11 0x00007fffee08382e in ?? ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libcudart-1b201d85.so.10.1
#12 0x00007fffee0732c8 in ?? ()
---Type &lt;return&gt; to continue, or q &lt;return&gt; to quit---
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libcudart-1b201d85.so.10.1
#13 0x00007fffee0a582c in cudaMalloc ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libcudart-1b201d85.so.10.1
#14 0x00007ffff295469e in c10::cuda::CUDACachingAllocator::THCCachingAllocator::malloc(void**, unsigned long, CUstream_st*) ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libc10_cuda.so
#15 0x00007ffff2955e6e in c10::cuda::CUDACachingAllocator::CudaCachingAllocator::allocate(unsigned long) const ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libc10_cuda.so
#16 0x00007fff9b607789 in at::native::empty_cuda(c10::ArrayRef&lt;long&gt;, c10::TensorOptions const&amp;, c10::optional&lt;c10::MemoryFormat&gt;) ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch.so
#17 0x00007fff9a04d728 in at::CUDAType::(anonymous namespace)::empty(c10::ArrayRef&lt;long&gt;, c10::TensorOptions const&amp;, c10::optional&lt;c10::MemoryFormat&gt;) ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch.so
#18 0x00007fff99b06e18 in torch::autograd::VariableType::(anonymous namespace)::empty(c10::ArrayRef&lt;long&gt;, c10::TensorOptions const&amp;, c10::optional&lt;c10::MemoryFormat&gt;) () from /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch.so
#19 0x00007fff97a54971 in at::native::to_impl(at::Tensor const&amp;, c10::TensorOptions const&amp;, bool) ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch.so
---Type &lt;return&gt; to continue, or q &lt;return&gt; to quit---
#20 0x00007fff97a55312 in at::native::to(at::Tensor const&amp;, c10::TensorOptions const&amp;, bool, bool) ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch.so
#21 0x00007fff97d7e280 in at::TypeDefault::to(at::Tensor const&amp;, c10::TensorOptions const&amp;, bool, bool) ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch.so
#22 0x00007fff99918a63 in torch::autograd::VariableType::(anonymous namespace)::to(at::Tensor const&amp;, c10::TensorOptions const&amp;, bool, bool) ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch.so
#23 0x00007fffdd857087 in torch::autograd::dispatch_to(at::Tensor const&amp;, c10::Device, bool, bool) ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so
#24 0x00007fffdd857e03 in torch::autograd::THPVariable_cuda(_object*, _object*, _object*) ()
   from /usr/local/lib/python3.6/dist-packages/torch/lib/libtorch_python.so
#25 0x000000000050abc5 in ?? ()
#26 0x000000000050c549 in _PyEval_EvalFrameDefault ()
#27 0x00000000005081d5 in ?? ()
#28 0x000000000050a020 in ?? ()
#29 0x000000000050aa1d in ?? ()
#30 0x000000000050c549 in _PyEval_EvalFrameDefault ()
#31 0x0000000000509ce8 in ?? ()
#32 0x000000000050aa1d in ?? ()
---Type &lt;return&gt; to continue, or q &lt;return&gt; to quit---
#33 0x000000000050c549 in _PyEval_EvalFrameDefault ()
#34 0x0000000000509ce8 in ?? ()
#35 0x000000000050aa1d in ?? ()
#36 0x000000000050c549 in _PyEval_EvalFrameDefault ()
#37 0x00000000005081d5 in ?? ()
#38 0x000000000050a020 in ?? ()
#39 0x000000000050aa1d in ?? ()
#40 0x000000000050c549 in _PyEval_EvalFrameDefault ()
#41 0x00000000005081d5 in ?? ()
#42 0x000000000050b3a3 in PyEval_EvalCode ()
#43 0x0000000000635082 in ?? ()
#44 0x0000000000635137 in PyRun_FileExFlags ()
#45 0x00000000006388ef in PyRun_SimpleFileExFlags ()
#46 0x0000000000639491 in Py_Main ()
#47 0x00000000004b0f60 in main ()
</code></pre>

username: Dwight_Foster
cooked: <pre><code class="lang-auto">(gdb) disas
No function contains program counter for selected frame.
</code></pre>