Unable to move model to cuda

Rishav_Sapahia: <p>I am trying to run my model on gpu, but I am getting<br>
<code>RuntimeError: CUDA error: device-side assert triggered</code></p>
<p>My model runs fine on CPU.<br>
I have set <code>os.environ['CUDA_LAUNCH_BLOCKING'] = "1"</code> and it is showing error when I am loading the model on cuda on the following line -</p>
<pre><code class="lang-auto">if train_on_gpu:
    unet = unet.to('cuda')
</code></pre>

albanD: <p>Hi,</p>
<p>Did you make sure to change the env variable before loading pytorch (or at least any cuda related stuff)?<br>
Is anything else than the assert getting printed?</p>

Rishav_Sapahia: <p>Hi,<br>
No, I didn’t change any env variables, could you please tell how to do it?</p>
<p>The whole error traceback is getting printed, I am posting it here-</p>
<pre><code class="lang-auto">RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-80-b3e46229c2c3&gt; in &lt;module&gt;
----&gt; 1 unet = unet.to('cuda')
      2 print(unet.device)

~\.conda\envs\opt\lib\site-packages\torch\nn\modules\module.py in to(self, *args, **kwargs)
    430             return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
    431 
--&gt; 432         return self._apply(convert)
    433 
    434     def register_backward_hook(self, hook):

~\.conda\envs\opt\lib\site-packages\torch\nn\modules\module.py in _apply(self, fn)
    206     def _apply(self, fn):
    207         for module in self.children():
--&gt; 208             module._apply(fn)
    209 
    210         def compute_should_use_set_data(tensor, tensor_applied):

~\.conda\envs\opt\lib\site-packages\torch\nn\modules\module.py in _apply(self, fn)
    206     def _apply(self, fn):
    207         for module in self.children():
--&gt; 208             module._apply(fn)
    209 
    210         def compute_should_use_set_data(tensor, tensor_applied):

~\.conda\envs\opt\lib\site-packages\torch\nn\modules\module.py in _apply(self, fn)
    228                 # `with torch.no_grad():`
    229                 with torch.no_grad():
--&gt; 230                     param_applied = fn(param)
    231                 should_use_set_data = compute_should_use_set_data(param, param_applied)
    232                 if should_use_set_data:

~\.conda\envs\opt\lib\site-packages\torch\nn\modules\module.py in convert(t)
    428 
    429         def convert(t):
--&gt; 430             return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
    431 
    432         return self._apply(convert)

RuntimeError: CUDA error: device-side assert triggered
</code></pre>

albanD: <p>Changing <code>os.environ</code> is actually changing an environment variable .<br>
A safer way to do it is to launch you program with <code>CUDA_LAUNCH_BLOCKING=1 python your_script.py</code>.</p>
<p>Is there anything printed before the traceback that is done your own printing?</p>

Rishav_Sapahia: <p>I have included in <code>os.environ['CUDA_LAUNCH_BLOCKING'] = "1"</code> in my jupyter notebook.<br>
No, I am not printing before the traceback</p>

albanD: <p>The thing is that this environment variable is only affecting the initialization of the cuda driver. This means that if you set it in your notebook after the driver has been initialized, it won’t have any effect.</p>
<p>It either needs to be set when you launch the notebook itself. Or the first thing a clean python kernel runs before calling any cuda related stuff.</p>

Rishav_Sapahia: <aside class="quote no-group" data-username="Rishav_Sapahia" data-post="1" data-topic="56390">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/r/51bf81/48.png" class="avatar"> Rishav_Sapahia:</div>
<blockquote>
<p>RuntimeError: CUDA error: device-side assert triggered</p>
</blockquote>
</aside>
<p>Thanks a lot. It worked. If I am running a script then how to do it?</p>

albanD: <p>You can run your command as: <code>CUDA_LAUNCH_BLOCKING=1 python your_script.py</code>. This will define the environment variable only for your command. You can also enable it in a given shell by running <code>export CUDA_LAUNCH_BLOCKING=1</code> and all subsequent commands in this shell will have it.</p>

JuyiLin: <p>I also meet this problem. How did you solve it?</p>