Inconsistent results of forwarding a cpu/gpu tensor with a cpu model

kaixin: <p>Hi, I found that forwarding the same tensor on cpu and gpu with a model on cpu yield different results.</p>
<pre><code class="lang-python">import torch
fc = torch.nn.Linear(2, 2)
t = torch.tensor([[0.5, 0.3]])
print(fc(t))
print(fc(t.cuda()))
</code></pre>
<p>The result of <code>fc(t)</code> is correct while the result of <code>fc(t.cuda())</code> is wrong. After a little digging, I found it is <code>torch.addmm()</code> that cause this issue. Though forwarding a gpu tensor with a cpu model generally makes little sense, my expectation is that pytorch can either throw an error (saying input device mismatch) or give the correct result.</p>

user_123454321: <p>This is interesting, I would expect it to throw error as well (and it did when I tried). Can you mention the pytorch version and the machine info.</p>

kaixin: <p><a class="mention" href="/u/user_123454321">@user_123454321</a> my  pytorch version is 1.6.0 and OS is ubuntu 16.04. I tried different GPU models but the issue persists. By the way, may I know your pytorch version?</p>
<p>Some updates: I found the output <code>fc(t.cuda())</code> equals to the bias of the Linear layer. When I set <code>bias=False</code>, calling <code>fc(t.cuda())</code> will raise an error <code>RuntimeError: copy_if failed to synchronize: an illegal memory access was encountered</code>. I guess it is probably related to the implementation of <code>torch.addmm()</code> but could not find out.</p>

user_123454321: <p>Hmmm…my pytorch version is 1.7.1, OS is ubuntu 18.0.4 and P100 GPU. I still don’t get why data on gpu is able to be run on model on cpu.</p>

kaixin: <p>I managed to try both 1.7.0 and 1.6.0 on colab and it seems the issue is with 1.6.0. Pytorch 1.7.0 can raise error as expected.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/2/62fd23f529109904dc23be7c21ca32bf555dba6d.png" data-download-href="https://discuss.pytorch.org/uploads/default/62fd23f529109904dc23be7c21ca32bf555dba6d" title="Screen Shot 2021-01-12 at 10.46.21 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/2/62fd23f529109904dc23be7c21ca32bf555dba6d_2_378x374.png" alt="Screen Shot 2021-01-12 at 10.46.21 PM" data-base62-sha1="e7H8RPOMoFsB8ucLccGqqd56dtP" width="378" height="374" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/2/62fd23f529109904dc23be7c21ca32bf555dba6d_2_378x374.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/2/62fd23f529109904dc23be7c21ca32bf555dba6d_2_567x561.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/2/62fd23f529109904dc23be7c21ca32bf555dba6d_2_756x748.png 2x" data-dominant-color="F5F5F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2021-01-12 at 10.46.21 PM</span><span class="informations">976×968 53.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

user_123454321: <p>Yes, it seems like the error is in 1.6.0 (tried in the server). Tried 1.5.0, that gives expected results too.</p>
<p><strong>1.6.0</strong></p>
<pre><code class="lang-auto">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.6.0'
&gt;&gt;&gt; fc = torch.nn.Linear(2, 2)
&gt;&gt;&gt; t = torch.tensor([[0.5, 0.3]])
&gt;&gt;&gt; print(fc(t))
tensor([[-0.1817,  0.4468]], grad_fn=&lt;AddmmBackward&gt;)
&gt;&gt;&gt; print(fc(t.cuda()))
tensor([[0.1775, 0.5823]], grad_fn=&lt;AddmmBackward&gt;)
&gt;&gt;&gt;
</code></pre>
<p><strong>1.5.0</strong></p>
<pre><code class="lang-auto">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.5.0'
&gt;&gt;&gt; fc = torch.nn.Linear(2, 2)
&gt;&gt;&gt; t = torch.tensor([[0.5, 0.3]])
&gt;&gt;&gt; print(fc(t))
tensor([[0.0865, 0.0900]], grad_fn=&lt;AddmmBackward&gt;)
&gt;&gt;&gt; print(fc(t.cuda()))
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "/home/ubuntu/anaconda3/envs/pytorch1.5/lib/python3.8/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch1.5/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 87, in forward
    return F.linear(input, self.weight, self.bias)
  File "/home/ubuntu/anaconda3/envs/pytorch1.5/lib/python3.8/site-packages/torch/nn/functional.py", line 1610, in linear
    ret = torch.addmm(bias, input, weight.t())
RuntimeError: Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _th_addmm
&gt;&gt;&gt;
</code></pre>