Pytorch is allowing incorrect matrix multiplication when using cuda

matthewleigh: <p>Hey everyone,</p>
<p>I have a minor issue and I am not sure if it is a bug or I am simply not understanding something. But I noticed that in my project I was getting some strange results for certain configurations and error messages not popping up when expected.</p>
<p>To boil it down, I am allowed to multiply matrices of incompatible sizes so long as it takes place on the gpu.</p>
<p>When running the following:</p>
<pre><code class="lang-auto">input = torch.randn(5, 10)
network = nn.Linear(3, 3)

output = network(input) ## This should fail
</code></pre>
<p>I get the expected error:</p>
<pre><code class="lang-auto">RuntimeError: mat1 and mat2 shapes cannot be multiplied (5x10 and 3x3)
</code></pre>
<p>However, the following runs without errors:</p>
<pre><code class="lang-auto">input = torch.randn(5, 10).cuda()
network = nn.Linear(3, 3)
network.cuda()

output = network(input) ## This should fail
</code></pre>
<p>It gives me an output with shape (5,3).<br>
In fact I can run with input shape (5, 8793289) and it will still work and I will still get a output with shape (5,3).</p>
<p>So what is happening here?</p>

KFrank: <p>Hi Matthew!</p>
<aside class="quote no-group" data-username="matthewleigh" data-post="1" data-topic="130664" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/matthewleigh/48/60035_2.png" class="avatar"> matthewleigh:</div>
<blockquote>
<p>I am allowed to multiply matrices of incompatible sizes so long as it takes place on the gpu.</p>
</blockquote>
</aside>
<p>I would say this is a bug (<a class="mention" href="/u/ptrblck">@ptrblck</a>) – in any event it violates the<br>
principle of least surprise.</p>
<p>I can reproduce this on version 1.9.0:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.9.0'
&gt;&gt;&gt; torch.nn.Linear (3, 3) (torch.randn (5, 10))
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "&lt;path_to_pytorch&gt;/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "&lt;path_to_pytorch&gt;/torch/nn/modules/linear.py", line 96, in forward
    return F.linear(input, self.weight, self.bias)
  File "&lt;path_to_pytorch&gt;/torch/nn/functional.py", line 1847, in linear
    return torch._C._nn.linear(input, weight, bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (5x10 and 3x3)
&gt;&gt;&gt; torch.nn.Linear (3, 3).cuda() (torch.randn (5, 10).cuda())
tensor([[-0.2980, -0.2856,  0.2935],
        [-0.3196, -0.7825, -0.0424],
        [-0.0242, -0.7375,  0.1639],
        [-0.9905, -0.4888,  0.2138],
        [-0.8091, -0.5870,  0.1135]], device='cuda:0', grad_fn=&lt;AddmmBackward&gt;)
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

ptrblck: <p>Thanks for pinging! Yes, this was a known <a href="https://github.com/pytorch/pytorch/issues/61291">issue</a>, which should have been fixed in the nightlies. Could you install the current nightly binary and rerun your code, please?</p>

KFrank: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>!</p>
<aside class="quote no-group" data-username="ptrblck" data-post="3" data-topic="130664" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>should have been fixed in the nightlies.</p>
</blockquote>
</aside>
<p>I can confirm that the issue has been fixed in the current nightly (that<br>
is, the cuda version also throws the shape-mismatch error):</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; torch.__version__
'1.10.0.dev20210830'
&gt;&gt;&gt; torch.nn.Linear (3, 3).cuda() (torch.randn (5, 10).cuda())
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "&lt;path_to_pytorch&gt;/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "&lt;path_to_pytorch&gt;/torch/nn/modules/linear.py", line 103, in forward
    return F.linear(input, self.weight, self.bias)
  File "&lt;path_to_pytorch&gt;/torch/nn/functional.py", line 1848, in linear
    return torch._C._nn.linear(input, weight, bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (5x10 and 3x3)
</code></pre>
<p>As an aside, this issue had not yet been fixed in the older nightly,<br>
version 1.10.0.dev20210624.</p>
<p>Best.</p>
<p>K. Frank</p>

matthewleigh: <p>I can confirm that the correct error is displayed with the new nightly release. Thanks</p>