Incorrect Model runs while on GPU!

Chirag: <p>Hi,</p>
<p>Here is a simple test model, which is incorrect due to size mismatch between fc1 and fc2. Trying to use the model gives a mat size mismatch error as expected. However, on computing the same on a GPU gives an output without any errors!</p>
<p>The model:</p>
<pre><code class="lang-auto">class testModel(nn.Module):

    def __init__(self) -&gt; None:
 
        super().__init__() #Python3 syntax
        
        self.fc1 = nn.Linear(10, 10)
        self.fc2 = nn.Linear(5,1)

    def forward(self, x):

        x = self.fc1(x)
        x = self.fc2(x)

        return x
</code></pre>
<p>This does not work (As expected):</p>
<pre><code class="lang-auto">inp   = torch.ones((1,10))
model = testModel()
out = model(inp)
</code></pre>
<p>The resulting error:<br>
“RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x10 and 5x1)”</p>
<p>But this works!</p>
<pre><code class="lang-auto">inp   = torch.ones((1,10)).to('cuda:0')
model = testModel().to('cuda:0')
out = model(inp)
</code></pre>
<p>I am assuming it has something to do with “has_torch_function_variadic” in <a href="https://github.com/pytorch/pytorch/blob/565cf47abfa212243242ab5a8d463c162b1556e3/torch/nn/functional.py#L1880" rel="noopener nofollow ugc">nn.functional</a>.</p>
<p>Is this a bug or any reason this is allowed to happen?</p>
<p>Thanks,<br>
Chirag</p>

Sayed_Nadim: <p>Hi,<br>
Can you please specify which version of PyTorch you are using?<br>
I can’t reproduce the issue with PyTorch 1.9. I am having the matrix mismatch error, as expected.</p>
<pre><code class="lang-auto">class testModel(nn.Module):

    def __init__(self) -&gt; None:
        super().__init__()

        self.fc1 = nn.Linear(10, 10)
        self.fc2 = nn.Linear(5, 1)

    def forward(self, x):
        x = self.fc1(x)
        x = self.fc2(x)
        return x

inp = torch.ones((1, 10)).to('cuda:0')
model = testModel().to('cuda:0')
out = model(inp)
</code></pre>
<p>And, the error, as it should be.</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "/home/la-belva/.config/JetBrains/PyCharmCE2021.2/scratches/scratch_15.py", line 20, in &lt;module&gt;
    out = model(inp)
  File "/home/la-belva/anaconda3/envs/latest/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/la-belva/.config/JetBrains/PyCharmCE2021.2/scratches/scratch_15.py", line 15, in forward
    x = self.fc2(x)
  File "/home/la-belva/anaconda3/envs/latest/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/la-belva/anaconda3/envs/latest/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 94, in forward
    return F.linear(input, self.weight, self.bias)
  File "/home/la-belva/anaconda3/envs/latest/lib/python3.8/site-packages/torch/nn/functional.py", line 1753, in linear
    return torch._C._nn.linear(input, weight, bias)
RuntimeError: mat1 dim 1 must match mat2 dim 0

Process finished with exit code 1

</code></pre>
<p><img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji only-emoji" alt=":slight_smile:"></p>

Chirag: <p>Hi,</p>
<p>I’m using torch version 1.9.0+cu102.</p>
<p>Thanks,<br>
Chirag</p>

Chirag: <p>And here is the output. It seems to work:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/3/5392e6062725c46a33bcafb20a9c2a8bf12aea11.png" data-download-href="https://discuss.pytorch.org/uploads/default/5392e6062725c46a33bcafb20a9c2a8bf12aea11" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/3/5392e6062725c46a33bcafb20a9c2a8bf12aea11_2_690x389.png" alt="image" data-base62-sha1="bVkmeQ4GDimSt7P1JdJAH6XE1DH" width="690" height="389" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/3/5392e6062725c46a33bcafb20a9c2a8bf12aea11_2_690x389.png, https://discuss.pytorch.org/uploads/default/original/3X/5/3/5392e6062725c46a33bcafb20a9c2a8bf12aea11.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/5/3/5392e6062725c46a33bcafb20a9c2a8bf12aea11.png 2x" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">898×507 46.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>You are most likely running into a known and already fixed issue described <a href="https://discuss.pytorch.org/t/cuda-error-cublas-status-invalid-value/132944/4">here</a>, so please update your PyTorch version.</p>

Sayed_Nadim: <p><a class="mention" href="/u/chirag">@Chirag</a><br>
Can confirm this issue in Colab for Pytorch 1.9.0.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/2/1/2196ee741e3cabbb5587976560674263049b2a1b.png" alt="Screenshot from 2021-10-13 15-54-49" data-base62-sha1="4N97N8gzFgiMvgG17tzhHNwlxSb" width="621" height="445"><br>
And, this issue is resolved after upgrading to PyTorch 1.9.1 as <a class="mention" href="/u/ptrblck">@ptrblck</a> suggested.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/b/8b95623ed25be906a6f0687272986f117e1e79a0.png" data-download-href="https://discuss.pytorch.org/uploads/default/8b95623ed25be906a6f0687272986f117e1e79a0" title="Screenshot from 2021-10-13 16-21-51"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8b95623ed25be906a6f0687272986f117e1e79a0_2_521x499.png" alt="Screenshot from 2021-10-13 16-21-51" data-base62-sha1="jUOq8O1pthwziv1KfDS6RHN1AFW" width="521" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8b95623ed25be906a6f0687272986f117e1e79a0_2_521x499.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/b/8b95623ed25be906a6f0687272986f117e1e79a0_2_781x748.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/8/b/8b95623ed25be906a6f0687272986f117e1e79a0.png 2x" data-dominant-color="F5F5F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2021-10-13 16-21-51</span><span class="informations">885×849 74.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Chirag: <p>Got it! Thanks <a class="mention" href="/u/sayed_nadim">@Sayed_Nadim</a> , <a class="mention" href="/u/ptrblck">@ptrblck</a> .</p>