Mat1 and mat2 shapes cannot be multiplied (784x10 and 784x10)

Shaz: <p>Hi People,</p>
<p>I am using the following code to create a simple neural network but it gives me the following error: mat1 and mat2 shapes cannot be multiplied (784x10 and 784x10)</p>
<p>Not sure why this is happening as matrix sizes are same i.e. 784*10</p>
<p><span class="hashtag">#import</span> libraries<br>
%matplotlib inline<br>
import torch<br>
import torchvision<br>
import torchvision.transforms as transforms</p>
<p><span class="hashtag">#load</span> data<br>
transform = transforms.Compose(</p>
<pre><code>[transforms.ToTensor(),

 transforms.Normalize((0.5), (0.5))])
</code></pre>
<p>batch_size = 10</p>
<p>trainset = torchvision.datasets.MNIST(root=’./data’, train=True,</p>
<pre><code>                                    download=True, transform=transform)
</code></pre>
<p>trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size,</p>
<pre><code>                                      shuffle=True, num_workers=2)
</code></pre>
<p>testset = torchvision.datasets.MNIST(root=’./data’, train=False,</p>
<pre><code>                                   download=True, transform=transform)
</code></pre>
<p>testloader = torch.utils.data.DataLoader(testset, batch_size=batch_size,</p>
<pre><code>                                     shuffle=False, num_workers=2)
</code></pre>
<p><span class="hashtag">#wrap</span> iterator around the batch<br>
dataiter = iter(trainloader)</p>
<p>x_train, y_train = dataiter.next()</p>
<p>dataiter = iter(testloader)</p>
<p>x_valid, y_valid = dataiter.next()</p>
<p><span class="hashtag">#initialize</span> weights and bias<br>
import math</p>
<p>weights = torch.randn(784, 10) / math.sqrt(784)</p>
<p>weights.requires_grad_()</p>
<p>bias = torch.zeros(10, requires_grad=True)</p>
<p><span class="hashtag">#Define</span> loss function<br>
def log_softmax(x):</p>
<pre><code>return x - x.exp().sum(-1).log().unsqueeze(-1)
</code></pre>
<p>def model(xb):</p>
<pre><code>return log_softmax(xb @ weights + bias)
</code></pre>
<p><span class="hashtag">#create</span> prediction</p>
<p>bs = 64  # batch size</p>
<p>xb = x_train[0:bs]  # a mini-batch from x</p>
<p>xb=xb.reshape(784,10)</p>
<p>preds = model(xb)  # predictions</p>
<p>preds[0], preds.shape</p>
<p>print(preds[0], preds.shape)</p>

Shaz: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> can you plz help this code…thanx</p>

ptrblck: <aside class="quote no-group" data-username="Shaz" data-post="1" data-topic="118234">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/ba9def/48.png" class="avatar"> Shaz:</div>
<blockquote>
<p>Not sure why this is happening as matrix sizes are same i.e. 784*10</p>
</blockquote>
</aside>
<p>You could have to make sure that the inner matrix sizes are equal:</p>
<pre data-code-wrap="python"><code class="lang-python">x = torch.randn(784, 10)
y = torch.randn(10, 784)
out = torch.matmul(x, y)
print(out.shape)
&gt; torch.Size([784, 784])
</code></pre>

Shaz: <p>Hi Thank you for your reply.</p>
<p>I have made following changes but still get this error : “The size of tensor a (28) must match the size of tensor b (10) at non-singleton dimension 3”</p>
<p>weights = (torch.randn(28, 28) / math.sqrt(28))<br>
weights.requires_grad_()<br>
bias = torch.zeros((10), requires_grad=True)<br>
print(weights.shape)<br>
print(bias.shape)<br>
def log_softmax(x):</p>
<pre><code>return (x - x.exp().sum(-1).log().unsqueeze(-1))
</code></pre>
<p>def model(xb):</p>
<pre><code>return log_softmax(xb @ weights + bias)
</code></pre>
<p>bs = 1  # batch size</p>
<p>xb = x_train[0:bs]  # a mini-batch from x</p>
<p>preds = model(xb)  # predictions</p>
<p>preds[0], preds.shape</p>
<p>print(preds[0], preds.shape)</p>

Shaz: <p>If i change bias to 28, the error goes away but prediction result is a matrix of 28*28 instead of 10  probability values for y</p>