RuntimeError: mat1 and mat2 shapes cannot be multiplied in regression neural network

Bloopit: <p>I am trying to use a neural network to do regression on a large 5D dataset, however to try and get a working neural network first i have setup a random set of data points to try with. However I am running into this error and I am not sure why.</p>
<p>Error Code:<br>
predicted = model(data.to(device))<br>
File “C:\Users\Owner\anaconda3\lib\site-packages\torch\nn\modules\module.py”, line 1051, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
File “C:\Users\Owner\anaconda3\lib\site-packages\torch\nn\modules\container.py”, line 139, in forward<br>
input = module(input)<br>
File “C:\Users\Owner\anaconda3\lib\site-packages\torch\nn\modules\module.py”, line 1051, in _call_impl<br>
return forward_call(*input, **kwargs)<br>
File “C:\Users\Owner\anaconda3\lib\site-packages\torch\nn\modules\linear.py”, line 96, in forward<br>
return F.linear(input, self.weight, self.bias)<br>
File “C:\Users\Owner\anaconda3\lib\site-packages\torch\nn\functional.py”, line 1847, in linear<br>
return torch._C._nn.linear(input, weight, bias)<br>
RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x64 and 1x200)</p>
<p>Process finished with exit code 1</p>
<p>I was roughly following a tutorial I found online: <a href="https://medium.com/@benjamin.phillips22/simple-regression-with-neural-networks-in-pytorch-313f06910379" class="inline-onebox" rel="noopener nofollow ugc">Regression with Neural Networks in PyTorch | by Ben Phillips | Medium</a></p>
<p>My Code:</p>
<pre><code class="lang-auto">trial_x_data = np.arange(100)
trial_y_data = trial_x_data ** 2 + 5 

model = torch.nn.Sequential(
        torch.nn.Linear(1, 200),
        torch.nn.LeakyReLU(),
        torch.nn.Linear(200, 100),
        torch.nn.LeakyReLU(),
        torch.nn.Linear(100, 1),
    ).to(device)

optimiser = torch.optim.Adam(model.parameters(), lr=0.01)
loss_func = torch.nn.MSELoss()


Batch_size = 64
Max_Epoch = 500

trial_x_data, trial_y_data = torch.tensor(trial_x_data), torch.tensor(trial_y_data)
torch_dataset = Data.TensorDataset(trial_x_data, trial_y_data)

loader = Data.DataLoader(dataset=torch_dataset, batch_size=Batch_size, shuffle=True)

loss_logger = []

for epoch in range(Max_Epoch):
    for idx, (data, expected) in enumerate(loader):
        predicted = model(data.to(device))
        loss = loss_func(predicted, expected.to(device))

        optimiser.zero_grad()
        loss.backward()
        optimiser.step()

        loss_logger.append(loss.item())

    print("Epoch: [%d/%d]" % (epoch + 1, Max_Epoch))
</code></pre>
<p>Any help would be greatly appreciated</p>

thecho7: <p>Your input <code>data</code> should be transposed.</p>
<p>According to the basic matrix multiplication: (A, B) x (B, C) = (A, C)</p>
<pre><code class="lang-auto">for idx, (data, expected) in enumerate(loader):
    data = data.transpose(1, 0)
    predicted = model(data.to(device))
</code></pre>

Bloopit: <p>Thanks for the reply, I’m not sure that is the root cause cos when I do transpose “data”, the error remains unchanged.</p>
<p>I have updated the error message so it might have more info.</p>

thecho7: <p>Would you print a shape of data by <code>print(data.shape)</code>?<br>
If it has 3-dimensions including batch then <code>data = data.transpose(2, 1)</code></p>
<pre><code class="lang-auto">if len(data.shape) == 3:
    # (A, B, C) --&gt; (A, C, B)
    data = data.transpose(2, 1)
elif len(data.shape) == 2:
    # (A, B) --&gt; (B, A)
    data = data.transpose(1, 0)
else:
    pass</code></pre>

Bloopit: <p>The shape of the data is apparently: torch.Size([64])</p>

thecho7: <p>Then it gonna work with <code>data = data.unsqueeze(1)</code>.</p>

Bloopit: <p>I have done that, and I got this error afterwords: RuntimeError: expected scalar type Float but found Int<br>
So I then added did <code>data = data.unsqueeze(1).float()</code>. But when I do this I get <code>RuntimeError: Found dtype Int but expected Float</code> in the loss.backward() part.</p>
<p>Thanks for all the help so far</p>

thecho7: <p>MSE takes inputs of float type.<br>
<code>data = data.unsqueeze(1).type(torch.float)</code><br>
<code>expected = expected.unsqueeze(1).type(torch.float)</code></p>
<p>Then move them to GPU by <code>.to(device)</code></p>

Bloopit: <p>Hi<br>
That works thanks for all the help.</p>
<p>On a quick side questions, if I want to test my model with another set of x points, is this the right code: <code>model(torch.tensor(x_points).to(device)).cpu().detach().clone().numpy()</code>.</p>
<p>Also how would you recommend improving the model.</p>
<p>Thanks</p>

thecho7: <p>I think clone is not necessary.<br>
Your approach is simply ok.</p>