RuntimeError: size mismatch, got 500, 500x8,1

username: mani_sohal
cooked: <p>X = torch.tensor(df_admission.values)<br>
class LinearRegressionGD():<br>
def <strong>init</strong>(self, nb_feats):<br>
super(LinearRegressionGD, self).<strong>init</strong>()<br>
# Create a bias torch.tensor and set it to 0<br>
# Crate a weight torch.tensor and set its values to 0’s<br>
self.nb_feats = nb_feats<br>
self.bias = torch.tensor(0.0, requires_grad=True)<br>
self.weights = torch.zeros(self.nb_feats, requires_grad=True)</p>
<p>def forward(self, X):<br>
# Do forward propagation to obtain yhat<br>
yhat = self.bias + torch.matmul(X, self.weights)<br>
return yhat</p>
<p>def backward(self, X, yhat, y):<br>
# Perform backward propagation to compute the gradients of the loss function<br>
# with respect to the bias and weights<br>
loss = MSELoss(yhat, y)<br>
loss.backward()</p>
<pre><code>return loss
</code></pre>
<p>def update(self, lr):<br>
# Update the bias and weights using the gradients computed in the backward pass<br>
with torch.no_grad():<br>
self.bias -= lr * self.bias.grad<br>
self.weights -= lr * self.weights.grad</p>
<pre><code>  # Set the gradients to zero
  self.bias.grad.zero_()
  self.weights.grad.zero_()
</code></pre>
<h1>
<a name="initialise-your-linear-regression-model-with-the-corresponding-number-of-features-1" class="anchor" href="#initialise-your-linear-regression-model-with-the-corresponding-number-of-features-1"></a>Initialise your Linear Regression model with the corresponding number of features.</h1>
<p>model = LinearRegressionGD(nb_feats=xTrainNorm.shape[1])</p>
<p>def trainManualGD(model, X, y, num_epochs, learning_rate=0.001):<br>
loss = []<br>
for epoch in range(num_epochs):</p>
<pre><code>    # 1. Perform forward propagation on the data samples and obtain the model's prediction.
    yhat = model.forward(X)

    # 2. Perform backward propagation to compute the gradients of the loss function over the weights, and over the bias term.
    grad_w, grad_bias = model.backward(X, y, yhat)

    # 3. Use gradient descent to update the weight and bias values: w = w - lr * grad_w
    model.weights = model.weights - learning_rate * grad_w
    model.bias = model.bias - learning_rate * grad_bias


    # 4. Performing forward propagation and computing the MSE loss for logging purposes
    yhat = model.forward(X) 
    curr_loss = MSELoss(yhat, y)
    loss.append(curr_loss)

    if not epoch % 100:
      print('Epoch: {:03d} | MSE: {:.7f}'.format(epoch+1, curr_loss))        

return loss
</code></pre>
<p>loss = trainManualGD(linear_model, X, y, num_epochs=10000, learning_rate=0.001)</p>
<p><strong>RuntimeError: size mismatch, got 500, 500x8,1</strong></p>

username: ptrblck
cooked: <p>I’m not familiar with your use case as you didn’t explain it but it seems your model definition is at least uncommon:</p>
<pre><code class="lang-python">class LinearRegressionGD(nn.Module):
    def __init__(self, nb_feats):
        super(LinearRegressionGD, self).__init__()
        # Create a bias torch.tensor and set it to 0
        # Crate a weight torch.tensor and set its values to 0’s
        self.nb_feats = nb_feats
        self.bias = torch.tensor(0.0, requires_grad=True)
        self.weights = torch.zeros(self.nb_feats, requires_grad=True)
        
    def forward(self, X):
        # Do forward propagation to obtain yhat
        yhat = self.bias + torch.matmul(X, self.weights)
        return yhat
</code></pre>
<p>In particular I would expect to see a 2D <code>weight</code> parameter while you are initializing it with a single dimension. Based on the error message I would also guess this line of code raises the error.<br>
In case you want to apply an affine transformation to the input you could directly use <code>nn.Linear</code> layers and specify the <code>in_features</code> as well as the <code>out_features</code>.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier.</p>