Model Training does not match the one that was shown in a tutorial

Paolo_Dano: <p>Good day!</p>
<p>Im new in pytorch and i am currently learning pytorch from a tutorial. The tutorial uses the kaggle advertising dataset here: <a href="https://www.kaggle.com/ashydv/advertising-dataset" class="inline-onebox" rel="noopener nofollow ugc">Advertising Dataset | Kaggle</a>. the Dataset has 4 columns and the last column is the one that needs prediction. In a normal model, it would be using a Linear Regression and R^2 would be around 80 to 90%. In the tutorial, the loss function decreases exponentially down (124, 32, 8, 6, 4, 3, 2, 1, 0). The teacher ended up with a 95% R^2 Score. However, when I copied the code down, it was returning a way bigger error (160, 67, 37, 34, 32, 31, 29, 28, 28). I even ended up with a -0.015 R^2. What could be the explanation for this? the code used is as shown below:</p>
<pre><code class="lang-auto">from sklearn import preprocessing
from sklearn.model_selection import train_test_split
import torch
import torch.optim as optim
import matplotlib.pyplot as plt

#Getting Data
advertising_data = pd.read_csv('advertising.csv')

#Scaling Data
advertising_data[['TV']] = preprocessing.scale(advertising_data[['TV']])
advertising_data[['Radio']] = preprocessing.scale(advertising_data[['Radio']])
advertising_data[['Newspaper']] = preprocessing.scale(advertising_data[['Newspaper']])

#Shuffling Data
advertising_data = advertising_data.sample(frac=1)

#Setting Features and Target
X = advertising_data.drop('Sales', axis=1)

Y = advertising_data[['Sales']]

#splitting Data
x_train, x_test, y_train ,y_test = train_test_split(X,y,test_size=0.2, random_state=0)

#Creating the Tensors
x_train_tensor = torch.tensor(x_train.values, dtype=torch.float)
x_test_tensor = torch.tensor(x_test.values, dtype=torch.float)

y_train_tensor = torch.tensor(y_train.values, dtype=torch.float)
y_test_tensor = torch.tensor(y_test.values, dtype=torch.float)

#Defining the Parameters
inp = 3
out = 1

hid = 100

loss_fn = torch.nn.MSELoss()

learning_rate = 0.0001

#Defining Model and Optimizer
model = torch.nn.Sequential(torch.nn.Linear(inp, hid),
                            torch.nn.ReLU(),
                            torch.nn.Linear(hid, out))

optimizer = optim.Adam(model.parameters(), lr=learning_rate)


#Training with 10000 epochs
for iter in range(10000):
  y_pred = model(x_train_tensor)
  loss = loss_fn(y_pred, y_train_tensor)
  if iter % 1000 == 0:
    print(iter, loss.item())
  optimizer.zero_grad()
  loss.backward()
  optimizer.step()


#Creating the predicted value and putting tensor into a numpy array
y_pred_tensor = model(x_test_tensor)
y_pred = y_pred_tensor.detach().numpy()


#Plotting the Prediction vs the Actual
plt.scatter(y_pred, y_test.values)

plt.xlabel('Actual Sale')
plt.ylabel('Predictied Sale')

plt.title('Predicted Sale vs Actual Sale')
plt.show()
</code></pre>
<p>I would end up with the Following outputs:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/4/d46d8b17bb63aee349d7cf5d6141fb7a021be7c7.png" data-download-href="https://discuss.pytorch.org/uploads/default/d46d8b17bb63aee349d7cf5d6141fb7a021be7c7" title="Screen Shot 2021-06-05 at 11.22.14 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/4/d46d8b17bb63aee349d7cf5d6141fb7a021be7c7_2_690x278.png" alt="Screen Shot 2021-06-05 at 11.22.14 PM" data-base62-sha1="ujdUZyr5QnfEb1U6HaYbUSKKldt" width="690" height="278" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/4/d46d8b17bb63aee349d7cf5d6141fb7a021be7c7_2_690x278.png, https://discuss.pytorch.org/uploads/default/optimized/3X/d/4/d46d8b17bb63aee349d7cf5d6141fb7a021be7c7_2_1035x417.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/d/4/d46d8b17bb63aee349d7cf5d6141fb7a021be7c7.png 2x" data-dominant-color="FAFBFB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2021-06-05 at 11.22.14 PM</span><span class="informations">1156×466 35.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Based on the provided screenshot it seems you have a shape mismatch in the loss calculation and one of the tensors is being broadcasted.<br>
Make sure both tensors (model output and target) have the same shape and rerun the code.</p>