PyTorch regression is producing the same numbers as prediction

username: abdalimran
cooked: <p>I have applied a simple NN  for regression to the traditional boston housing price dataset. The problem I’m getting is, when I make predictions using the trained model it always predicts the same numbers. Here is my code:</p>
<pre><code class="lang-auto">import numpy as  np
import pandas as pd
from sklearn import datasets
data = datasets.load_boston()

X = pd.DataFrame(data.data, columns=data.feature_names)
Y = pd.DataFrame(data.target, columns=["MEDV"])
</code></pre>
<pre><code class="lang-auto">from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.20, random_state=1234)
</code></pre>
<pre><code class="lang-auto">import torch

x_train = torch.tensor(X_train.values, dtype=torch.float)
y_train = torch.tensor(y_train.values, dtype=torch.float)
x_test = torch.tensor(X_test.values, dtype=torch.float)
y_test = torch.tensor(y_test.values, dtype=torch.float)
</code></pre>
<pre><code class="lang-auto">import torch.nn.functional as F

class Net(torch.nn.Module):
    def __init__(self, n_feature, n_hidden, n_output):
        super(Net, self).__init__()
        self.hidden = torch.nn.Linear(n_feature, n_hidden)
        self.predict = torch.nn.Linear(n_hidden, n_output)

    def forward(self, x):
        x = F.relu(self.hidden(x))
        x = self.predict(x)
        return x

net = Net(n_feature=13, n_hidden=50, n_output=1)
optimizer = torch.optim.SGD(net.parameters(), lr=0.2)
loss_func = torch.nn.MSELoss() 


for t in range(200):
    prediction = net(x_train)
    
    loss = loss_func(prediction, y_train)
    print(t, loss.item())
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
</code></pre>
<p><strong>Here is the loss per epoch log:</strong><br>
0 6414.029296875<br>
1 3.883837028532696e+19<br>
2 5.963952279684907e+18<br>
3 2.1470207536047063e+18<br>
4 7.729279495852524e+17<br>
5 2.7825424945486234e+17<br>
6 1.001715136546734e+17<br>
7 3.606170935335322e+16<br>
8 1.2982211888283648e+16<br>
9 4673591211720704.0<br>
10 1682493577101312.0<br>
11 605696890503168.0<br>
12 218050926215168.0<br>
13 78498328739840.0<br>
14 28259370663936.0<br>
15 10173364043776.0<br>
16 3662410940416.0<br>
17 1318467796992.0<br>
18 474648543232.0<br>
19 170873470976.0<br>
20 61514457088.0<br>
21 22145208320.0<br>
22 7972275200.0<br>
23 2870019072.0<br>
24 1033206912.0<br>
25 371954528.0<br>
26 133903616.0<br>
27 48205360.0<br>
28 17353982.0<br>
29 6247483.5<br>
30 2249145.25<br>
31 809743.25<br>
32 291558.65625<br>
33 105012.1171875<br>
34 37855.3984375<br>
35 13678.982421875<br>
36 4975.46826171875<br>
37 1842.20361328125<br>
38 714.2283325195312<br>
39 308.15716552734375<br>
40 161.97158813476562<br>
41 109.34477233886719<br>
42 90.39911651611328<br>
43 83.57867431640625<br>
44 81.12332153320312<br>
45 80.23938751220703<br>
46 79.92117309570312<br>
47 79.8066177368164<br>
48 79.76537322998047<br>
49 79.75053405761719<br>
50 79.74518585205078<br>
51 79.7432632446289<br>
52 79.74256896972656<br>
53 79.74231719970703<br>
54 79.74223327636719<br>
55 79.74219512939453<br>
56 79.7421875<br>
57 79.74217987060547<br>
58 79.74217987060547<br>
59 79.74217987060547<br>
60 79.74217987060547<br>
61 79.74217987060547<br>
62 79.74217987060547<br>
63 79.74217987060547<br>
64 79.74217987060547<br>
65 79.74217987060547<br>
66 79.74217987060547<br>
67 79.74217987060547<br>
68 79.74217987060547<br>
69 79.74217987060547<br>
70 79.74217987060547<br>
71 79.74217987060547<br>
72 79.74217987060547<br>
73 79.74217987060547<br>
74 79.74217987060547<br>
75 79.74217987060547<br>
76 79.74217987060547<br>
77 79.74217987060547<br>
78 79.74217987060547<br>
79 79.74217987060547<br>
80 79.74217987060547<br>
81 79.74217987060547<br>
82 79.74217987060547<br>
83 79.74217987060547<br>
84 79.74217987060547<br>
85 79.74217987060547<br>
86 79.74217987060547<br>
87 79.74217987060547<br>
88 79.74217987060547<br>
89 79.74217987060547<br>
90 79.74217987060547<br>
91 79.74217987060547<br>
92 79.74217987060547<br>
93 79.74217987060547<br>
94 79.74217987060547<br>
95 79.74217987060547<br>
96 79.74217987060547<br>
97 79.74217987060547<br>
98 79.74217987060547<br>
99 79.74217987060547<br>
100 79.74217987060547<br>
101 79.74217987060547<br>
102 79.74217987060547<br>
103 79.74217987060547<br>
104 79.74217987060547<br>
105 79.74217987060547<br>
106 79.74217987060547<br>
107 79.74217987060547<br>
108 79.74217987060547<br>
109 79.74217987060547<br>
110 79.74217987060547<br>
111 79.74217987060547<br>
112 79.74217987060547<br>
113 79.74217987060547<br>
114 79.74217987060547<br>
115 79.74217987060547<br>
116 79.74217987060547<br>
117 79.74217987060547<br>
118 79.74217987060547<br>
119 79.74217987060547<br>
120 79.74217987060547<br>
121 79.74217987060547<br>
122 79.74217987060547<br>
123 79.74217987060547<br>
124 79.74217987060547<br>
125 79.74217987060547<br>
126 79.74217987060547<br>
127 79.74217987060547<br>
128 79.74217987060547<br>
129 79.74217987060547<br>
130 79.74217987060547<br>
131 79.74217987060547<br>
132 79.74217987060547<br>
133 79.74217987060547<br>
134 79.74217987060547<br>
135 79.74217987060547<br>
136 79.74217987060547<br>
137 79.74217987060547<br>
138 79.74217987060547<br>
139 79.74217987060547<br>
140 79.74217987060547<br>
141 79.74217987060547<br>
142 79.74217987060547<br>
143 79.74217987060547<br>
144 79.74217987060547<br>
145 79.74217987060547<br>
146 79.74217987060547<br>
147 79.74217987060547<br>
148 79.74217987060547<br>
149 79.74217987060547<br>
150 79.74217987060547<br>
151 79.74217987060547<br>
152 79.74217987060547<br>
153 79.74217987060547<br>
154 79.74217987060547<br>
155 79.74217987060547<br>
156 79.74217987060547<br>
157 79.74217987060547<br>
158 79.74217987060547<br>
159 79.74217987060547<br>
160 79.74217987060547<br>
161 79.74217987060547<br>
162 79.74217987060547<br>
163 79.74217987060547<br>
164 79.74217987060547<br>
165 79.74217987060547<br>
166 79.74217987060547<br>
167 79.74217987060547<br>
168 79.74217987060547<br>
169 79.74217987060547<br>
170 79.74217987060547<br>
171 79.74217987060547<br>
172 79.74217987060547<br>
173 79.74217987060547<br>
174 79.74217987060547<br>
175 79.74217987060547<br>
176 79.74217987060547<br>
177 79.74217987060547<br>
178 79.74217987060547<br>
179 79.74217987060547<br>
180 79.74217987060547<br>
181 79.74217987060547<br>
182 79.74217987060547<br>
183 79.74217987060547<br>
184 79.74217987060547<br>
185 79.74217987060547<br>
186 79.74217987060547<br>
187 79.74217987060547<br>
188 79.74217987060547<br>
189 79.74217987060547<br>
190 79.74217987060547<br>
191 79.74217987060547<br>
192 79.74217987060547<br>
193 79.74217987060547<br>
194 79.74217987060547<br>
195 79.74217987060547<br>
196 79.74217987060547<br>
197 79.74217987060547<br>
198 79.74217987060547<br>
199 79.74217987060547</p>
<p><strong>After training the model, I’ve used the following code to make predictions:</strong></p>
<pre><code class="lang-auto">with torch.no_grad():
    y_val = net(x_test)
</code></pre>
<p><strong>When I print the predictions I get the following results:</strong></p>
<p>tensor([[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099],<br>
[22.4099]])</p>

username: ptrblck
cooked: <p>I would assume the learning rate is too high, as the loss in the second iteration is very high. Also, your model gets stuck at after 50 epochs and I assume the training output is also constant from this point?<br>
Did you normalize the data? This could help training the model.</p>