All prediction values identical when using model for inference

DaftJamrock: <p>I am using a saved model for inference. When I used it to predict values, the values converge very quickly and the majority are predicted as the same value. This is even though my input dataset has significant variability. The loss decreases when training.</p>
<p>I am not sure why this is happening, any suggestions would be much appreciated!</p>
<p>Here is my code and some sample input data and output data:</p>
<pre><code class="lang-auto">model_state = torch.load(model_path)
model = LSTMModel(model_state['input_dim'], model_state['hidden_dim'],
                                model_state['num_layers'], model_state['output_dim'])
model.load_state_dict(model_state['state_dict'])
for parameter in model.parameters():
     parameter.requires_grad = False

model.eval()

num_features = (len(data.columns)-1)
actual_data = torch.tensor(data[target_col].values).float()
prediction_data = torch.tensor(data.drop(columns=target_col).values).float()
actual_data = actual_data.view(-1, 1, 1)
prediction_data = prediction_data.view(-1, 1, num_features)

prediction = model(prediction_data)
prediction = prediction.data.numpy()
</code></pre>
<p>Input Data:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/1/21c426ec450f85d1c4e835944fac3069b174389a.png" data-download-href="https://discuss.pytorch.org/uploads/default/21c426ec450f85d1c4e835944fac3069b174389a" title="28%20PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/1/21c426ec450f85d1c4e835944fac3069b174389a_2_519x500.png" alt="28%20PM" data-base62-sha1="4OI0DJR8QuQ15RNuZOp24uK2XQ6" width="519" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/1/21c426ec450f85d1c4e835944fac3069b174389a_2_519x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/1/21c426ec450f85d1c4e835944fac3069b174389a_2_778x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/1/21c426ec450f85d1c4e835944fac3069b174389a.png 2x" data-dominant-color="F4F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">28%20PM</span><span class="informations">996×958 79.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Output predictions:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/c/1c915228f0a04462c6d9366768cba09b0090b0c2.png" data-download-href="https://discuss.pytorch.org/uploads/default/1c915228f0a04462c6d9366768cba09b0090b0c2" title="30%20PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/c/1c915228f0a04462c6d9366768cba09b0090b0c2_2_140x500.png" alt="30%20PM" data-base62-sha1="44IIvosOn58TF0sv2itf3AL8KUa" width="140" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/c/1c915228f0a04462c6d9366768cba09b0090b0c2_2_140x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/1/c/1c915228f0a04462c6d9366768cba09b0090b0c2_2_210x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/c/1c915228f0a04462c6d9366768cba09b0090b0c2_2_280x1000.png 2x" data-dominant-color="D9D9D9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">30%20PM</span><span class="informations">314×1118 123 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Have you seen the same effect during validation?<br>
Do you handle the validation and test set in the same manner, i.e. regarding the initial hidden state etc.?</p>

DaftJamrock: <p>Yes I am seeing the same effect in validation and I am handling both sets in the same manner.<br>
For clarity my LSTM has 10 neurons per layer, 1 hidden layer, a learning rate of 0.001 and 20000 training epochs. Might it be that my network is not deep enough?<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/4/f/4f7a46692f8d133dd8d710e031fe57b818398bde.png" alt="20000%20epochs" data-base62-sha1="bl5GYNvHc6QnAklGnj7sQsNVZJY" width="320" height="240"><br>
Attached is a figure showing the output I am receiving.<br>
The orange line is the actual data and the blue line is my forecast.</p>