Same prediction for different values in GRU model

username: Harardin
cooked: <p>Made a simple GRU model for prediction.</p>
<p>Example of prediction and values:</p>
<pre><code class="lang-auto">Prediction: [[73.24892]] | Value: [[73.86]]
Prediction: [[73.24892]] | Value: [[74.01]]
Prediction: [[73.24892]] | Value: [[73.5825]]
Prediction: [[73.24892]] | Value: [[76.3825]]
Prediction: [[73.24892]] | Value: [[77.]]
Prediction: [[73.24892]] | Value: [[74.0375]]
Prediction: [[73.24892]] | Value: [[75.7375]]
Prediction: [[73.24892]] | Value: [[76.1975]]
Prediction: [[73.24892]] | Value: [[76.21]]
Prediction: [[73.24892]] | Value: [[77.5575]]
Prediction: [[73.24892]] | Value: [[73.7875]]
</code></pre>
<p>Model:</p>
<pre><code class="lang-auto">class GRU_Model_Renued(nn.Module):
    def __init__(self, input_dim, hidden_dim, layer_dim, output_dim, dropout_prob):
        super(GRU_Model_Renued, self).__init__()

        # Defining the number of layers and the nodes in each layer
        self.layer_dim = layer_dim
        self.hidden_dim = hidden_dim

        # GRU layers
        self.gru = nn.GRU(
            input_dim, hidden_dim, layer_dim, batch_first=True, dropout=dropout_prob
        )

        # Fully connected layer
        self.fc = nn.Linear(hidden_dim, output_dim)

    def forward(self, x):
        # Initializing hidden state for first input with zeros
        h0 = torch.zeros(self.layer_dim, x.size(0), self.hidden_dim).requires_grad_()

        # Forward propagation by passing in the input and hidden state into the model
        out, _ = self.gru(x, h0.detach())

        # Reshaping the outputs in the shape of (batch_size, seq_length, hidden_size)
        # so that it can fit into the fully connected layer
        out = out[:, -1, :]

        # Convert the final state to our desired output shape (batch_size, output_dim)
        out = self.fc(out)

        return out
</code></pre>
<p>And how I train it:</p>
<pre><code class="lang-auto">def train_step(self, x, y):
        # Sets model to train mode
        self.model.train()

        # Makes predictions
        yhat = self.model(x)

        # Computes loss
        loss = self.loss_fn(y, yhat)

        # Computes gradients
        loss.backward()

        # Updates parameters and zeroes gradients
        self.optimizer.step()
        self.optimizer.zero_grad()

        # Returns the loss
        return loss.item()

    # Train function
    def train(self, train_loader, val_loader, batch_size=20, num_epochs=10, n_features=1):
        for epoch in range(num_epochs):
            self.model.train()
            batch_loses = []
            for x_batch, y_batch in train_loader:
                # x_batch = x_batch.view([batch_size, -1, n_features])
                loss = self.train_step(x_batch, y_batch)
                batch_loses.append(loss)
            training_loss = np.mean(batch_loses)
            self.train_losses.append(training_loss)
            print(f'[{epoch}/{num_epochs}] Train loss: {training_loss:.4f}')

            with torch.no_grad():
                self.model.eval()
                batch_val_losses = []
                for x_val, y_val in val_loader:
                    # x_val = x_val.view([batch_size, -1, n_features])
                    ev = self.model(x_val)
                    val_loss = self.loss_fn(y_val, ev).item()
                    batch_val_losses.append(val_loss)
                validation_loss = np.mean(batch_val_losses)
                self.val_losses.append(validation_loss)
                print(f'Validation loss: {validation_loss:.4f}')
        
        # Saving model in case something drops
        torch.save(self.model.state_dict(), f'models/{self.model_name}')
</code></pre>
<p>I don’t understand why predictions for different values are always the same. I am quite new in deep learning can, someone please point me in right direction.</p>
<p>For prediction I use this function:</p>
<pre><code class="lang-auto">def evaluate(self, test_loader, batch_size=1, n_features=1):
        with torch.no_grad():
            predictions = []
            values = []
            for x_test, y_test in test_loader:
                self.model.eval()
                yhat = self.model(x_test)
                predictions.append(yhat.numpy())
                values.append(y_test.numpy())

        return predictions, values
</code></pre>