Using LSTM stateful for passing context b/w batches; may be some error in context passing, not getting good results?

username: harsh_grover
cooked: <p>I have checked the data before giving it to network. Data is correct.</p>
<p>I have multi-class classification problem based upon an imbalanced dataset</p>
<p>Dataset_type: CSV</p>
<p>Dataset_size: 20000</p>
<p>Based upon CSV data of sensors</p>
<p>X = 0.6986111111111111,0,0,1,0,1,0,0,0,1,0,0,0,0,1,0,0,0,1,1,0,0,0</p>
<p>Y = leaveHouse</p>
<p>Per class accuracy:<br>
{‘leaveHouse’: 0.34932855, ‘getDressed’: 1.0, ‘idle’: 0.8074534, ‘prepareBreakfast’: 0.8, ‘goToBed’: 0.35583413, ‘getDrink’: 0.0, ‘takeShower’: 1.0, ‘useToilet’: 0.0, ‘eatBreakfast’: 0.8857143}</p>
<h1>Training:</h1>
<pre><code class="lang-auto"># Using loss weights, the inverse of class frequency

criterion = nn.CrossEntropyLoss(weight = class_weights)

 hn, cn = model.init_hidden(batch_size)
            for i, (input, label) in enumerate(trainLoader):
                hn.detach_()
                cn.detach_()
                input = input.view(-1, seq_dim, input_dim)

                if torch.cuda.is_available():
                    input = input.float().cuda()
                    label = label.cuda()
                else:
                    input = input.float()
                    label = label

                # Forward pass to get output/logits
                output, (hn, cn) = model((input, (hn, cn)))

                # Calculate Loss: softmax --&gt; cross entropy loss
                loss = criterion(output, label)#weig pram
                running_loss += loss
                loss.backward()  # Backward pass
                optimizer.step()  # Now we can do an optimizer step
                optimizer.zero_grad()  # Reset gradients tensors
</code></pre>
<h1>Network</h1>
<pre><code class="lang-auto">class LSTMModel(nn.Module):
    def init_hidden(self, batch_size):
        self.batch_size = batch_size
        if torch.cuda.is_available():
            hn = torch.zeros(self.layer_dim, self.batch_size, self.hidden_dim).cuda()
            # Initialize cell state
            cn = torch.zeros(self.layer_dim, self.batch_size, self.hidden_dim).cuda()
        else:
            hn = torch.zeros(self.layer_dim, self.batch_size, self.hidden_dim)
            # Initialize cell state
            cn = torch.zeros(self.layer_dim, self.batch_size, self.hidden_dim)
        return hn, cn

    def __init__(self, input_dim, hidden_dim, layer_dim, output_dim, seq_dim):
        super(LSTMModel, self).__init__()
        # Hidden dimensions
        self.hidden_dim = hidden_dim

        # Number of hidden layers
        self.layer_dim = layer_dim

        self.input_dim = input_dim
        # Building your LSTM
        # batch_first=True causes input/output tensors to be of shape
        # (batch_dim, seq_dim, feature_dim)
        self.lstm = nn.LSTM(self.input_dim, hidden_dim, layer_dim, batch_first=True)

        # Readout layer
        self.fc = nn.Linear(hidden_dim, output_dim)
        self.relu = nn.ReLU()
        self.softmax = nn.Softmax(dim=1)
        self.seq_dim = seq_dim
</code></pre>

username: harsh_grover
cooked: <p>I solved it, error was that Pytorch implicitly applies softmax when you use cross entropy and I was applying it again and also the dim =1 part was wrong, since it was a seq output.</p>