Rnn model accuracy not improving

Ramesh_Kumar: <p>Hello, I am using character level RNN to classify the names. I am basically using same example given in official documentation of Pytorch, but minor modification.<br>
However, overall accuracy is not improving (like it is jumping between 10 to 30%), and I can’t figure out the reason.<br>
Here is module class:</p>
<pre><code class="lang-auto">class Model(nn.Module):
    
    def __init__(self, input_size, hidden_size, output_size):
        super(Model, self).__init__()
        
        self.in_size = input_size
        self.h_size = hidden_size
        self.o_size = output_size
        
        # input to hidden layer
        self.i2h = nn.Linear(input_size+hidden_size, hidden_size)
        self.i2o = nn.Linear(input_size+hidden_size, output_size)
        
    def forward(self, inp, hidden):
        
        combined = torch.cat((inp, hidden), 1)
        hidden_unit = self.i2h(combined)
        output_unit = F.log_softmax(self.i2o(combined), dim=1)
        
        return output_unit, hidden_unit
        
    def initialize_hidden_layer(self):
        
        return Variable(torch.zeros(1, self.h_size))```

This is how my training function look like:

</code></pre>
<pre><code>def train_model(model, epoch, features, labels):
  count = 0
  accuracy = 0
  total_loss = 0
  for (feature, target) in zip(features, labels):
    
    hidden = model.initialize_hidden_layer()
    class_index = [i for i in range(len(classes_names)) if target==classes_names[i]][0]
    class_tensor = Variable(torch.LongTensor([class_index]))
    word_tensor = Variable(word_to_tensor(feature))
    for i in range(word_tensor.size()[0]):
        predicted_output, hidden = model(word_tensor[i], hidden)
    
    loss = criterion(predicted_output, class_tensor)
    predicted_label = find_label(predicted_output)
    total_loss += loss.data.item()
    
    if predicted_label[0] == target:
        accuracy += 1
        
    optimizer.zero_grad()
    
    #backpropagation
    loss.backward()
    
    #update params
    optimizer.step()
    count += 1
    
print ("count ", count)
print ("accurate predictions ", accuracy)
print ("total accuracy ", (accuracy/len(features)*100))
print ("total loss ", total_loss)


def get_all_examples():

  labels = []
  features = []
  for k, v in classes_dict.items():
    
    for name in v:
        labels.append(k)
        features.append(name)
        
   return features, labels
</code></pre>
<pre><code class="lang-auto">features, labels = get_all_examples()
shuffle_list = list(zip(features, labels))
random.shuffle(shuffle_list)
features, labels = zip(*shuffle_list)
model.train()
for epoch in range(10):
   train_model(model, epoch, features, labels)
</code></pre>
<p>Other auxiliary functions are same as present in the documentation.</p>

Abhilash_Srivastava: <ol>
<li>What criterion are you using?</li>
<li>See if using an LSTM instead of vanilla RNN helps.</li>
</ol>

Ramesh_Kumar: <p>Thanks for your response.<br>
I tried using <code>SGD</code> and <code>Adam</code>, but there is no such improvement. However, I am going to try using LSTM and will post here, if I face any problems <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Abhilash_Srivastava: <aside class="quote no-group" data-username="Ramesh_Kumar" data-post="1" data-topic="56731">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ramesh_kumar/48/12432_2.png" class="avatar"> Ramesh_Kumar:</div>
<blockquote>
<p>Variable</p>
</blockquote>
</aside>
<p>is deprecated: <a href="https://pytorch.org/docs/stable/autograd.html#variable-deprecated" class="inline-onebox" rel="noopener nofollow ugc">Automatic differentiation package - torch.autograd — PyTorch 2.1 documentation</a><br>
I think neither <code>class_tensor</code> and <code>word_tensor</code> need <code>requires_grad</code> to be set to True.</p>