Lstm training accuarcy stays same

username: Hu_Penglong
cooked: <p>I am new to Pytorch. Here is my problem,  I train a lstm classifier<br>
my training precision don’t increase in a tiny dataset.<br>
the classifier seems to guess the max num of label of the dataset for all the example.<br>
epoch 0 total_loss = 1.409505<br>
train data precision:58/100 = 58.00%<br>
valid data precision:92/401 = 22.94%<br>
save model<br>
epoch 1 total_loss = 1.146850<br>
train data precision:58/100 = 58.00%<br>
valid data precision:92/401 = 22.94%<br>
epoch 2 total_loss = 1.060287<br>
train data precision:58/100 = 58.00%<br>
valid data precision:92/401 = 22.94%<br>
epoch 3 total_loss = 1.019745<br>
train data precision:58/100 = 58.00%<br>
valid data precision:92/401 = 22.94%<br>
epoch 4 total_loss = 0.997164<br>
train data precision:59/100 = 59.00%<br>
valid data precision:92/401 = 22.94%<br>
save model<br>
epoch 5 total_loss = 0.983069<br>
train data precision:59/100 = 59.00%<br>
valid data precision:92/401 = 22.94%<br>
epoch 6 total_loss = 0.973470<br>
train data precision:59/100 = 59.00%<br>
valid data precision:92/401 = 22.94%<br>
epoch 7 total_loss = 0.966532<br>
train data precision:59/100 = 59.00%<br>
valid data precision:92/401 = 22.94%<br>
epoch 8 total_loss = 0.961307<br>
train data precision:59/100 = 59.00%<br>
valid data precision:92/401 = 22.94%<br>
here is my code.<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/lonelydancer/intent/blob/master/intent.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/lonelydancer/intent/blob/master/intent.py" target="_blank" rel="nofollow noopener">lonelydancer/intent/blob/master/intent.py</a></h4>
<pre><code class="lang-py">import sys
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torch.utils.data import Dataset, DataLoader
from torch.nn.utils.rnn import pad_sequence, pack_padded_sequence

train_filename = '../data/a'
#train_filename = '../data/train.data'
#train_filename = '../data/intent.data'
valid_filename = '../data/valid.data'
batch_size = 128
#batch_size = 16
input_dim = 128
hidden_dim = 64
train_model = sys.argv[1]
save_path = '../model/model.test'
epoch_num = 200

</code></pre>

  This file has been truncated. <a href="https://github.com/lonelydancer/intent/blob/master/intent.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

username: tom
cooked: <p>PyTorch RNNs take seq,batch,feature inputs unless you instantiate with batch_first=True. The full details are in the documentation.</p>
<p>Best regards</p>
<p>Thomas</p>

username: Hu_Penglong
cooked: <p><a class="mention" href="/u/tom">@tom</a><br>
my training precision don’t increase in a tiny dataset.<br>
the classifier seems to guess the max num of label of the dataset for all the example.<br>
epoch 0 total_loss = 1.409505<br>
train data precision:58/100 = 58.00%<br>
valid data precision:92/401 = 22.94%<br>
save model<br>
epoch 1 total_loss = 1.146850<br>
train data precision:58/100 = 58.00%<br>
valid data precision:92/401 = 22.94%<br>
epoch 2 total_loss = 1.060287<br>
train data precision:58/100 = 58.00%<br>
valid data precision:92/401 = 22.94%<br>
epoch 3 total_loss = 1.019745<br>
train data precision:58/100 = 58.00%<br>
valid data precision:92/401 = 22.94%<br>
epoch 4 total_loss = 0.997164<br>
train data precision:59/100 = 59.00%<br>
valid data precision:92/401 = 22.94%<br>
save model<br>
epoch 5 total_loss = 0.983069<br>
train data precision:59/100 = 59.00%<br>
valid data precision:92/401 = 22.94%<br>
epoch 6 total_loss = 0.973470<br>
train data precision:59/100 = 59.00%<br>
valid data precision:92/401 = 22.94%<br>
epoch 7 total_loss = 0.966532<br>
train data precision:59/100 = 59.00%<br>
valid data precision:92/401 = 22.94%<br>
epoch 8 total_loss = 0.961307<br>
train data precision:59/100 = 59.00%<br>
valid data precision:92/401 = 22.94%<br>
here is my code.<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/lonelydancer/intent/blob/master/intent.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/lonelydancer/intent/blob/master/intent.py" target="_blank" rel="nofollow noopener">lonelydancer/intent/blob/master/intent.py</a></h4>
<pre><code class="lang-py">import sys
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torch.utils.data import Dataset, DataLoader
from torch.nn.utils.rnn import pad_sequence, pack_padded_sequence

train_filename = '../data/a'
#train_filename = '../data/train.data'
#train_filename = '../data/intent.data'
valid_filename = '../data/valid.data'
batch_size = 128
#batch_size = 16
input_dim = 128
hidden_dim = 64
train_model = sys.argv[1]
save_path = '../model/model.test'
epoch_num = 200

</code></pre>

  This file has been truncated. <a href="https://github.com/lonelydancer/intent/blob/master/intent.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

username: DoubtWang
cooked: <p>I have met this problem,<br>
and my reason is the shape of output.<br>
You can check your code.</p>

username: Hu_Penglong
cooked: <p>i think my output shape is ok. (batch_num, cls_num)</p>

username: Hu_Penglong
cooked: <p>my learning rate is too large.<br>
thank you~</p>