Only one category classified on the whole dataset in the confusion matrix

username: Chame_call
cooked: <p>I’ve been trying to train model to classify surname language with a char-level rnn.<br>
But model during 10 epochs shows almost the same train and validation loss.<br>
And in confusion matrix all predicts point at only one column.<br>
That’s my <a href="https://colab.research.google.com/drive/1kqzS4AR55txl8dSHSwFsOOuzS_D17rdL" rel="nofollow noopener">notebook on colab</a>.</p>
<p>What am I doing wrong?<br>
Can someone give an advice?</p>

username: JamesTrick
cooked: <p>Hi Alger_none!</p>
<p>I’ve only had a really quick look at your code - I can have a more in depth look if this suggestion doesn’t work.</p>
<p>Specifically, I’m looking at your <code>train</code> function. It looks as though you’re trying to <code>zero_grad()</code> your model - not your optimizer. My suggestion would be to refactor your train function have:</p>
<ul>
<li>zero_grad() your optimizer, to avoid accumulating gradients. Usually do this at the start of your training loop.</li>
<li>
<code>step()</code> your optimizer after `loss.backward()</li>
</ul>
<p>This <a href="https://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#sphx-glr-beginner-blitz-neural-networks-tutorial-py" rel="nofollow noopener">tutorial</a> should help you out!</p>

username: Chame_call
cooked: <p>Actually my notebook is modified version of <a href="https://pytorch.org/tutorials/intermediate/char_rnn_classification_tutorial.html" rel="nofollow noopener">that nice notebook</a> in which there’s no optimizer and SGD is obtained by manual subtracting of gradients.</p>
<p>This way first we zero our net grads:</p>
<pre><code class="lang-auto">rnn.zero_grad()
</code></pre>
<p>After that we get output of net and calculate our loss:</p>
<p><code>loss = criterion(output, category_tensor)</code><br>
Next we calculate derivatives via backward():</p>
<pre><code class="lang-auto">loss.backward()
</code></pre>
<p>and finally manual subtracting:</p>
<pre><code class="lang-auto">for p in rnn.parameters():
        p.data.add_(-learning_rate, p.grad.data)
</code></pre>
<p>What I’ve changed in that is that I replace random data point choosing by creating train/validation datasets and add loss output.<br>
But I can’t find the place where I spoil something.<br>
Can you pls take another look at that?</p>

username: Chame_call
cooked: <p>I forgot to reply your message but my previous message is  exactly that:grinning:</p>

username: Chame_call
cooked: <p>And after every such training I got different confusion matrices which have the same pattern:<br>
only one category was classified for whole dataset (maybe with some noise).</p>
<p>Examples:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/e/0/e0bff24edc8ededb1fde112d4e83dabcb9054af0.png" alt="plot" data-base62-sha1="w4eccRnIkUqAdGYuYR9X1CQkQHm" width="432" height="288"></p>

username: Chame_call
cooked: <p><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/a/da4de323ae6b3fb4ea7691701215ec456e159e5e.png" alt="plot%20(1)" data-base62-sha1="v9cXB2Ix3K3pkQFe3LrcHmkXQWq" width="432" height="288"><br>
and so forth</p>

username: Chame_call
cooked: <p>Also I’ve got the following losses plot:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/9/2/92a0160db099e0baf1c5285c6e786cc9fcdd7ea5.png" alt="losses" data-base62-sha1="kV6H0tycxJNGeSnie2iy2oDxXEx" width="432" height="288"></p>

username: Chame_call
cooked: <p>It seems to me that a trouble related with the hidden state but I’m not sure about that.<br>
Any thoughts pls??</p>

username: beaupreda
cooked: <p>I am not really familiar with RNNs, but looking at your code, I see hidden.detach_(), which removes the variable from the computation graph and therefore may cause your error. Here’s a link that explains what detach() does.</p>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <a href="http://www.bnikolic.co.uk/blog/pytorch-detach.html" target="_blank" title="12:00AM - 14 November 2018" rel="nofollow noopener">B. Nikolic Software and Computing Blog – 14 Nov 18</a>
  </header>
  <article class="onebox-body">
    <img src="" class="thumbnail">

<h3><a href="http://www.bnikolic.co.uk/blog/pytorch-detach.html" target="_blank" rel="nofollow noopener">PyTorch .detach() method</a></h3>

<p>In order to enable automatic differentiation, PyTorch keeps track of all operations involving tensors for which the gradient may need to be computed (i.e., require_grad is True). The operations are recorded as a directed graph. The detach() method...</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


username: Chame_call
cooked: <p>But It has the same effect even without that, that’s the problem(<br>
Any other thoughts?</p>

username: beaupreda
cooked: <p>Unfortunately, not really (not a lot of experience with RNNs). Were you able to reproduce the results from the tutorial that you linked earlier?</p>

username: Chame_call
cooked: <p>Yes, I did.<br>
But there were no data set splitting into train/valid and data points were chosen by random.<br>
So I only added data splitting.<br>
The most confusing moment is that in confusion matrix is “active” only one category column (screens above) after every next training</p>

username: Chame_call
cooked: <p>And from the first epoch validation loss is increasing while training loss is decreasing</p>

username: beaupreda
cooked: <p>The loss difference between training and validation suggests overfitting, which is normal considering that your model seems to always be predicting the same class, as shown in the confusion matrix.</p>
<p>I’m afraid I can’t be of more help, but I would try to find where your model and the one from the tutorial diverge (check if the variable hidden is similar in both cases).</p>
<p>Best of luck!</p>

username: Chame_call
cooked: <p>That’s a pity(<br>
I’ll try to track the divergence…</p>