Getting this RuntimeError:Cudnn RNN backward can only be called in training mode

username: Sajid_Ahmed
cooked: <p>I’m using Pytorch 1.0 Pythn 3.7.3 Cuda 10.0 CudNn 7.5  got This ERROR</p>
<p>Traceback (most recent call last):<br>
File “main.py”, line 96, in <br>
main(args)<br>
File “main.py”, line 70, in main<br>
disc_loss, gen_loss = trainer.train_one_epoch_adversarial()<br>
File “C:\Users\CSE495-NBM\Desktop\unmt_2\libs\trainer.py”, line 432, in train_one_epoch_adversarial<br>
loss.backward()<br>
File “C:\Users\CSE495-NBM\Miniconda3\lib\site-packages\torch\tensor.py”, line 107, in backward<br>
torch.autograd.backward(self, gradient, retain_graph, create_graph)<br>
File “C:\Users\CSE495-NBM\Miniconda3\lib\site-packages\torch\autograd_<em>init</em>_.py”, line 93, in backward<br>
allow_unreachable=True)  # allow_unreachable flag<br>
RuntimeError: cudnn RNN backward can only be called in training mode</p>

username: Prerna_Dhareshwar
cooked: <p>Hi Sajid,</p>
<p>Could you post some of your code? Are you doing <code>model.eval()</code> at some point before calling backward?</p>

username: aknirala
cooked: <p>I believe it will work by  calling trainer.train() before:</p>
<p>disc_loss, gen_loss = trainer.train_one_epoch_adversarial()</p>

username: Thabang_Lukhetho
cooked: <p>Make sure that training and testing procedure is as follows:</p>
<pre><code class="lang-auto">
model = my_model()

for epoch in Epochs:
  model.train()
  for train_batch in Batches:
            ######Train
  model.eval()
  for test_batch in batches:
            ######Test


 







</code></pre>

username: Reshma_Jindal
cooked: <pre><code class="lang-auto">Loading model parameters.
/usr/local/lib/python3.7/dist-packages/torchtext/data/field.py:197: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  return Variable(arr, volatile=not train), lengths
/usr/local/lib/python3.7/dist-packages/torchtext/data/field.py:198: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  return Variable(arr, volatile=not train)
/content/Seq2Sick/onmt/translate/Translator.py:48: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  def var(a): return Variable(a, volatile=True)
/content/Seq2Sick/onmt/modules/GlobalAttention.py:179: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  align_vectors = self.sm(align.view(batch*targetL, sourceL))
/usr/local/lib/python3.7/dist-packages/torch/nn/modules/container.py:119: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
  input = module(input)
/content/Seq2Sick/onmt/translate/Translator.py:191: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  src.volatile = False
attack.py:64: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
  output_a, attn, output_i= translator.getOutput(new_embedding, src, batch)
tensor(18.6335, device='cuda:0') 	 tensor(0., device='cuda:0')
tensor(999., device='cuda:0') 	 tensor(0., device='cuda:0')
Traceback (most recent call last):
  File "attack.py", line 312, in &lt;module&gt;
    main()
  File "attack.py", line 272, in main
    modifier, output_a, attn, new_word, output_i, CFLAG = attack(all_word_embedding, label_onehot, translator, src, batch, new_embedding, input_embedding, modifier, const, GROUP_LASSO, TARGETED, GRAD_REG, NN)
  File "attack.py", line 138, in attack
    loss.backward(retain_graph=True)
  File "/usr/local/lib/python3.7/dist-packages/torch/tensor.py", line 245, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/usr/local/lib/python3.7/dist-packages/torch/autograd/__init__.py", line 147, in backward
    allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag
RuntimeError: cudnn RNN backward can only be called in training mode
</code></pre>

username: Reshma_Jindal
cooked: <aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/cmhcbb/Seq2Sick/issues/2" target="_blank" rel="noopener nofollow ugc">github.com/cmhcbb/Seq2Sick</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/cmhcbb/Seq2Sick/issues/2" target="_blank" rel="noopener nofollow ugc">RuntimeError: cudnn RNN backward can only be called in training mode</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2021-05-30" data-time="09:19:44" data-timezone="UTC">09:19AM - 30 May 21 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/reshmajindal" target="_blank" rel="noopener nofollow ugc">
          <img alt="reshmajindal" src="https://avatars.githubusercontent.com/u/85056408?v=4" class="onebox-avatar-inline" width="20" height="20">
          reshmajindal
        </a>
      </div>
    </div>

    <div class="labels">
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">Loading model parameters.
/usr/local/lib/python3.7/dist-packages/torchtext/data<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">/field.py:197: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  return Variable(arr, volatile=not train), lengths
/usr/local/lib/python3.7/dist-packages/torchtext/data/field.py:198: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  return Variable(arr, volatile=not train)
/content/Seq2Sick/onmt/translate/Translator.py:48: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  def var(a): return Variable(a, volatile=True)
/content/Seq2Sick/onmt/modules/GlobalAttention.py:179: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  align_vectors = self.sm(align.view(batch*targetL, sourceL))
/usr/local/lib/python3.7/dist-packages/torch/nn/modules/container.py:119: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
  input = module(input)
/content/Seq2Sick/onmt/translate/Translator.py:191: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  src.volatile = False
attack.py:64: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
  output_a, attn, output_i= translator.getOutput(new_embedding, src, batch)
tensor(18.6335, device='cuda:0') 	 tensor(0., device='cuda:0')
tensor(999., device='cuda:0') 	 tensor(0., device='cuda:0')
Traceback (most recent call last):
  File "attack.py", line 312, in &lt;module&gt;
    main()
  File "attack.py", line 272, in main
    modifier, output_a, attn, new_word, output_i, CFLAG = attack(all_word_embedding, label_onehot, translator, src, batch, new_embedding, input_embedding, modifier, const, GROUP_LASSO, TARGETED, GRAD_REG, NN)
  File "attack.py", line 138, in attack
    loss.backward(retain_graph=True)
  File "/usr/local/lib/python3.7/dist-packages/torch/tensor.py", line 245, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/usr/local/lib/python3.7/dist-packages/torch/autograd/__init__.py", line 147, in backward
    allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag
RuntimeError: cudnn RNN backward can only be called in training mode</span></p>
  </div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p><a class="mention" href="/u/prerna_dhareshwar">@Prerna_Dhareshwar</a> <a class="mention" href="/u/sajid_ahmed">@Sajid_Ahmed</a></p>
<p>Please have a look at my issue mentioned above</p>

username: Reshma_Jindal
cooked: <p><a class="mention" href="/u/thabang_lukhetho">@Thabang_Lukhetho</a> <a class="mention" href="/u/aknirala">@aknirala</a></p>
<p>Please have a look at my issue mentioned above</p>

username: ptrblck
cooked: <p>Answered in your <a href="https://github.com/cmhcbb/Seq2Sick/issues/2#issuecomment-851078120">cross post</a>.</p>