Autograd not registering tensors : does not require grad and does not have a grad_fn

username: nilansh_rajput1
cooked: <pre><code class="lang-auto">loss = nn.CrossEntropyLoss()
input = torch.randn(3, 5, requires_grad=True)
target = torch.empty(3, dtype=torch.long).random_(5)
output = loss(input, target)
output.backward()
</code></pre>
<p>This is an example of nn.crossentropy loss from Pytorch docs.<br>
<a href="https://pytorch.org/docs/stable/nn.html#torch.nn.CrossEntropyLoss" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/docs/stable/nn.html#torch.nn.CrossEntropyLoss</a></p>
<p>This is run on torch 0.1.5 version and shows no grad_fn error</p>

username: ptrblck
cooked: <p>I assume you are running this code in PyTorch <code>1.5.0</code> (not <code>0.1.5</code>)?<br>
This code snippet should work and works for me in this version.<br>
Could you post the error message as well as the output of <code>print(torch.__version__)</code>, please?</p>

username: nilansh_rajput1
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/a/5a7ff69402518b7dcc9b05c93681ee1f5b6e6c3d.png" data-download-href="https://discuss.pytorch.org/uploads/default/5a7ff69402518b7dcc9b05c93681ee1f5b6e6c3d" title="torch"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/a/5a7ff69402518b7dcc9b05c93681ee1f5b6e6c3d_2_690x276.png" alt="torch" data-base62-sha1="cUB8cgvWiUiY5JtmFveoObw1eot" width="690" height="276" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/a/5a7ff69402518b7dcc9b05c93681ee1f5b6e6c3d_2_690x276.png, https://discuss.pytorch.org/uploads/default/optimized/3X/5/a/5a7ff69402518b7dcc9b05c93681ee1f5b6e6c3d_2_1035x414.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/5/a/5a7ff69402518b7dcc9b05c93681ee1f5b6e6c3d_2_1380x552.png 2x" data-dominant-color="363637"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">torch</span><span class="informations">1732×693 95.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
I am doing this on colab, when i run this snippet on my system with torch 1.4.0 it runs correctly.</p>

username: ptrblck
cooked: <p>Could you add <code>print(output)</code> before calling <code>output.backward()</code> and post it here?</p>
<p>You can add code snippets by wrapping them into three backticks ```, which makes debugging easier.</p>

username: nilansh_rajput1
cooked: <pre><code class="lang-auto">loss = nn.CrossEntropyLoss()
input = torch.randn(3, 5, requires_grad=True)
target = torch.empty(3, dtype=torch.long).random_(5)
output = loss(input, target)
print(output)
output.backward()

tensor(1.6685)
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-5-835ec6ff9004&gt; in &lt;module&gt;()
      4 output = loss(input, target)
      5 print(output)
----&gt; 6 output.backward()

1 frames
/usr/local/lib/python3.6/dist-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
     98     Variable._execution_engine.run_backward(
     99         tensors, grad_tensors, retain_graph, create_graph,
--&gt; 100         allow_unreachable=True)  # allow_unreachable flag
    101 
    102 

RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
</code></pre>

username: ptrblck
cooked: <p>Sorry, no idea what might be going on.<br>
Could you restart the kernel or are you running in a clean environment?<br>
I would recommend to create a new virtual environment, reinstall PyTorch and rerun the script.</p>

username: nilansh_rajput1
cooked: <p>I am running on google colab, I only installed Transformers nothing else, maybe you can try it on google colab, same thing happening on kaggle notebooks also</p>

username: nilansh_rajput1
cooked: <p>Hey sorry I got it, in my code there was <code>torch.set_grad_enabled(False);</code> this line, I was struggling with my custom loss function for a day for this error:( . I tried this example to see if same error comes up.</p>

username: ptrblck
cooked: <p>Oh OK, so the gradient calculation was disabled globally. I’m glad you figured it out. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>