RuntimeError: element 0 of variables does not require grad and does not have a grad_fn

Zhang_Wen: <p>hi, i have a problem here, i got a sequence of Variables which are the outputs of the bi-directional RNN, and i stacked them into a matrix <code>xs_h</code> whose dimension is <code>(seq_length, batch_size, hidden_size)</code>, them i want to update the matrix <code>xs_h</code> by convoluting on two slices in <code>xs_h</code>, some codes are as follows:</p>
<p><code>new_xs_h = xs_h.clone()</code><br>
<code>vp, vc = xs_h[idx_0, bidx], xs_h[idx_1, bidx]</code><br>
<code>x = tc.stack([self.f1(vp), self.f2(vc)], dim=1)[None, :, :]</code><br>
<code>new_xs_h[idx_1, bidx] = self.tanh(self.l_f2(self.conv(x).squeeze()))</code></p>
<p>actually, i want to update the Variable <code>xs_h</code> and then let the new updated matrix <code>new_xs_h</code> get into my computation graph again. However, i got following errors when i call <code>backward()</code> after the running of above code:</p>
<p><code>RuntimeError: element 0 of variables does not require grad and does not have a grad_fn</code></p>
<p>i do not kown why, any reply will be appreciated.<br>
thanks.</p>

richard: <p>It sounds like the problem is that your <code>xs_h</code> don’t have <code>requires_grad=True</code>. Have you tried creating Variables with <code>requires_grad=True</code>?</p>

Zhang_Wen: <p>thanks for reply, the Variable xs_h is not created by myself, it is the output of the Bi-RNN by feeding the embedding of the words. so the requires_grad attribute is False.</p>

richard: <p>Okay. You can make a new Variable with requires_grad = True:</p>
<pre><code class="lang-auto">var_xs_h = Variable(xs_h.data, requires_grad=True)
</code></pre>

Brando_Miranda: <p>Did the suggestion solve your problem? I have the same error thrown at me but the error isn’t very helpful since I don’t know what of my code has the <code>require_gradients</code> set to false. I went ahead and set everything I could find to trainable but it still didn’t fix it…</p>
<pre><code>RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn</code></pre>

gt_tugsuu: <p>For me  ‘loss = Variable(loss, requires_grad = True)’ worked.</p>
<p>I was trying to use float type of loss, and it was giving me the same error.</p>

mnazaal: <p>Got the same error in a very different problem and gt_tugsuu 's post helped. thanks</p>

tomorrow: <p>Thank you for your answer</p>

sonal_garg: <p>Hi richard, I was also getting same error using this it got fixed but now loss is not decreasing with every epochs . It is constant. I think this is related to this variable.</p>

Mylinda: <p>Could you give the link of gt_tugsuu’s post? I meet the problem of getting the same output and loss. Thanks</p>

Mylinda: <p>Seems my problem is not caused by the same error, so I do not need the post. Thanks!</p>

yGalindo: <p><a class="mention" href="/u/sonal_garg">@sonal_garg</a> and other people that may be tempted to use loss.requires_grad=True or some similar fix, note that if you are even able to set the requires_grad of your loss, something is very wrong.</p>
<p>You can only set requires_grad of leaf nodes of the computation graph, that is, tensors that do not further propagate the gradient (see <a href="https://towardsdatascience.com/pytorch-autograd-understanding-the-heart-of-pytorchs-magic-2686cd94ec95" rel="nofollow noopener">this explanation</a>). If your loss does not propagate the gradient to the rest of your model, you are not training the model, and the loss is useless (even if you force it to require a gradient).</p>
<p>Chances are, some layer is set as requires_grad=False somewhere in the code (probably all of them or the last layer), or the computation graph is detached. Possibly the backward call is wrapped in a with torch.no_grad().</p>

RylanSchaeffer: <p>I have the same error for different code. <a class="mention" href="/u/richard">@richard</a>  <a class="mention" href="/u/ygalindo">@yGalindo</a> , how does one generally locate the source of the error? I’ve already checked that my loss has <code>requires_grad=True</code> and that the model’s input to the loss function has <code>requires_grad=True</code>.</p>

RylanSchaeffer: <p>I found my problem. For anyone else after me, I had a loop making multiple passes through the model and adding each pass’s loss to a running total. But I initialized the total loss to 0 outside the loop and that tensor had <code>requires_grad=False</code>, a property that was preserved as each pass’s loss was added to the total loss. I solved this using <code>total_reward = torch.zeros(1, requires_grad=True)</code>.</p>

angelinaG: <p>Thank you for sharing this!<br>
My problem was resolved right away!<br>
loss.requires_grad = True  (edited the typo)</p>

Vijit_Mehrotra: <p>Hi Everyone i am getting RuntimeError: element 11 of tensors does not require grad and does not have a grad_fn. when I am running a GAN Architecture for getting the Gaussian Model and I am using  WGANLOSS as well.</p>
<p>How to fix the error so that i can get th required output.</p>

ptrblck: <p>The error is usually thrown, if you detach a tensor from the computation graph, e.g. by:</p>
<ul>
<li>calling <code>tensor.detach()</code> on it directly</li>
<li>transforming it to a numpy array or a Python literal</li>
<li>recreating a tensor via <code>torch.tensor(tensor)</code>
</li>
</ul>
<p>Could you check your code for any of these ops?</p>

Vijit_Mehrotra: <p>Actauly I have not callled torch.detach() nor I have trasnformed any tensor to numpy or python literal.</p>
<p>It is like when i am trying to do Higher order differencation of tensors it is giving me error on them.<br>
For me : HVP_Y = torch.autograd.grad(inputs =y,outputs =DerivateX_g ,grad_outputs=B_X, retain_graph =True, allow_unused = True) is the place where i am getting the error at in my code.</p>

ptrblck: <p>Could you print the tensors <code>y</code> and <code>DerivateX_g</code> please?</p>

Vijit_Mehrotra: <p>I have printed both the Tensor y and DerivateX_g and put that in image for easy purpose.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/6/c617ec8933d2dba28390b1c2f707f467bba9fec3.png" data-download-href="https://discuss.pytorch.org/uploads/default/c617ec8933d2dba28390b1c2f707f467bba9fec3" title="Pytorch_Error"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/6/c617ec8933d2dba28390b1c2f707f467bba9fec3_2_70x500.png" alt="Pytorch_Error" data-base62-sha1="sgpNlSGTPJTa58hQBhE2ct3I2Dp" width="70" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/6/c617ec8933d2dba28390b1c2f707f467bba9fec3_2_70x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/6/c617ec8933d2dba28390b1c2f707f467bba9fec3_2_105x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/6/c617ec8933d2dba28390b1c2f707f467bba9fec3_2_140x1000.png 2x" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Pytorch_Error</span><span class="informations">1550×10983 1.42 MB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>