RunTime Error: One of the differentiated Tensors appears to not have been used in the graph

Mohamed_Magdy: <p>Hello,<br>
I’m currently implementing a project for my machine learning course, and i have faced an issue in gradient computation based on additional trainable parameters. I’m trying actually to implement the algorithm<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/a/5aa9a54f39bf9395bd15124f8940df01084daf2b.png" data-download-href="https://discuss.pytorch.org/uploads/default/5aa9a54f39bf9395bd15124f8940df01084daf2b" title="Screen Shot 2021-03-16 at 1.17.57 AM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/a/5aa9a54f39bf9395bd15124f8940df01084daf2b_2_581x500.png" alt="Screen Shot 2021-03-16 at 1.17.57 AM" data-base62-sha1="cW2r4wS0z6AYOP9w4Ky5WUdqUlR" width="581" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/a/5aa9a54f39bf9395bd15124f8940df01084daf2b_2_581x500.png, https://discuss.pytorch.org/uploads/default/original/3X/5/a/5aa9a54f39bf9395bd15124f8940df01084daf2b.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/5/a/5aa9a54f39bf9395bd15124f8940df01084daf2b.png 2x" data-dominant-color="F1F1F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2021-03-16 at 1.17.57 AM</span><span class="informations">816×702 42 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
with the help also of a github project that i’m taking as reference.<br>
I’m receving “RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior” during step 10 “Computation of gradients w.r.t epsilon”<br>
The piece of my code that implements from step 1 to step 10 is as follows</p>
<pre><code>criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(net.parameters(), lr=gamma, momentum=rho)
for epoch in range(T):
    running_loss = 0.0
    for idx, (train_x, train_label) in enumerate(trainloader):
      net.train()
      # initialize a dummy network for the meta learning of the weights
      meta_net = LeNet().to(device)
      meta_net_optimizer = torch.optim.SGD(meta_net.parameters(), lr=gamma, momentum=rho)
      # Step 4 - 5 forward pass to compute the initial weighted loss
      # Forward propagation
      outputs = meta_net(train_x)
      # Error evaluation
      train_label = train_label.squeeze()
      loss = criterion(outputs,train_label.long())
      eps = to_var(torch.zeros(loss.size()))
      l_f_meta = torch.sum(loss * eps)
      meta_net.zero_grad()
      # Line 6 perform a parameter update
      grads = torch.autograd.grad(l_f_meta, meta_net.parameters(), create_graph=True)
      meta_net_optimizer.step()
      # Line 8 - 10 2nd forward pass and getting the gradients with respect to epsilon
      y_g_hat = meta_net(test_data_t)
      l_g_meta = criterion(y_g_hat,test_labels_t.long())  
      grad_eps = torch.autograd.grad(l_g_meta, eps, only_inputs=True) # &gt;&gt; Step 10
</code></pre>
<p>Could you advise please why i’m receiving that? Supposedly, epsilon is a tensor. Issue is in<br>
grad_eps = torch.autograd.grad(l_g_meta, eps, only_inputs=True). My target is to</p>

Dwight_Foster: <p>In torch.zeros have you tried setting requires grad to True like this:</p>
<pre><code class="lang-auto">      eps = to_var(torch.zeros(loss.size(), requires_grad=True))
</code></pre>

Mohamed_Magdy: <p>Same issue. Is there a way to determine the differentiable variables? Because i get that eps cannot be found but no idea why</p>

Dwight_Foster: <p>No because eps is not in the graph the derivative will always be none. I do not know a way to get around this. You could try looking at <a href="https://pytorch.org/docs/stable/notes/extending.html" rel="noopener nofollow ugc">these</a> functions to see if you can use them to calculate the correct gradients but I cannot find a why to add eps to the graph.</p>

Liangbo_Ning: <p>Maybe you will get inspired from here : (<a href="https://github.com/TinfoilHat0/Learning-to-Reweight-Examples-for-Robust-Deep-Learning-with-PyTorch-Higher" class="inline-onebox" rel="noopener nofollow ugc">GitHub - TinfoilHat0/Learning-to-Reweight-Examples-for-Robust-Deep-Learning-with-PyTorch-Higher: An implementation of the paper "Learning to Reweight Examples for Robust Deep Learning" from ICML 2018 with PyTorch and Higher.</a>)<br>
“Higher” package can be used to  compute the gradient in this case.</p>