Custom Loss Function - Error: element 0 of tensors does not require grad and does not have grad_fn

username: ben_mi
cooked: <p>Hi everyone! I’m trying to implement the global pair loss function from “Recognition of Action Units in the Wild with Deep Nets and a New Global-Local Loss”.</p>
<p>I have to look at all possible pairs of the predictions. If the values in a pair are the same, then g_predictions = 1. If they are different, then g_predictions = 0.</p>
<p>I do the same thing for the targets.</p>
<p>Then I need to calculate the loss, by adding them all together.</p>
<p>total += (g_prediction - g_target)**2</p>
<p>This is my implementation:</p>
<pre><code class="lang-auto">def custom_loss(preds, targs):
    total = 0
    for pair_p, pair_t in zip(torch.combinations(preds, r=2), torch.combinations(targs, r=2)):
        g1 = torch.eq(pair_p[0], pair_p[1]).type(torch.uint8)
        g2 = torch.eq(pair_t[0], pair_t[1]).type(torch.uint8)
        total += (g2 - g1)**2
    return total

output = model(x)
loss = custom_loss(output, targets)
loss.backward()
</code></pre>
<p>I’m getting this error after backward() is called:</p>
<pre><code class="lang-auto">RuntimeError: element 0 of tensors does not require grad and does not have grad_fn
</code></pre>
<p>Am I retaining the graph properly? Does anyone know how to fix this?</p>
<p>Thanks for the help</p>

username: chetan_patil
cooked: <p>Hi, try this.</p>
<pre><code class="lang-auto">output = model(x)
loss = custom_loss(output, targets)
loss.requires_grad= True
loss.backward()
</code></pre>

username: ben_mi
cooked: <p>When I do that I no longer get the error, however when I…</p>
<pre><code class="lang-auto">print(model.weight.grad)
</code></pre>
<p>… it still prints None, so it doesn’t seem to be the solution.<br>
I think the way I’m adding or computing the g1 and g2 value might still not be correct.</p>

username: chetan_patil
cooked: <p>That is the correct way.<br>
Try this,<br>
<code>[x.grad.data for x in model.parameters()]</code> after <code>loss.backward()</code>.<br>
If the gradients are calculated, there won’t be any <code>None</code>s in the list.</p>

username: ben_mi
cooked: <p>Thanks for your help. When I try that I still get an error:</p>
<pre><code class="lang-auto">AttributeError: 'NoneType' object has no attribute 'data'
</code></pre>
<p>When I try</p>
<pre><code class="lang-auto">[x.grad for x in model parameters()]
</code></pre>
<p>I get a list of Nones. <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=9" title=":sweat_smile:" class="emoji" alt=":sweat_smile:"></p>

username: ben_mi
cooked: <p>I now changed the loss function to this, so I only use torch functions:</p>
<pre><code class="lang-auto">def custom_loss(p, t):
    total = torch.zeros(1, dtype=torch.float, requires_grad=True)
    for pair_p, pair_t in zip(torch.combinations(p, r=2), torch.combinations(t, r=2)):
        g1 = torch.eq(torch.gather(pair_p, 0, torch.tensor([0])), torch.gather(pair_p, 0, torch.tensor([1]))).type(torch.uint8)
        g2 = torch.eq(torch.gather(pair_t, 0, torch.tensor([0])), torch.gather(pair_t, 0, torch.tensor([1]))).type(torch.uint8)
        total += (g2 - g1)**2
    return total

model = nn.Linear(3, 5)
output = model(x)

loss = custom_loss(output, t)
loss.backward()
print('loss = ', loss)
</code></pre>
<p>And the loss printed now prints this:<code>loss =  tensor([10.], device='cuda:0', dtype=torch.float32, grad_fn=&lt;AddBackward0&gt;)</code></p>
<p>But when I check the gradients it is still None. Do you know why this?</p>
<pre><code class="lang-auto">print(model.weight.grad)
# None
print([x.grad for x in model.parameters()])
# [None, None]
</code></pre>

username: chetan_patil
cooked: <p>Can you provide  a sample of <code>x</code> and <code>t</code> ?</p>

username: ben_mi
cooked: <p>Yes sure. I just used random tensors for testing, like these:</p>
<pre><code class="lang-auto">t = torch.tensor([1, 1, 1, 1, 1])
x = torch.tensor([2, 3, 3], dtype=torch.double)
</code></pre>

username: chetan_patil
cooked: <p>Hi, sorry, unfortunately, I cant’ seem to debug this.<br>
<a class="mention" href="/u/ptrblck">@ptrblck</a>, could you have a look at this ?</p>

username: ben_mi
cooked: <p>No worries, thanks for helping so far. : )</p>

username: ptrblck
cooked: <p><code>torch.eq</code> will break the computation graph and the output will not have a valid <code>grad_fn</code> associated with it as seen here:</p>
<pre><code class="lang-python">x = torch.randn(1, requires_grad=True)
y = torch.randn(1)
z = torch.eq(x, y)
print(z)
&gt; tensor([False])
</code></pre>
<p>Are you sure you need to calculate <code>total</code> using <code>g1</code> and <code>g2</code> or could use use <code>g1</code> and <code>g2</code> to somehow “index” <code>preds</code> and <code>targs</code> and calculate the paired loss function?</p>

username: ben_mi
cooked: <p>Thank you for your response. I now changed the function to this and used indexing on <code>pair_p</code> and <code>pair_t</code>.</p>
<pre><code class="lang-auto">def custom_loss(p, t):
    total = torch.zeros(1, dtype=torch.float, requires_grad=True)
    for pair_p2, pair_t2 in zip(torch.combinations(p, r=2), torch.combinations(t, r=2)):
        pair_p, _ = torch.sort(pair_p2)
        pair_t, _ = torch.sort(pair_t2)
        pair_p = pair_p.double()
        pair_t = pair_t.double()
        
        g1 = (pair_p[0] == pair_p[1]).type(torch.uint8)
        g2 = (pair_t[0] == pair_t[1]).type(torch.uint8)
        total = total + (torch.max(pair_t[g2.item()], g2.type(torch.double)) - torch.max(pair_p[g1.item()], g1.type(torch.double)))**2
    return total
</code></pre>
<pre><code class="lang-auto">t = torch.tensor([1, 1, 0, 0, 1])
x = torch.tensor([2, 3, 3], dtype=torch.double)

model = nn.Linear(3, 5)
output = model(x)

loss = custom_loss(output, t)
loss.backward()
print('loss = ', loss)
# loss = tensor([4.], device='cuda:0', grad_fn=&lt;AddBackward0&gt;)
</code></pre>
<p>Now I don’t get a list of <code>None</code>s anymore, but instead I get this tensor full of zeros.</p>
<pre><code class="lang-auto">print([x.grad for x in model.parameters()])
#     [tensor([[0., 0., 0,],
#              [0., 0., 0.],
#              [0., 0., 0.],
#              [0., 0., 0.],
#              [0., 0., 0.]], device='cuda:0'), tensor([0., 0., 0., 0., 0.], device='cuda:0')]
</code></pre>
<p>Do you know what might cause this?</p>

username: ptrblck
cooked: <p>Your initialization might be “unlucky”, as I’ve got some valid gradients in a couple of iterations (I also got a full zero gradient output):</p>
<pre><code class="lang-python">tensor([[ 0.0000,  0.0000,  0.0000],
        [ 6.8986, 10.3479, 10.3479],
        [ 0.0000,  0.0000,  0.0000],
        [13.0105, 19.5157, 19.5157],
        [ 8.8299, 13.2449, 13.2449]])
tensor([0.0000, 3.4493, 0.0000, 6.5052, 4.4150])
</code></pre>
<p>Note that, <code>torch.max</code> will return a <code>1</code> gradient for the max value and <code>0</code>s everywhere else.<br>
Based on the input and your calculations you might end up removing the gradient in the last operation.</p>

username: ben_mi
cooked: <p>Thank you so much! That helped a lot.<br>
I might have to try to find a different way other than using <code>torch.max</code> then. I somehow have to get a <code>1</code> value if the pair consists of <code>[0, 0]</code> or <code>[1, 1]</code> and a <code>0</code> value in all other cases, so I’ll have to think about that some more.</p>

username: ptrblck
cooked: <p>You might experiment with a “soft” step function, such as sigmoid or <code>x / (sqrt(x*x + delta)) * 0.5 + 0.5</code>, where a smaller <code>delta</code> approximates the step function. This would yield smoother gradients and might help.</p>
<p>Let me know, if these approaches would work.</p>

username: ben_mi
cooked: <p>In the actual classifier I’m using, I apply a <code>Sigmoid()</code> in the last layer, so that the classifier outputs are between <code>0</code> and <code>1</code>. The custom loss function I’m trying to implement is supposed to be an extra regularization loss function. For the main loss function I use <code>BCEWithLogitsLoss()</code>. For the extra custom regularization loss function I then wanted to take the <code>predicted</code> values and put them in the same format as the <code>targets</code>, so putting all <code>predicted[predicted &lt; 0.5] = 0</code> and all <code>predicted[predicted &gt;= 0.5] = 1</code>. I tested this again, when I don’t apply these changes into my custom loss function I also get gradients, like you did during your iterations. However, when I add this change, I just get <code>0</code> gradients again, no matter how often I run the program. Shouldn’t I at least still get gradients through the values that were changed to <code>1</code>? I’m a bit confused about this.</p>
<pre><code class="lang-auto">def custom_loss(preds, t):
    total = torch.zeros(1, dtype=torch.float, requires_grad=True)

    p = preds.clone()
    p[preds &lt; 0.5] = 0
    p[preds &gt;= 0.5] = 1

    for pair_p2, pair_t2 in zip(torch.combinations(p, r=2), torch.combinations(t, r=2)):
        pair_p, _ = torch.sort(pair_p2)
        pair_t, _ = torch.sort(pair_t2)
        pair_p = pair_p.double()
        pair_t = pair_t.double()
        
        g1 = (pair_p[0] == pair_p[1]).type(torch.uint8)
        g2 = (pair_t[0] == pair_t[1]).type(torch.uint8)
        total = total + (torch.max(pair_t[g2.item()], g2.type(torch.double)) - torch.max(pair_p[g1.item()], g1.type(torch.double)))**2
    return total
</code></pre>