Multiple loss gradients

vishal_ib: <p>Hi, I’m working on implementing the Pareto efficient fairness algorithm for fairness mitigation that involves a composite loss function as given below:</p>
<p>loss1 = cross-entropy loss<br>
loss2 = lambda * (alpha * L1-norm(<strong>error</strong>) + (1-alpha)* variance(<strong>error</strong>))</p>
<p>loss = loss1 + loss2</p>
<p>where, <strong>error</strong> is a vector of subgroup metrics(accuracies).</p>
<p>Is loss2 differentiable? Also, how do I get the gradients of loss1 and loss 2 w.r.t model parameters separately?</p>

ptrblck: <p>If depends how <code>error</code> is calculated and in particular if the operations are differentiable.<br>
E.g. if you’ve calculated the subgroup metrics by using <code>torch.argmax</code> it won’t be differentiable and you would need to define the gradients manually e.g. via a custom backward operation.</p>

vishal_ib: <p>Hi, Thanks for the reply!</p>
<p>So, the <strong>error</strong> is defined as,</p>
<p>ce_n = cross-entropy loss of subgroup n<br>
f1 = torch.tensor([1/ce_1, 1/ce_2, 1/ce_3])<br>
f2 = torch.tensor([1/ce_4, 1/ce_5, 1/ce_6])</p>
<p><strong>error</strong> = 1 - f1/f2</p>
<p>The gradient of loss2 always seems to be zero. Could you please help me figure out why?</p>

ptrblck: <p>Rewrapping tensors into a new tensor will detach them from the computation graph, so you would need to use <code>torch.stack</code> or <code>torch.cat</code> instead to create <code>f1</code> and <code>f2</code>.</p>

vishal_ib: <p>Thank you. I made some changes. Now, the following issue occurs:</p>
<p>I need to update the values in f1 for every batch based on the model’s subgroup performance.<br>
When I tried using the indices like f1[i] = new value, I got the following error:</p>
<p><code>RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [64, 1]], which is output 0 of TBackward, is at version 3; expected version 2 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!</code></p>
<p>How can I update the value of f1 for every batch while training?</p>

ptrblck: <p>I’m not familiar with your use case, but it seems you are currently replacing a computed activation value, which is needed to calculate the gradients, which will yield the error.<br>
Where is <code>new_value</code> coming from and would it be possible to change the computation of <code>f1</code> so that the value is computed directly?</p>

vishal_ib: <p>Sorry, I wasn’t clear.</p>
<p><code>f1</code> is the inverse of the cross-entropy losses and it needs to be calculated after every batch is forward propagated. So, <code>new_value</code> basically denotes the cross-entropy losses for that batch. I am trying to update the values in <code>f1</code> and use it to calculate <code>error</code> as mentioned above, which will be used in calculating the final loss. After updating <code>f1</code>, calling <code>loss.backward()</code> is resulting in the above error. How can I update <code>f1</code> in every iteration?</p>

ptrblck: <p>You won’t be able to assign values to tensors, if they are needed to compute the gradient.<br>
In your case it seems that <code>f1</code> is indeed needed in the <code>backward</code> pass to compute some gradients, so that manipulating it inplace is disallowed.<br>
This short example demonstrates the issue:</p>
<pre><code class="lang-python">w1 = nn.Parameter(torch.randn(1))
w2 = nn.Parameter(torch.randn(1))
x = torch.randn(1)

y = w1 * x
out = y * w2
#y[0] = 1.
out.mean().backward()

print(w2.grad, y)
</code></pre>
<p>Here you can see, that <code>y</code> is needed to compute the gradient for <code>w2</code>.<br>
If you try to manipulate it after its usage (uncomment the assignment), an error will be raised:</p>
<pre><code class="lang-python">RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [1]], which is output 0 of MulBackward0, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
</code></pre>

vishal_ib: <p>Thanks for the example. So from my understanding, there is no way to manipulate the tensors that are required in the <code>backward</code> pass? And also creating a new tensor would detach it from the computation graph?</p>
<p>Is this also the case if I set <code>retain_graph = True</code>?</p>

ptrblck: <p><code>retain_graph</code> wouldn’t change the issue, since the calculation itself is failing, because needed values are gone.<br>
If you know how the gradient should be calculated after the manipulation, you could try to implement a custom <code>autograd.Function</code> as described <a href="https://pytorch.org/tutorials/beginner/examples_autograd/two_layer_net_custom_function.html">here</a>.<br>
In any case, I’m still unsure <em>why</em> you want to manipulate the values.</p>

vishal_ib: <p>I will check that implementation.</p>
<p>I want a tensor with inverse cross-entropy losses. These loss values change for every forward pass. I’m not sure how to update the tensor values without manipulating the existing values.</p>

ptrblck: <p>You are already calculating the “inverse” ce losses via: <code>[1/ce_1, 1/ce_2, 1/ce_3]</code>, so instead of rewrapping them in a tensor, you could use <code>torch.stack([1/ce_1, 1/ce_2, 1/ce_3])</code>, which would not detach these values.</p>