Multi-level loss optimization

mhamdan: <p>Hi,<br>
I am trying to optimize an MLP using multi-losses that are computed at different times for the same neural network. Basically, the problem is a combination of supervised and semi-supervised learning.</p>
<p>I wish to optimize the algorithm using SGD with labels for each training example, and then would like to also optimize the network using SGD with labels for a set of training examples (e.g. sum of predictions over a certain period of time will have a label)</p>
<pre><code class="lang-auto">dataset = {clip_label: frame1_label:frame1,frame2_label: frame2, frame3_label:frame3}
# unrolled loop
optimizer.zero_grad()
output = net(frame1)
loss = criterion(output, frame1_label)
loss.backward()
 optimizer.step()
summed_output += torch.sum(output)  # sum predicted output

optimizer.zero_grad()
output = net(frame2)
loss = criterion(output, frame2_label)
loss.backward()
 optimizer.step()
summed_output += torch.sum(output)  # sum predicted output

optimizer.zero_grad()
output = net(frame3)
loss = criterion(output, frame3_label)
loss.backward()
 optimizer.step()
summed_output += torch.sum(output)  # sum predicted output



# Optimize at the clip_level 
optimizer.zero_grad() 
loss = criterion(summed_output , clip_label)
loss.backward()
optimizer.step()
</code></pre>
<p>This is how the code should look like. Zeroing the gradients might be problematic for the clip_level optimization. When I run my code, I get this error:</p>
<pre><code class="lang-auto">RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [8, 1]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

</code></pre>
<p>If I detach summed_output from the graph, then this error disappears, but I do not think the training process is correct.</p>
<p>Any idea how to accomplish this process correctly?</p>
<p>Thanks</p>

albanD: <p>Hi,</p>
<p>I think you have two issues here:</p>
<ul>
<li>The error you see happens because when you do <code>optimizer.step()</code> in the first iterations, you modify the weights of <code>net</code> inplace. But the final backward would actually need the original values to be able to backprop though the original net. Hence the error you see.</li>
<li>The final backward will backprop in the same net as the first ones so you need the first ones to set <code>retain_graph=True</code> so that the last backward won’t fail saying that buffers have been freed.</li>
</ul>
<p>The question I would have is: do you want to compute the gradients for the final loss based on the original parameter values or the updated ones?<br>
If you want the original, then you will have to use different version of the net so that you can keep the original values around to be able to backprop to them.<br>
If you want to get wrt the updated ones, you should re-do the forward based on the final parameter values to compute the summed output at the end.</p>

mhamdan: <p>Hi,<br>
Thank you for your reply <a class="mention" href="/u/alband">@albanD</a>.</p>
<p>I am currently using retain_graph and compute the gradient wrt the last updated weights, however, the issue is that summed_output needs to be detached from the original graph since I already made several optimization steps and zeroed the gradients at each step. what is happening now, is that I detach it, then compute output and add it summed_output so that when I do backward, it is done wrt to the last forward pass. This however, is not what I want and I do not think it’s valid.  Per the second option, is this how you considered doing it?</p>
<p>I would like to calculate gradients based off the original parameters (from every iteration). As you noted that I have to use a different version of the net, how can I do that?</p>

albanD: <aside class="quote no-group" data-username="mhamdan" data-post="3" data-topic="104907">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mhamdan/48/29936_2.png" class="avatar"> mhamdan:</div>
<blockquote>
<p>Per the second option, is this how you considered doing it?</p>
</blockquote>
</aside>
<p>What I had in mind was more to add before the end something like:</p>
<pre><code class="lang-auto">
# Optimize at the clip_level 
summed_output = 0.
summed_output += net(frame1).sum()
summed_output += net(frame2).sum()
summed_output += net(frame3).sum()
optimizer.zero_grad() 
loss = criterion(summed_output , clip_label)
loss.backward()
optimizer.step()

</code></pre>

mhamdan: <p>you’re suggesting that I append all frames of a clip from each iteration then compute them in a single batch for the clip level optimization.  At the same time, I’d be doing per-frame optimization for each iteration.</p>
<p>I think this would work, but it would be slow since we’re doing a forward pass twice for the same frames.</p>

mhamdan: <p>Thanks <a class="mention" href="/u/alband">@albanD</a> for the suggestion. Can you elaborate though on the other method (original parameters)</p>

albanD: <p>Yes you will do the forward twice, but you don’t really have a choice since you want gradients for different versions of your parameters.</p>
<p>The only other way around it I can think about (in either method) is to save the gradients and do all the updates at the end. But even then, you will use gradients computed for different weight values to update the new ones which doesn’t really make sense…<br>
You have to redo the forward with the new weights if you want to have the right gradients.</p>