Iterative Loss: Loss from first iteration when added in the second iteration does not work

mbsyed: <p>Hi,<br>
I am working on implementing an iterative loss for a project I am working on.<br>
Basically what I am trying to do is the following.</p>
<pre><code class="lang-auto">model = toy_net().to(device)
opt = optim.Adam(model.parameters()) #optimizer
epochs = 50                          #epochs
k = 3                                #iterations
loss_fn = nn.BCELoss()

for epoch in range(epochs):
    for inputs, labels in dataloader:
        losses_iter = [0]*3
        x,y = inputs.to(device), labels.to(device)
        for i in range(k):
            Z = ((i+1)*(i+2))/2
            out = model(x)            #get model output
            loss = loss_fn(out,y)   #get loss
            losses_iter[i] = (loss)*(i+1)
            total_loss = sum((loss_iter)/Z) #total loss is summing losses
                                                   #and scaling by Z            
            total_loss.backward()
            opt.step()
            losses_iter[i] = (loss.detach())*(i+1)
</code></pre>
<p>The Problem I am having right now is that once I detach the loss from the first iteration and add it in the second iteration it is just a constant and is not taken into account for the second iteration since the derivative is of the first loss which is a constant now is 0.<br>
Any help would be appreciated</p>

ptrblck: <aside class="quote no-group" data-username="mbsyed" data-post="1" data-topic="118981">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/9dc877/48.png" class="avatar"> mbsyed:</div>
<blockquote>
<p>The Problem I am having right now is that once I detach the loss from the first iteration and add it in the second iteration it is just a constant and is not taken into account for the second iteration since the derivative is of the first loss which is a constant now is 0.</p>
</blockquote>
</aside>
<p>That is the expected behavior of detached tensors. If you want to add the loss into account in the second iteration, you would have to keep it attached to the computation graph and might need to set <code>retain_graph=True</code> in the <code>backward</code> operation.</p>

mbsyed: <p>Thank you for the response. That works but only if I do not update the weights using <code>opt.step()</code>. In my use case I need to update the weights before I run the next iteration.<br>
Basically, I am working on a segmentation problem. My input are images with 4 channels (RGB and a fourth channel which is initialized to zeros). After the first iteration the 4th channel (zeros) is replaced with outputs from the updated model in the first iteration. Then this new input is fed into the same model again and we calculate the loss for the current iteration and add the loss from the last iteration. It would essentially look something like the following.</p>
<pre><code class="lang-auto">model = Net()
x         = (samples,4,height,width)
y         = (samples,1,height,width)
opt      = Adam(model.parameters())

#-------- iteration 1 -----------
opt.zero_grad()
outputs1 = model(x)
loss1    = loss_fn(outputs1,y)
loss = (loss1)*1
loss.backward(retain_graph=True)
opt.step()
x[samples,3,:,:] = model(inputs).detach()          #inputs updated for iteration 2

#-------- iteration 2 -----------
opt.zero_grad()
outputs2 = model(x)
loss2      = loss_fn(outputs2,y)
loss        = (2/3)*(loss2) + (1/3)*(loss1)           #loss weighted from previous iter and current iter
loss.backward(retain_graph=True)                      #This gives an error:  one of the variables needed for 
                                                       gradient computation has been modified by an inplace operation. 
                                                       #If I don't do opt.step() then there is no error
opt.step()
x[samples,3,:,:] = model(inputs).detach()          #inputs updated for iteration 3

#-------- iteration 3 -----------
opt.zero_grad()
outputs3 = model(x)
loss3      = loss_fn(outputs3,y)
loss        = (1/2)*(loss3) + (1/3)*(loss2) + (1/6)*(loss1)           #loss weighted from previous iters and current iter
loss.backward()             
opt.step()

</code></pre>
<p>I think the main issue I am having now is that I need to update the model to update my input for the next iteration using the outputs from the updated model. But when I update the model and try to do a backprop for the second iteration whose loss is the added loss of iteration 1 and iteration 2 even with <code>retain_graph=True</code> I get an error. I am not sure how to get around this and what is causing this issue.</p>

ptrblck: <aside class="quote no-group" data-username="mbsyed" data-post="3" data-topic="118981">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/9dc877/48.png" class="avatar"> mbsyed:</div>
<blockquote>
<p>But when I update the model and try to do a backprop for the second iteration whose loss is the added loss of iteration 1 and iteration 2 even with <code>retain_graph=True</code> I get an error. I am not sure how to get around this and what is causing this issue.</p>
</blockquote>
</aside>
<p>I guess you are seeing an error, since you are trying to calculate the gradients (in iter2) from stale forward activations (calculated in iter1).<br>
Have a look at <a href="https://discuss.pytorch.org/t/struggling-with-a-runtime-error-related-to-in-place-operations/102222/6">this post</a> for more information and check, if you are hitting the same error.</p>

mbsyed: <p>It sort of fits the error I am getting the only difference being I am calculating a different output for iteration 2 and a different loss function but the <code>backward</code> is done on the added loss where <code>loss=loss_iter1+loss_iter2</code>. Is there a way I can fix this?</p>

ptrblck: <p>This would explain the error, since <code>loss_iter1</code> would still reference the old computation graph.</p>
<aside class="quote no-group" data-username="mbsyed" data-post="5" data-topic="118981">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/9dc877/48.png" class="avatar"> mbsyed:</div>
<blockquote>
<p>Is there a way I can fix this?</p>
</blockquote>
</aside>
<p>It depends on your actual use case, since you are using stale forward activations to calculate the parameter updates, which is wrong.<br>
From the linked issue:</p>
<ul>
<li><code>loss_iter1</code> was calculated using the first forward pass and the model with <code>parameter_set_0</code></li>
<li>this forward pass also calculated all intermediate forward activations (<code>fwd_set_0</code>) and stored them, which are needed to compute the gradients</li>
<li>you are updating the model to <code>parameter_set_1</code>, all forward activations are now stale, since they were <em>not</em> calculated by <code>parameter_set_1</code></li>
<li><code>loss_iter1.backward()</code> tries to compute the gradients using <code>fwd_set_0</code> and <code>parameter_set_1</code>, which is wrong and fails</li>
</ul>
<p>You could either delay the <code>optimizer.step()</code> call or recompute the forward activations depending on your use case.</p>