Trying to backward a second time (ESRGAN)

Valentin_Fontanger: <p>Hi,<br>
I have a model for super resolution (ESRGAN) and I have the following error :</p>
<p><strong>&gt; Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.</strong></p>
<p>Here is my training loop :</p>
<pre><code class="lang-auto">    for i, sample in enumerate(train_loader): 
        
        hr = sample["hr"].to(config.device) 
        lr = sample["lr"].to(config.device) 

        # generated super resolution 
        sr = generator(lr)

        ###### train discriminator ######


        for param in discriminator.parameters():
            param.requires_grad = True

        discriminator.zero_grad()

        true_label = torch.full(size=(sr.shape[0], 1), fill_value=1.0, device=config.device)
        fake_label = torch.full(size=(sr.shape[0], 1), fill_value=1.0, device=config.device)

        predicted_true = discriminator(hr)
        predicted_fake = discriminator(sr.detach())

        d_loss_true = adversarial_criterion(torch.sigmoid(predicted_true - predicted_fake.mean(dim=0)), true_label)
        d_loss_true.backward()
        d_loss_fake = adversarial_criterion(torch.sigmoid(predicted_fake - predicted_true.mean(dim=0)), fake_label)
        d_loss_fake.backward()
        # optimization 
        
        d_loss = d_loss_fake + d_loss_true
        d_optim.step()


        ###### train generator ######
        for param in discriminator.parameters():
            param.requires_grad = False
        
        generator.zero_grad()

        d_out_generated = discriminator(sr)
        # mse/vgg loss
        vgg_loss = vgg_criterion(sr, hr)
        vgg_loss.backward()
        # tricking the discriminator
        adversarial_loss = config.adversarial_coefficient * adversarial_criterion(d_out_generated, true_label) 
        adversarial_loss.backward()
        # l1 criterion
        l1_loss = config.l1_coefficient * l1_criterion(sr, hr)
        l1_loss.backward()
        # relativistic loss
        relativistic_loss = config.relativistic_coefficient * adversarial_criterion(torch.sigmoid(d_out_generated - predicted_true.mean(dim=0)), true_label)
        relativistic_loss.backward()

        # complete loss
        g_loss = vgg_loss + adversarial_loss + l1_loss + relativistic_loss
        # optimization 
        g_optim.step()

        # writing with tensorboard
        writer.add_scalar(f"{config.train_mode}/D_LOSS", d_loss, epoch*len(train_loader) + i + 1)
        writer.add_scalar(f"{config.train_mode}/G_LOSS", g_loss, epoch*len(train_loader) + i + 1)
        writer.add_scalar(f"{config.train_mode}/l1_loss", l1_loss, epoch*len(train_loader) + i + 1)
        writer.add_scalar(f"{config.train_mode}/vgg_loss", vgg_loss, epoch*len(train_loader) + i + 1)
        writer.add_scalar(f"{config.train_mode}/adversarial_loss", adversarial_loss, epoch*len(train_loader) + i + 1)
        writer.add_scalar(f"{config.train_mode}/relativistic_loss", relativistic_loss, epoch*len(train_loader) + i + 1)

        if i % 200 == 0 and i != 0: 
            print(f"EPOCH={epoch} [{i}/{len(train_loader)}]D_LOSS in {config.train_mode} mode : {d_loss} ")  
            print(f"EPOCH={epoch} [{i}/{len(train_loader)}]G_LOSS in {config.train_mode} : {g_loss} ")  

</code></pre>
<p>Do you see where the error is ?</p>
<p>Thank you ! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

ptrblck: <p>Based on your code it seems you are trying to calculate the gradients in the <code>generator</code> multiple times via:</p>
<pre><code class="lang-python">vgg_loss = vgg_criterion(sr, hr)
vgg_loss.backward()
...
adversarial_loss = config.adversarial_coefficient * adversarial_criterion(d_out_generated, true_label) 
adversarial_loss.backward()
...
l1_loss = config.l1_coefficient * l1_criterion(sr, hr)
l1_loss.backward()
...
relativistic_loss = config.relativistic_coefficient * adversarial_criterion(torch.sigmoid(d_out_generated - predicted_true.mean(dim=0)), true_label)
relativistic_loss.backward()
</code></pre>
<p>If I’m not mistaken, these losses were created by some output of the <code>generator</code> and would thus try to calculate the gradients of its parameters w.r.t. the current loss.<br>
If so, then note that each <code>backward()</code> operation would delete the intermediate forward activations in the <code>generator</code> (which are needed for the gradient computation) if you are not using <code>retain_graph=True</code>. Use either this argument or add the losses to a final loss and call <code>backward</code> once.</p>

Valentin_Fontanger: <p>Hi,</p>
<p>Thank you for your answer.<br>
As mentioned by you, I added all losses together but still got the error.<br>
On the other hand, the error goes away when I remove the relativistic_loss to the sum.<br>
Do you have any idea ?</p>
<p>Thank you once again</p>

ptrblck: <p>Maybe call <code>.detach()</code> on <code>predicted_true</code> as I assume you don’t want to use it to calculate the gradients in the <code>discriminator</code> again.</p>

Valentin_Fontanger: <p>Incredible, I just did that and went back to the forum to post the solution.<br>
Thank you very much ! this whole detach() thing is confusing to me, I need to pay more attention <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"><br>
Thank you !</p>