Why can't I accumulate gradients?

username: bongbonglemon
cooked: <p>Following is snippet from DCGAN pytorch tutorial</p>
<pre><code class="lang-auto">        netD.zero_grad()
        real_cpu = data[0].to(device)
        b_size = real_cpu.size(0)
        label = torch.full((b_size,), real_label, dtype=torch.float, device=device)
        output = netD(real_cpu).view(-1)
        errD_real = criterion(output, label)
        errD_real.backward() # &lt;------------------------ BACKWARD CALLED ONCE
        D_x = output.mean().item()

        noise = torch.randn(b_size, nz, 1, 1, device=device)
        fake = netG(noise)
        label.fill_(fake_label)
        output = netD(fake.detach()).view(-1)
        errD_fake = criterion(output, label)
        errD_fake.backward() # &lt;------------------------ BACKWARD CALLED AGAIN
        D_G_z1 = output.mean().item()
        errD = errD_real + errD_fake
        optimizerD.step()
</code></pre>
<p>Here backward is called twice with no issue. Gradients are “accumulated” (according to the tutorial) and are used to update the weights with <code>optimizerD.step</code></p>
<p>However when I did the following</p>
<pre><code class="lang-auto">        D_x = netD(x)
        D_T_x = netD(T_x) 
        errD_real = criterion(D_x, label)
        errD_real.backward() # &lt;------------------------ BACKWARD CALLED ONCE
        L_real = l2loss(D_x, D_T_x)
        L_real.backward() # &lt;------------------------ BACKWARD CALLED AGAIN
</code></pre>
<p>I get this error</p>
<pre><code class="lang-auto">RuntimeError: Trying to backward through the graph a second time (or directly access saved variables after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved variables after calling backward.
</code></pre>
<p>And I don’t see why the former is okay but the latter isn’t when they should be concerning the same graph for the netD parameters.</p>
<p>Please help me clear my misunderstanding. Thank you so much.</p>
<p>Best,<br>
Gordon</p>

username: tom
cooked: <p>The key bit is that you have the <code>Dx = netD(x)</code> evaluation as a grad-requiring bit that would be backwarded through. Note how the the example you cite uses <code>output = netD(fake.detach()).view(-1)</code> before the second backward. It detaches fake, so the backward doesn’t go through the <code>netG</code> computation and then computes the <code>netD</code> evaluation which has not been used in the first backward (which has its own <code>netD</code> evaluation).</p>
<p>If you don’t have much computation between the two backwards (if l2loss is what it sounds like) and want to accumulate grads for both losses in the <code>netD</code> parameters, maybe doing the backward through the sum of the two losses is a good option. (It should be more efficient than using <code>retain_graph=True</code> in the first backward, although that would work, too.)</p>
<p>Best regards</p>
<p>Thomas</p>

username: bongbonglemon
cooked: <p>That makes perfect sense especially since the pseducode did in fact used an addition of losses.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/3/d3cedda58eb1beecfd0adf652e02d98655e6bd99.png" alt="Screenshot 2021-08-17 at 3.32.41 PM" data-base62-sha1="udJX9ZIq30T3f6B4GVhUU3v4TCp" width="296" height="24"></p>
<p>Much thanks,<br>
Gordon</p>

username: bongbonglemon
cooked: <p>I’m afraid my understanding isn’t complete yet.</p>
<pre><code class="lang-auto">        D_x = netD(x)
        
        # bCR: Forward pass augmented real batch through D
        D_T_x = netD(T_x) 
        
        errD_real = criterion(D_x, label)

        # bCR: Calculate L_real: |D(x) − D(T(x))|^2 
        L_real = l2loss(D_x, D_T_x)
        
        (errD_real + L_real).backward() # &lt;--- backward called once for first netD evaluation
        
        # Format for print
        D_x = D_x.mean().item()
        
        # train with fake
        z = torch.randn(batch_size, nz, 1, 1, device=device)
        G_z = netG(z)
        
        # bCR: Augment generated images
        T_G_z = transform(G_z.detach())
        
        label.fill_(fake_label)
        D_G_z = netD(G_z.detach())
        
        # bCR: Forward pass augmented fake batch through D
        D_T_G_z = netD(T_G_z)
        
        errD_fake = criterion(D_G_z, label)
        
        # bCR: Calculate L_fake: |D(G(z)) − D(T(G(z)))|^2
        L_fake = l2loss(D_G_z, D_T_x)
        
        (errD_fake + L_fake).backward() &lt;--- backward called once for second evaluation of netD
</code></pre>
<p>The first backward() works but the second one gives me the same error as above. Perhaps my understanding of ‘evaluation of netD’ is wrong. Do you mean evaluation of netD as in netD(x) being one evaluation and netD(y) being another?</p>

username: gphilip
cooked: <p>Try recomputing <code>D_T_x</code> just before computing <code>L_fake</code> and see if this fixes this error.</p>

username: bongbonglemon
cooked: <p>Dang it does! Thanks! You also made me realise that I was using the wrong target anyways. Like its literally in the comment above to use <code>D_T_G_z</code> <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=10" title=":sweat_smile:" class="emoji" alt=":sweat_smile:"> But I gotta ask…are the weights also backprogagated for the target in a loss function? Because the issue seems to be that the gradients were backprogated twice for the same evaluation for D_T_x. If you could help me clear this up, I would deeply appreciate it.</p>

username: gphilip
cooked: <p>Glad to be of help. <a href="https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html" rel="noopener nofollow ugc">This tutorial</a> may help with some of your questions.</p>