Why "netD.zero_grad()" is not required before "errD_fake.backward()" in Pytorch DCGAN tutorial code?

jayjay: <p>I have two quick questions about the Pytorch DCGAN tutorial code.</p>
<p>To avoid gradient accumulation, “netD.zero_grad()” looks like required before “errD_fake.backward()”  but not. Why?</p>
<p>In the training of discriminator network, “fake.detach()” is used, which means the fake network is detached from the graph, and gradient calculation is not conducted in “errD_fake.backward()”. Then, why do we have to initialize grad of netG (“netG.zero_grad()”) before backpropagation?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/2/62f5b1d8291941d2e7274eef6ddd992ab044723a.png" data-download-href="https://discuss.pytorch.org/uploads/default/62f5b1d8291941d2e7274eef6ddd992ab044723a" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/2/62f5b1d8291941d2e7274eef6ddd992ab044723a_2_372x500.png" alt="image" data-base62-sha1="e7rbOALbOu8ybucRblip6TPZC54" width="372" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/2/62f5b1d8291941d2e7274eef6ddd992ab044723a_2_372x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/2/62f5b1d8291941d2e7274eef6ddd992ab044723a_2_558x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/2/62f5b1d8291941d2e7274eef6ddd992ab044723a_2_744x1000.png 2x" data-dominant-color="F0F0EC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1358×1823 294 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <aside class="quote no-group" data-username="jayjay" data-post="1" data-topic="172338">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/35a633/48.png" class="avatar"> jayjay:</div>
<blockquote>
<p>To avoid gradient accumulation, “netD.zero_grad()” looks like required before “errD_fake.backward()” but not. Why?</p>
</blockquote>
</aside>
<p>The gradients w.r.t. <code>errD_real</code> and <code>errD_fake</code> are accumulated and <code>optimizerD.step()</code> is called once to update the parameters of the discriminator.<br>
You could also update the discriminator separately but I don’t know if this would negatively affect the training.</p>
<aside class="quote no-group" data-username="jayjay" data-post="1" data-topic="172338">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/35a633/48.png" class="avatar"> jayjay:</div>
<blockquote>
<p>Then, why do we have to initialize grad of netG (“netG.zero_grad()”) before backpropagation?</p>
</blockquote>
</aside>
<p>You would still need to zero out the gradient from the generator update performed in the previous iteration.</p>

jayjay: <p>Thank you for the clear answer.<br>
However, I still get confused with gradient accumulation in training netG.</p>
<p>I attached the simple figure describing my questions here.<br>
Please refer to the figure and let me know what I misunderstand. (my questions are red colored)</p>
<p>It looks as if ‘netD.parameters.grad’ is accumulated while training netG, but the gradient accumulation doesn’t affect autograd result of ‘netG.parameters.grad’ (I checked it by testing toy codes).</p>
<p>I don’t understand why accumulated gradients of netD parameters don’t affect autograd result of netG parameters.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/7/e7791ccc5ca5d1aeb424c71c85c4c4fed49a0e7e.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/e7791ccc5ca5d1aeb424c71c85c4c4fed49a0e7e" title="Screenshot 2023-02-12 054919"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/7/e7791ccc5ca5d1aeb424c71c85c4c4fed49a0e7e_2_690x472.jpeg" alt="Screenshot 2023-02-12 054919" data-base62-sha1="x1HMeBJQZOeRd26pohPR5YVVFEy" width="690" height="472" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/7/e7791ccc5ca5d1aeb424c71c85c4c4fed49a0e7e_2_690x472.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/e/7/e7791ccc5ca5d1aeb424c71c85c4c4fed49a0e7e_2_1035x708.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/7/e7791ccc5ca5d1aeb424c71c85c4c4fed49a0e7e_2_1380x944.jpeg 2x" data-dominant-color="F2F1EE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2023-02-12 054919</span><span class="informations">1920×1315 192 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

InnovArul: <p>To your question, <code>netD</code>’s already accumulated gradients do not affect the backpropagation.</p>
<p>The accumulated gradients do not participate in the backpropagation (or chain rule).<br>
i.e., When calculating gradients for the current forward data, at a particular layer, only the gradients from current forward data are used to calculate gradients to be backpropagated.</p>

jayjay: <p>Thank InnovArul,</p>
<p>Your answer explains a lot to me, and do you know where I can get more details about “When calculating gradients for the current forward data, at a particular layer, only the gradients from current forward data are used to calculate gradients to be backpropagated”?</p>
<p>I want to read more about it.</p>

InnovArul: <p>You can read any standard blog about chain rule I guess.<br>
Pytorch’s backpropagation is the same as any standard chain rule and gradient calculation.</p>
<p>If it helps, you can also check using a simple program like below to verify that the <code>.grad</code> is just a buffer to hold/accumulate gradients &amp; they do not participate in backpropagation.</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.nn.functional as F

class NN(nn.Module):
    def __init__(self) -&gt; None:
        super().__init__()
        self.lin0 = nn.Linear(4,6)
        self.lin1 = nn.Linear(6,3)

    def forward(self, x):
        out = self.lin0(x)
        out = F.relu(out)
        out = self.lin1(out)
        return out

if __name__ == "__main__":
    net = NN()
    data = torch.randn(2,4,requires_grad=True)

    def print_grad():
        print(f"data grad {data.grad}")
        print(f"lin0 grad {net.lin0.weight.grad}")
        print(f"lin1 grad {net.lin1.weight.grad}")

    def zero_grad():
        # zero grad on both data and model
        net.zero_grad()
        data.grad = None

    print("forward 0")
    out = net(data)
    out.sum().backward()
    print_grad()
    zero_grad()
    print("grads after zero_grad")
    print_grad()

    print("forwarding 2 times without zero_grad()")
    out = net(data)
    out.sum().backward()
    out = net(data)
    out.sum().backward()
    print_grad()

</code></pre>

jayjay: <p>Thank you for the toy code to help my understanding.<br>
I summarized what I understood about your answer here.<br>
If you don’t mind, would you take a look at the attached figure and let me know whether my understanding is correct or not?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/0/00cf1865a147c96ab53b8162b93b6f921ff81641.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/00cf1865a147c96ab53b8162b93b6f921ff81641" title="Screenshot 2023-02-12 205002"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/0/00cf1865a147c96ab53b8162b93b6f921ff81641_2_690x410.jpeg" alt="Screenshot 2023-02-12 205002" data-base62-sha1="79HnoJMyvYxFTCPUzUULztA00N" width="690" height="410" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/0/00cf1865a147c96ab53b8162b93b6f921ff81641_2_690x410.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/0/0/00cf1865a147c96ab53b8162b93b6f921ff81641_2_1035x615.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/0/0/00cf1865a147c96ab53b8162b93b6f921ff81641_2_1380x820.jpeg 2x" data-dominant-color="F6F5F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2023-02-12 205002</span><span class="informations">1920×1142 99.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

InnovArul: <p>Your diagram seems correct to me.</p>
<p>One comment about <code>dL_2 / do_2</code>:<br>
You mentioned that</p>
<blockquote>
<p><code>dL_2 / do_2</code> is calculated by relationship between netB output and netB input (not parameters)`</p>
</blockquote>
<p>Note that <code>w_1</code> could appear in <code>dL_2 / do_2</code>. It depends on the functionality of <code>netB</code>.<br>
For instance, if <code>netB</code> calculates <code>w_1 * o_2</code> and <code>L_2</code> is squared error (i.e., <code>(w_1 * o_2 - t) ^2</code>) with target <code>t</code>, then   <code>dL_2 / do_2 = 2 * (w_1*o_2 - t) * w_1</code>.<br>
maybe you already know this. Just wanted to mention it.</p>

jayjay: <p>You are right about " w_1 could appear in dL_2 / do_2", but what I intended to talk about in “dL_2 / do_2 is calculated by the relationship between netB output and netB input (not parameters)” is “dL_2/do_2 doesn’t depend on partial derivatives w.r.t. w_1 (dw_1)”.</p>
<p>I made the statement confusing as if the meaning is “dL_2 / do_2 doesn’t depend on w_1”.</p>
<p>Do you think “dL_2/do_2 doesn’t depend on partial derivatives w.r.t. w_1 (dw_1)'” is correct?</p>

InnovArul: <p>Yes that statement (<code>dL_2/do_2</code> doesn’t depend on partial derivatives w.r.t. w_1 (dw_1)) seems correct to me.</p>