What are the parameters involved to calculate the gradient during backpropagation?

username: Jeet
cooked: <p>I have 2 neural networks namely the generator and the discriminator. I feed the output of the generator to the discriminator and the output of the dicriminator tot he generator. In this context I have some questions. My code is as follows  -</p>
<p>gen_fake=generator(z)<br>
disc_fake= disc(gen_fake)<br>
lossDreal=criterion(xxxx) //Here a forward graph has been created with the parameters of the discriminator?<br>
lossDfake=criterion(disc_fake,torch.ones()) //Here again a forward graph has been created but with which parameters? parameters of discriminator and the generator as disc_fake is fed in which is the output of the discriminator?<br>
lossD=lossDreal+lossDfake // Which parameters are involved here?</p>
<p>Now when I do :-</p>
<p>lossD.backward()// according to which paramters will be the gradient calculated?</p>
<p>and if I do lossD.backward(retain_graph=True) then later if I call lossG(loss of the generator)<br>
lossG.backward() //Will this also calculate the gradient with respect to both the paramters of the discriminator and the generator?</p>

username: srishti-git1110
cooked: <p>Hi Jeet,</p>
<p>Note that autograd records every operation (until explicitly turned off by some means), and so all the tensors that are involved in calculating a certain tensor shall be in the computation graph.</p>
<p>When backward is called on a certain tensor, the grad attribute of only the leaf tensors is populated by default. If you want to populate the grad attribute of intermediate tensors, use <a href="https://pytorch.org/docs/stable/generated/torch.Tensor.retain_grad.html" rel="noopener nofollow ugc">retain_grad</a> on the intermediate tensors.</p>
<p>This shall help you find better answers to your questions. If I try to answer them -</p>
<aside class="quote no-group" data-username="Jeet" data-post="1" data-topic="162145">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>Here a forward graph has been created with the parameters of the discriminator?</p>
</blockquote>
</aside>
<p>Depends on what you pass in place of xxxx.</p>
<aside class="quote no-group" data-username="Jeet" data-post="1" data-topic="162145">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>Here again a forward graph has been created but with which parameters? parameters of discriminator and the generator as disc_fake is fed in which is the output of the discriminator?</p>
</blockquote>
</aside>
<p>Parameters of both the discriminator and the generator should be the leaf tensors in the graph of <code>lossDfake</code>.</p>
<aside class="quote no-group" data-username="Jeet" data-post="1" data-topic="162145">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>// Which parameters are involved here?</p>
</blockquote>
</aside>
<p>Parameters of both the discriminator and the generator should be the leaf tensors in the graph of <code>lossD</code> as well.</p>
<aside class="quote no-group" data-username="Jeet" data-post="1" data-topic="162145">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>according to which paramters will be the gradient calculated?</p>
</blockquote>
</aside>
<p>This is answered by what I explained above about leaf tensors.</p>
<aside class="quote no-group" data-username="Jeet" data-post="1" data-topic="162145">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>and if I do lossD.backward(retain_graph=True) then later if I call lossG(loss of the generator)<br>
lossG.backward() //Will this also calculate the gradient with respect to both the paramters of the discriminator and the generator?</p>
</blockquote>
</aside>
<p>How do you calculate <code>lossG</code>?<br>
Assuming using <code>gen_fake</code> only, <code>lossG.backward(</code>) shall populate the grad attributes of the parameters of generator only.</p>
<p>Hope this helps,<br>
Srishti</p>

username: Jeet
cooked: <p>Hi Shrishti,</p>
<p>Thanks a lot for the detailed answer. It was really helpful but it has also given birth to few more questions that I would like to clarify.  Please help me out if possible.</p>
<p>Question 1 :- I am actually interested in the gradients of all the parameters involved in the discriminator and the generator but I have not done retain_grad=True anywhere. Does that affect my results? What do you mean by leaf tensors? So do I get the gradients of all the parameters in the discriminator and the generator by doing the following after calling <strong>lossD.backward()</strong>?<br>
x=torch.empty(0,0)<br>
x=torch.flatten(x)<br>
for param in disc.parameters():<br>
flat=torch.flatten(param.grad)<br>
con=torch.cat((x,flat),0)<br>
x=con</p>
<pre><code>w=torch.empty(0,0)
w=torch.flatten(w)
for param in gen.parameters():
    flat=torch.flatten(param.grad)
    con=torch.cat((w,flat),0)
    w=con
</code></pre>
<p>If not then where should I set the <strong>retain_grad=True</strong> ?</p>
<p>Question 2:- Since I am doing<br>
lossD=lossDreal+lossDfake<br>
lossG = criterion(disc_fake, torch.ones_like(disc_fake)) //tensors of discriminators are also there.<br>
lossD.backward(retain_graph=True)<br>
lossG.backward()<br>
How does <strong>retain_graph=True</strong>  matter here? What impact will it make if I remove the same because I could already get the gradients of all the parameters from the discriminator and the generator just by calling <strong>lossG.backward() after calling lossD.backward()</strong>?</p>
<p>It would be really very helpful if you could clarify the above two questions!</p>

username: srishti-git1110
cooked: <p>Hi Jeet,</p>
<aside class="quote no-group" data-username="Jeet" data-post="3" data-topic="162145">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>but I have not done retain_grad=True anywhere</p>
</blockquote>
</aside>
<p>retain_grad isn’t a tensor attribute, so there’s nothing like setting it to True. It’s a method call, like so</p>
<pre><code class="lang-auto">x = torch.tensor([1.0], requires_grad=True)
y = x**2
y.retain_grad()
print(y.requires_grad) # gives True
</code></pre>
<aside class="quote no-group" data-username="Jeet" data-post="3" data-topic="162145">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>Does that affect my results? What do you mean by leaf tensors?</p>
</blockquote>
</aside>
<p>When a tensor is first initialized, it becomes a leaf node/tensor.<br>
Basically, all inputs and weights of a neural network are leaf tensors and hence leaf nodes in the computational graph.</p>
<p>Whenever <em>any</em> operation is performed on a tensor, it does not remain a leaf node anymore. In the above example y is not a leaf is it a result of some operations on x; x is a leaf.</p>
<p>This also answers your question :</p>
<aside class="quote no-group" data-username="Jeet" data-post="3" data-topic="162145">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>So do I get the gradients of all the parameters in the discriminator and the generator by doing the following after calling <strong>lossD.backward()</strong>?</p>
</blockquote>
</aside>
<p>Yes, assuming <code>lossD</code> is what you’ve described in your first post.</p>
<p>[To check whether a tensor is a leaf tensor or not, use is_leaf.<br>
<code>y.is_leaf</code> gives False]</p>

username: srishti-git1110
cooked: <aside class="quote no-group" data-username="Jeet" data-post="3" data-topic="162145">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>How does <strong>retain_graph=True</strong> matter here?</p>
</blockquote>
</aside>
<p>Refer <a href="https://discuss.pytorch.org/t/why-is-retain-graph-true-needed-in-some-case-but-not-in-other/161705/4">this</a></p>
<aside class="quote no-group" data-username="Jeet" data-post="3" data-topic="162145">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>What impact will it make if I remove the same because I could already get the gradients of all the parameters from the discriminator and the generator just by calling <strong>lossG.backward() after calling lossD.backward()</strong>?</p>
</blockquote>
</aside>
<p>No need to use <code>retain_graph=True</code> if that runs error free for you.<br>
However, I’m anticipating it’ll not. <code>retain_graph=True</code> shall be required here as when you call <code>lossG.backward()</code> without it, you are basically trying to backward through the graph a second time after the saved tensors have already been freed which will produce an error.</p>

username: Jeet
cooked: <p>Thanks! You are correct! retain_graph=True is required here. Does these saved tensors also save their corresponding gradients which was calculated during lossD.backward() ? If these gradients are also saved then I need to do discriminator.zero_grad() and generator().zero_grad(), before I call lossG.backward(). Otherwise the gradient for the generator will be calculated over the gradient of the discriminator which I would like to avoid. So that when I do param.grad I should get the gradients calculated by the lossG and not by the lossD after calling lossG.backward().</p>

username: srishti-git1110
cooked: <aside class="quote no-group" data-username="Jeet" data-post="6" data-topic="162145">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>I need to do discriminator.zero_grad() and generator().zero_grad(), before I call lossG.backward(). Otherwise the gradient for the generator will be calculated over the gradient of the discriminator which I would like to avoid</p>
</blockquote>
</aside>
<p>Right. You’ll need to use <code>optimizer.zero_grad()</code> to prevent accumulation of gradients.</p>

username: Jeet
cooked: <p>Hi Shrishti, since I have no optimizer in my training as I am optimizing the network manually, how should I use the zero_grad()?<br>
If I have Discriminator and the Generator as two neural networks then can I do?<br>
disc = Discriminator(image_dim).to(device)<br>
disc.zero_grad()<br>
lossG.backward()</p>
<p>since I already have retain_graph=true while calling the lossD.backward()</p>

username: srishti-git1110
cooked: <p>Could you please elaborate what exactly do you mean by optimising manually?</p>
<p>Also, any specific reason not to use an optimization algorithm?</p>

username: Jeet
cooked: <p>I am optimizing the parameters of the neural network with the help of an optimizing equation. Due to this reason I need the partial prder derivatiove of the loss function of the neural networks(generator and the discriminator) with respect to all the parameters. So I am afraid that I need only the first order partial derivative of the lossg and lossD with respect to all the parameters(weights and biases). If somehow the gradient is calculated wrong or twice then my descent direction will be miscalculated.</p>

username: srishti-git1110
cooked: <p>Please post the exact code where you update the parameters using gradients.</p>
<p>As for your question, <code>optimizer.zero_grad()</code> essentially clears out the <code>grad</code> attribute of the parameters.</p>
<p>So, if you are doing things manually, you’d want to do that manually, too, to prevent accumulation of gradients on subsequent backward calls.</p>

username: Jeet
cooked: <p>What does the below code do?</p>
<p>disc = Discriminator(image_dim)<br>
disc.zero_grad()</p>
<p>Does it clear all the gradients of the Discriminator?</p>

username: srishti-git1110
cooked: <p>Calling zero_grad on a model instance created using <code>nn.Sequential</code> works.</p>
<p>I’m not sure if that’s the case with instances of custom model classes as well.</p>
<p>Update: Apparently, it works just as fine. Custom model classes should inherit from nn.module which has this method <code>nn.module.zero_grad()</code> &amp; so can be called on instances of subclasses.</p>
<aside class="quote no-group" data-username="Jeet" data-post="12" data-topic="162145">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>Does it clear all the gradients of the Discriminator?</p>
</blockquote>
</aside>
<p>Yes.</p>