Loss.grad always prints None

Jeet: <pre><code class="lang-auto">class Parent(nn.Module):

  def __init__(self,in_features,z_dim, img_dim):
        super().__init__()
        self.my_child1 = Child1 (z_dim, img_dim)
        self.my_child2 = Child2 (in_features)
      

  def forward(self,input):
         input=self.my_child1(input)
         input=self.my_child2(input)    
         return input
  
  def forward1(self,input):
         input=self.my_child1(input)
         return input
         
  def forward2(self,input):
         input=self.my_child2(input)
         return input


class Child2(nn.Module):
    def __init__(self, in_features):
        super().__init__()
        self.child2 = nn.Sequential(
            nn.Linear(in_features, 128),
            nn.LeakyReLU(0.01),
            nn.Linear(128, 1),
            nn.Sigmoid(),
        )

    def forward(self, x):
        return self.child2(x)


class Child1(nn.Module):
    def __init__(self, z_dim, img_dim):
        super().__init__()
        self.child1 = nn.Sequential(
            nn.Linear(z_dim, 256),
            nn.LeakyReLU(0.01),
            nn.Linear(256, img_dim),
            nn.Tanh(), 
        )

    def forward(self, x):
        return self.child1(x)

criterion=nn.BCELoss()
noise = torch.randn(batch_size, z_dim).to(device)
model=Parent(in_features,z_dim, img_dim)
output1=model(noise)
loss1=criterion(output1,torch.ones_like(output1))
loss2=criterion(output1,torch.zeroes_like(output1))
loss3=(loss1+loss2)/2
model.zero_grad()
loss3.backward(retain_graph=True)
print(loss3.grad)
</code></pre>
<p>I have not used any optimizer here because updating the parameters are done using a seperate formula which I will employ only after I get the gradients. The formula requires the gradients to be stored in a matrix. However, the gradient always prints “None”.</p>
<p>What is the issue here?</p>

KFrank: <p>Hi Jeet!</p>
<aside class="quote no-group" data-username="Jeet" data-post="1" data-topic="148809" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>the gradient always prints “None”.</p>
</blockquote>
</aside>
<p>First, be aware of the following:</p>
<p>When you call <code>loss.backward()</code>, the gradients (that I think) you want<br>
won’t be stored in <code>loss.grad</code>.  Rather, the gradient of <code>loss</code> with respect<br>
to, for example, <code>my_child1</code>’s first <code>Linear</code>’s <code>bias</code> parameter will be stored<br>
in that parameter’s <code>.grad</code> property, <code>.bias.grad</code>.</p>
<p>(You won’t have any single tensor that has all of the gradients in it unless<br>
you assemble such a tensor yourself.)</p>
<p>As an aside, if you want to get an actual gradient value for <code>loss</code> itself, call<br>
<code>loss.retain_grad()</code> before calling <code>loss.backward()</code>, but, as <code>loss</code> is the<br>
root of the computation graph, <code>loss.grad</code>  will always be set to the trivial<br>
value <code>1.0</code> (which makes sense, but isn’t useful).</p>
<p>Best.</p>
<p>K. Frank</p>