In case of more than 1 Neural network, which parameters are considered for backpropagation in pyTorch?

Jeet: <pre><code class="lang-auto">class Parent(nn.Module):

  def __init__(self,in_features,z_dim, img_dim):
        super().__init__()
        self.my_child1 = Child1 (z_dim, img_dim)
        self.my_child2 = Child2 (in_features)
      

  def forward(self,input):
         input=self.my_child1(input)
         input=self.my_child2(input)    
         return input
  
  def forward1(self,input):
         input=self.my_child1(input)
         return input
         
  def forward2(self,input):
         input=self.my_child2(input)
         return input


class Child2(nn.Module):
    def __init__(self, in_features):
        super().__init__()
        self.child2 = nn.Sequential(
            nn.Linear(in_features, 128),
            nn.LeakyReLU(0.01),
            nn.Linear(128, 1),
            nn.Sigmoid(),
        )

    def forward(self, x):
        return self.child2(x)


class Child1(nn.Module):
    def __init__(self, z_dim, img_dim):
        super().__init__()
        self.child1 = nn.Sequential(
            nn.Linear(z_dim, 256),
            nn.LeakyReLU(0.01),
            nn.Linear(256, img_dim),
            nn.Tanh(), 
        )

    def forward(self, x):
        return self.child1(x)

criterion=nn.BCELoss()
model=Parent(in_features,z_dim, img_dim)
output1=model.forward(noise)
loss=criterion(output1,torch.ones_like(output1))
loss.backward()
</code></pre>
<p>I have a parent module named as Parent and it has 2 child components packed in it. The child modules are defined below the parent component.</p>
<p>Now when loss.backward() is called, backpropagation is conducted with respect to which parameters? (Parameters of child1/child2 or both?)</p>
<p>What should I do if I need to conduct backpropagation on any of the child network? Could I take help of forward1() or forward2() method from the Parent module or do I need to call the child modules seperately?</p>

ptrblck: <aside class="quote no-group" data-username="Jeet" data-post="1" data-topic="148721">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>Now when loss.backward() is called, backpropagation is conducted with respect to which parameters? (Parameters of child1/child2 or both?)</p>
</blockquote>
</aside>
<p>All parameters which were used in the forward pass will get the corresponding gradient accumulated into their <code>.grad</code> attribute.<br>
In your example <code>my_child1</code> and <code>my_child2</code> are used and thus their parameters will get the gradients.</p>
<aside class="quote no-group" data-username="Jeet" data-post="1" data-topic="148721">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jeet/48/47500_2.png" class="avatar"> Jeet:</div>
<blockquote>
<p>What should I do if I need to conduct backpropagation on any of the child network? Could I take help of forward1() or forward2() method from the Parent module or do I need to call the child modules seperately?</p>
</blockquote>
</aside>
<p>You could use your custom <code>forwardX</code> method, but note that you would break e.g. hooks.<br>
PyTorch depends on the implementation of the <code>forward</code> method and will call it internally in its <code>__call__</code> method. This is also the reason why you should call the model directly:</p>
<pre data-code-wrap="python"><code class="lang-python">out = model(input)
</code></pre>
<p>instead of its <code>forward</code> method:</p>
<pre data-code-wrap="python"><code class="lang-python">out = model.forward(input)
</code></pre>

Jeet: <p>Thanks a lot for the clarification! I have a requirement to build a Jacobian matrix where the row1 will have the partial derivative of the loss of the child1 with respect to all the parameters and the second row should have the loss of the child2 with repect to all the parameters. How can I build a matrix like that with the help of loss.backward() ?</p>