Update network weights and then calculate gradients to those gradients

otb: <p>Hi,<br>
I’m trying to train 2 networks in the following scheme:</p>
<p>netA, netB = Net(), Net()<br>
netA_opt = torch.optim.SGD(netA.parameters(), lr=0.1)<br>
netB_opt = torch.optim.SGD(netB.parameters(), lr=0.1)</p>
<p><span class="hashtag">#Phase</span> 1:<br>
y1_A = netA(X1)<br>
l1 = cretirion1(y,y1_A)<br>
netA_opt.zero_grad()<br>
l1.backward()</p>
<p><span class="hashtag">#Phase</span> 2:<br>
y2_A = netA(X2)<br>
y2_B = netB(X2)<br>
l2 = cretirion2(y2_A,y2_B)<br>
netB_opt.zero_grad()<br>
l2.backward(retain_graph=True)<br>
netB_opt.step()</p>
<p><span class="hashtag">#Phase</span> 3:<br>
y1_B = netB(X1)<br>
l3 = cretirion3(y,y1_B)<br>
l3.backward()<br>
netA_opt.step()</p>
<p>Now, in phase 2 netB is updated as a function of netA predictions. In phase 3 I want to optimize netA to give better updates in phase 2, so netB will minimize l3. In practice I see that l3.backward() does not calculate any gradients w.r.t netA. What should I change?</p>
<p>Thanks in advance!</p>

ptrblck: <p>Phase3 doesn’t use any parameters of <code>netA</code> to calculate the loss <code>l3</code>, so no gradients will be computed for parameters in <code>netA</code>.<br>
If you want to optimizer <code>netA</code> in phase3, you would have to pass <code>y1_a</code> to <code>netB</code> so that the computation graph includes both models.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

otb: <p>Thanks for the answer and the tip <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="101255">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Phase3 doesn’t use any parameters of <code>netA</code> to calculate the loss <code>l3</code></p>
</blockquote>
</aside>
<p>This is the key problem.<br>
In phase 2, <code>netB</code> is trained as a function of <code>netA</code>, so I want <code>netA</code> to be a part of the computation graph created by <code>netB</code>. So when <code>netB</code> giving predictions, <code>netA</code> will get gradients.</p>
<p>In other words,</p>
<aside class="quote no-group" data-username="otb" data-post="1" data-topic="101255">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/otb/48/30373_2.png" class="avatar"> otb:</div>
<blockquote>
<p>l2.backward(retain_graph=True)<br>
netB_opt.step()</p>
</blockquote>
</aside>
<p>Calculate the gradients of <code>netB</code> w.r.t <code>netA</code> predictions and update <code>netB</code>, but it does not save this update as a function of <code>netA</code> (which it is what I want to do…)</p>