How to handle two separate optimizers and separate losses?

JimW: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/f/8ffcc572806bdf18fee9781114e184da2cba16bd.png" data-download-href="https://discuss.pytorch.org/uploads/default/8ffcc572806bdf18fee9781114e184da2cba16bd" title="Pytorch"><img src="https://discuss.pytorch.org/uploads/default/original/3X/8/f/8ffcc572806bdf18fee9781114e184da2cba16bd.png" alt="Pytorch" data-base62-sha1="kxLPYcc3ongJMGEaOPuWHr3KVvT" width="551" height="500" data-dominant-color="F4F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Pytorch</span><span class="informations">690×626 4.17 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Sorry for my terrible drawing. But according to the above figure, I want to use 2 separate losses to update different modules in the entire network. The data flow is illustrated with arrows in the above figure, I hope to define a loss1 to only update “trainable 1”, and another term loss2 to only update “trainable 2”.</p>
<p>I apologize because I can’t paste our internal code here, so I have to briefly mention what I did:<br>
I defined two optimizers: optimizer_1 (with parameters in “trainable 1” as trainable variables) and optimizer_2 (with parameters in “trainable 2” as trainable variables).</p>
<p>For each iteration of my training loop, I did something like this:</p>
<pre><code class="lang-auto">optimizer_1.step()
optimizer_2.step()
loss1.backward()
loss2.backward()
</code></pre>
<p>And crashed with the following error message<br>
<code># [RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time]</code></p>
<p>I am not sure if I should handle these two losses in this way. Could someone tell me what  might be potentially wrong in my implementation, and how to fix it?</p>
<p>Thanks.</p>

Programmer-RD-AI: <aside class="quote" data-post="1" data-topic="11085">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/f04885/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/two-optimizers-for-one-model/11085">Two optimizers for one model</a> 
  </div>
  <blockquote>
    Is there a way to use two optimizers for one model in more beautiful way? 
Now, as I understand we should do something like these: 
net = Model()
part_1_parameters = ...
part_2_parameters = ...
opt1 = optimizer_1(part_1_parameters)
opt2 = optimizer_2(part_1_parameters)

### train epoch
opt1.zero_grad()
opt2.zero_grad()

loss = ...
loss.backward()

opt1.step()
opt2.step()
  </blockquote>
</aside>

<aside class="quote quote-modified" data-post="1" data-topic="117693">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tcsn_wty/48/11956_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-to-run-with-multiple-gpu-when-having-2-optimizers/117693">How to run with multiple GPU when having 2 optimizers?</a> 
  </div>
  <blockquote>
    I think I totally understood the tutorial on using the following line of code to do data parallel. 
model = nn.DataParallel(model) 

I tested it on my device with 2 GPUs and it worked. However, for a complicated model I recently wrote, it doesn’t truly train on both GPUs when I printed the ‘Inside’ ‘Outside’ debugging message. 
I managed to figure out that it was because of the optimizers. To my purpose, I need to have two optimizers each contains some shared part and some unique part. 
model = …
  </blockquote>
</aside>

<aside class="quote quote-modified" data-post="1" data-topic="87348">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/scott_hoang/48/9907_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/correct-way-to-declare-multiple-optimizers-for-single-model-in-ddp/87348">Correct way to declare multiple optimizers for single model in DDP</a> <a class="badge-category__wrapper " href="/c/distributed/12"><span data-category-id="12" data-drop-close="true" class="badge-category "><span class="badge-category__name">distributed</span></span></a>
  </div>
  <blockquote>
    Given the pseudo model below: 
class model(nn.Module):
     def __init__(self):
            self.alpha_params1 = nn.Parameters(&lt;size&gt;, requires_grad=True)
            self.alpha_params2 = nn.Parameters(&lt;size&gt;, requires_grad=True)

           &lt; typical Conv2d layers&gt;....
     
    def forward(self, x):
           &lt;feed forward&gt;
            return output

   def net_parameters(self, filtered_name='alpha_', recurse=True):
          # return torch layer params      
          for name, param in self…
  </blockquote>
</aside>

<p>check the above discussions</p>

JimW: <p>Thank you so much for the pointers.<br>
Could you also comment on this “# [RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time]” error?</p>
<p>I tried to use splited parameters (i.e. trainable_1 assigned to optimizer1, and trainable_2 assigned to optimizer2, while there should not be overlap btw trainable_1 and trainable_2) for different optimizers to separate the two training flow.  Where could be wrong to trigger this “Trying to backward through the graph a second time” error?  BTW, I don’t fully understand this error, and  will “retain_graph=True” be a correct fix in this case?</p>

my3bikaht: <p>When you do backward pass, some of the values of the graph for the intermediate (non-leaf) nodes are freed. So, second pass over same nodes will trigger this error. To be able to proceed with second pass, you need to add parameter to loss1.backward(retain_graph=True)</p>