Backprop min loss of losses from two models to both models

alwynmathew: <p>I have two models(<code>model1</code> and <code>model2</code>) running in coupled manner</p>
<pre><code class="lang-auto">output1 = model1(input)
output2 = model2(input)
loss1= loss_fn(output1)
loss2= loss_fn(output2)
loss = min(loss1, loss2)
</code></pre>
<p>How do I propogate this <code>loss</code> to both <code>model1</code> and <code>model2</code> ?</p>
<p>NOTE: I understand</p>
<pre><code class="lang-auto">loss = loss1 + loss2
loss.backward()
</code></pre>
<p>the above could work but I need the <code>min(loss1, loss2)</code> to be back prop to both models.</p>

ptrblck: <p><code>torch.min</code> acts like a switch, so I’m not sure how both models should get valid gradients.</p>

alwynmathew: <p>Yea I understand if <code>min(loss1, loss2)</code> is <code>loss1</code>, <code>loss.backward()</code> will only backprop through <code>model1</code> and vice versa. Please suggest a better or alternative way to solve this. I need to backprop min of <code>loss1</code> and <code>loss2</code> through both models.</p>

KFrank: <p>Hi Alwyn!</p>
<aside class="quote no-group" data-username="alwynmathew" data-post="3" data-topic="59746" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alwynmathew/48/8684_2.png" class="avatar"> alwynmathew:</div>
<blockquote>
<p>Yea I understand if <code>min(loss1, loss2)</code> is <code>loss1</code>, <code>loss.backward()</code> will only backprop through <code>model1</code> and vice versa. Please suggest a better or alternative way to solve this. I need to backprop min of <code>loss1</code> and <code>loss2</code> through both models.</p>
</blockquote>
</aside>
<p>In order for the gradients to flow through both branches of your<br>
“min” function,  you need a smooth version of min that transitions<br>
from one branch to the other adequately gradually.</p>
<p><a href="https://en.wikipedia.org/wiki/LogSumExp" rel="noopener nofollow ugc">LogSumExp</a> is a smooth version of the max function, and is<br>
implemented in pytorch as <a href="https://pytorch.org/docs/stable/torch.html#torch.logsumexp" rel="noopener nofollow ugc">torch.logsumexp()</a>.  You can turn it into<br>
a smooth minimum by adding minus signs:</p>
<p><code>smooth_min = -torch.logsumexp (-arg, dim)</code></p>
<p>You can make it transition more or less abruptly by adding a<br>
parameter <code>q</code>:</p>
<p><code>-torch.logsumexp (-q * arg, dim) / q</code></p>
<p>Note, if <code>loss1</code> is much larger than <code>loss2</code> (or if you transition<br>
abruptly), then the gradients will flow only weakly back through<br>
<code>model2</code> (which may well be what you want).</p>
<p>Good luck.</p>
<p>K. Frank</p>
<p>[Edit:  However, upon further consideration, this scheme might<br>
not do what you want.  Let’s say, that by happenstance, <code>model2</code><br>
starts doing better than <code>model1</code>.  Then gradients will flow more<br>
weakly through <code>model1</code>, so it will learn more slowly.  So <code>model2</code><br>
will get better still while <code>model1</code> stagnates.</p>
<p>If <code>model2</code> gets sufficiently better than <code>model1</code>, then <code>model1</code><br>
will effectively drop out of the picture, regards of whether, given<br>
the chance and adequate training, it might have ended up doing<br>
better than <code>model2</code>.]</p>