Merging two models

muammar: <p>I want to implement a model similar to the one described in the picture below taken from <a href="https://datascience.stackexchange.com/questions/26103/merging-two-different-models-in-keras" class="inline-onebox" rel="noopener nofollow ugc">machine learning - Merging two different models in Keras - Data Science Stack Exchange</a></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/e/ee94c295235fb52a2cdcb10c526406f93ea3791b.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/ee94c295235fb52a2cdcb10c526406f93ea3791b" title="mergedmodels"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/e/ee94c295235fb52a2cdcb10c526406f93ea3791b_2_474x500.jpeg" alt="mergedmodels" data-base62-sha1="y2Am0z4s3xCUK86fEDEp2uO0Ys3" width="474" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/e/ee94c295235fb52a2cdcb10c526406f93ea3791b_2_474x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/2X/e/ee94c295235fb52a2cdcb10c526406f93ea3791b_2_711x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/e/ee94c295235fb52a2cdcb10c526406f93ea3791b.jpeg 2x" data-dominant-color="EFF1F0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">mergedmodels</span><span class="informations">851×897 98 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I have implementations of <code>ModelA</code> and <code>ModelB</code> that work fine when I train them separately. I am thinking of creating a class that will merge both of them inspired by this: <a href="https://discuss.pytorch.org/t/combining-trained-models-in-pytorch/28383/2" class="inline-onebox">Combining Trained Models in PyTorch - #2 by ptrblck</a>.</p>
<p>My questions would be:</p>
<ol>
<li>How do I handle the parameters? I want to pass <code>ModelA</code> and <code>ModelB</code> parameters to the same optimizer object. Can they be concatenated easily with some pytorch funcion?.</li>
<li>Should the loss function be the sum of <code>lossA</code> and <code>lossB</code>?. That question comes because even though the data set is the same for both models, <code>ModelA</code>’s loss “units” are different from those of <code>ModelB</code>’s. For example: <code>lossA</code> are in units of length and   <code>lossB</code> are in units of force or energy.</li>
</ol>
<p>I would appreciate any advice with this and even some code <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"> I am working on this and when coming to a solution I will post it here for reference.</p>

ptrblck: <ol>
<li>You could just add the parameters lists:</li>
</ol>
<pre><code class="lang-python">optimizer = optim.SGD(list(modelA.parameters()) + list(modelB.parameters()), lr=1e-3)
</code></pre>
<p>How are you transferring the parameters from layer A2 to B1? If so, the weight matrix will have a size mismatch (<code>[30, 8]</code> vs. <code>[40, 8]</code>).</p>

muammar: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="45637">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<ol>
<li>You could just add the parameters lists:</li>
</ol>
<pre><code class="lang-auto">optimizer = optim.SGD(list(modelA.parameters()) + list(modelB.parameters()), lr=1e-3)
</code></pre>
</blockquote>
</aside>
<p>This is so cool about pytorch. So simple.</p>
<aside class="quote no-group">
<blockquote>
<p>How are you transferring the parameters from layer A2 to B1? If so, the weight matrix will have a size mismatch ( <code>[30, 8]</code> vs. <code>[40, 8]</code> ).</p>
</blockquote>
</aside>
<p>I have a method class that extract those activations and I will be using them as inputs for B1. I hope, but might be wrong, that I won’t encounter size mismatches! I will report back later about it.</p>

alex.veuthey: <p>So you are still using the beginning part of model A (A1 and A2) as inputs to B2? The hybrid model would then be A1, A2/B1, B2, B3 with shapes 30x8, 8x40, 40x30 and it should work…</p>

muammar: <aside class="quote no-group" data-username="alex.veuthey" data-post="4" data-topic="45637" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alex.veuthey/48/12176_2.png" class="avatar"> alex.veuthey:</div>
<blockquote>
<p>So you are still using the beginning part of model A (A1 and A2) as inputs to B2? The hybrid model would then be A1, A2/B1, B2, B3 with shapes 30x8, 8x40, 40x30 and it should work…</p>
</blockquote>
</aside>
<p>I will be using just the activations of the latent layer. I also think it should work.</p>

alex.veuthey: <p>Do you compute the activations beforehand / offline for all your data and store them as features / inputs for model B later? That should also work, but I was thinking you would compute them on the fly at each forward pass. But in any case, both should work.</p>

muammar: <p><a class="mention" href="/u/alex.veuthey">@alex.veuthey</a>  I don’t. What I do is I forward propagate model A (an autoencoder), and subsequently use the latent hidden layer as input for model B (everything online?). I finally got time to keep working again on this and am to the point where I concatenate the parameters of the models in one single list.</p>

muammar: <p>I think this is working! However, I had a doubt. In my implementation, I am doing the following:</p>
<p>Create a <code>ModelMerger</code> class where I pass a list of models, and I initialize an <code>optimizer</code> with what <a class="mention" href="/u/ptrblck">@ptrblck</a> mentioned above.</p>
<p>Then I iteratively do:</p>
<ol>
<li>Forward propagation of <code>ModelA</code>, computation of loss function, backward propagation, accumulation of gradients following idea in <a href="https://discuss.pytorch.org/t/how-to-get-and-or-set-gradient-data-during-training">https://discuss.pytorch.org/t/how-to-get-and-or-set-gradient-data-during-training</a>.</li>
<li>Extraction of layer <code>A2</code> to use it as input for <code>ModelB</code>.</li>
<li>Forward propagation of <code>ModelB</code> using <code>A2</code> as inputs, and computation of loss function, accumulation of gradients.</li>
</ol>
<p>Once each model finishes the steps above, I do a <code>self.optimizer.step()</code>. I just do a</p>
<pre><code class="lang-auto">optimizer = optim.SGD(list(modelA.parameters()) + list(modelB.parameters()), lr=1e-3)
</code></pre>
<p>Is this correct? It seems to me that pytorch is so smart that figures out everything. I wonder if I should sum the losses before doing <code>loss.backward()</code> or running them independently is fine? Would that change anything?</p>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a>, <a class="mention" href="/u/alex.veuthey">@alex.veuthey</a> I would appreciate any input from you guys :), and thanks for your help!</p>

alex.veuthey: <p>Summing the losses (can be done with weighting or equal proportions) is definitely different from running them separately. If you do <code>loss.backward()</code> with the (weighted) sum then everything is backpropagated, which is not necessarily what you want.</p>
<p>In your case, I think it depends on whether you want to train weights in <code>ModelA</code> when you train <code>ModelB</code>, or if you keep them separate (i.e. the weights in A are fixed when training B). If you include parameters of A in the second loss as a (possibly weighted) sum, then those parameters are going to be updated by the optimizer step.</p>
<p>Also if you simply take the A loss and add it to the B loss, then there are weights in A that contribute to the A loss but should not contribute to the B loss (in your drawing: layer A3). So I guess you would need an intermediate A loss that takes that into account.</p>
<p>Disclaimer: I’m not familiar with and have never trained an encoder/decoder network, but what I’ve described seems like the logical behaviour to me…</p>

muammar: <blockquote>
<p>Summing the losses (can be done with weighting or equal proportions) is definitely different from running them separately. If you do <code>loss.backward()</code> with the (weighted) sum then everything is backpropagated, which is not necessarily what you want.</p>
</blockquote>
<p>Thanks for clarifying this. I would like the weights of <code>ModelA </code> to be dependent on <code>ModelB</code>, too. Therefore I should be combining the losses to backward propagate.  I guess something like:</p>
<pre><code class="lang-auto">losses =  (a * lossA) + (b * lossB)
losses.backward()
</code></pre>
<blockquote>
<p>In your case, I think it depends on whether you want to train weights in <code>ModelA</code> when you train <code>ModelB</code> , or if you keep them separate (i.e. the weights in A are fixed when training B). If you include parameters of A in the second loss as a (possibly weighted) sum, then those parameters are going to be updated by the optimizer step.</p>
</blockquote>
<p>I was getting confused because I am basically calling <code>lossA.backward()</code> and <code>lossB.backward() </code> separately but the <code>optimizer</code> object gets <code>list(modelA.parameters()) + list(modelB.parameters())</code>. I thought that that would make the trick, but as your reply suggests, that is not correct. When I do that I am basically updating, for example, <code>modelA</code>’s  parameters using gradients of  <code>modelA.parameters()</code> only.</p>
<blockquote>
<p>Also if you simply take the A loss and add it to the B loss, then there are weights in A that contribute to the A loss but should not contribute to the B loss (in your drawing: layer A3). So I guess you would need an intermediate A loss that takes that into account.</p>
</blockquote>
<p>This is an interesting point. I think I will have to explore because I have no idea.</p>
<blockquote>
<p>Disclaimer: I’m not familiar with and have never trained an encoder/decoder network, but what I’ve described seems like the logical behaviour to me…</p>
</blockquote>
<p>That is fine. Your comments make sense to me. I am not familiar with these models either <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20">. Thank you, this has been very useful. This community is very cool.</p>

css_ai: <p>Hi, you club two model weight file into one mode weight, but my question is is there any catastrophic loss is happen ya not for data A and Data B</p>