Possibility of combining two model with different classes on Pytorch

AnhMinhTran: <p>Hi guys, just wondering if it is possible to combine two different model training on different dataset into one?<br>
To illustrate my goal<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/7/474660f9aa8931ca2891117e7e2922f87c7b46d8.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/474660f9aa8931ca2891117e7e2922f87c7b46d8" title="Inkedannotation_LI"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/7/474660f9aa8931ca2891117e7e2922f87c7b46d8_2_666x500.jpeg" alt="Inkedannotation_LI" data-base62-sha1="aawGy9WEveyiZbFbhVdph2lrRJC" width="666" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/7/474660f9aa8931ca2891117e7e2922f87c7b46d8_2_666x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/4/7/474660f9aa8931ca2891117e7e2922f87c7b46d8_2_999x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/7/474660f9aa8931ca2891117e7e2922f87c7b46d8_2_1332x1000.jpeg 2x" data-dominant-color="F4F4F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Inkedannotation_LI</span><span class="informations">3840×2880 489 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>For model A: I have A.pth<br>
For model B(red colour): I have B.pth</p>
<p>Is it possible to combine them into one since the data is too skewed</p>

KFrank: <p>Hi Ahn!</p>
<aside class="quote no-group" data-username="AnhMinhTran" data-post="1" data-topic="118300" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/anhminhtran/48/37156_2.png" class="avatar"> AnhMinhTran:</div>
<blockquote>
<p>Hi guys, just wondering if it is possible to combine two different model training on different dataset into one?</p>
</blockquote>
</aside>
<p>You won’t be able to combine the two models together in some naive<br>
way.  The problem is that while model A has learned to distinguish a<br>
“Worm” from a “Brittlestar” and model B has learned to distinguish a<br>
“Prawn” from a “Seaspider,” neither model has learned to distinguish<br>
a “Worm” from a “Prawn.”</p>
<p>So unless you do some training where your training data includes<br>
<em>both</em> “Worm” and “Prawn” samples, your combined model will likely<br>
not work well.</p>
<aside class="quote no-group">
<blockquote>
<p>Is it possible to combine them into one since the data is too skewed</p>
</blockquote>
</aside>
<p>You appear to be training a multi-class classifier.  A standard loss<br>
function for doing so is <a href="https://pytorch.org/docs/1.8.1/generated/torch.nn.CrossEntropyLoss.html#crossentropyloss" rel="noopener nofollow ugc">CrossEntropyLoss</a>.  One common technique for<br>
dealing with skewed (unbalanced) data is to use <code>CrossEntropyLoss</code>’s<br>
<code>weight</code> constructor-argument to weight less common classes more<br>
heavily in your training.</p>
<p>Depending on how your training data is structured, you could also<br>
experiment with <a href="https://pytorch.org/docs/1.8.1/data.html#torch.utils.data.WeightedRandomSampler" rel="noopener nofollow ugc">WeightedRandomSampler</a>.</p>
<p>Good luck.</p>
<p>K. Frank</p>