Save and load specific layers of yolov5, to aplly in diferent heads during inference?

username: aspiranteaphd
cooked: <p>I have trained a yolov5 (Model A), and I want to train another model (Model B) using the backbone of “A”, and a specific head.</p>
<p>Example problem.<br>
I have a dataset of traffic scenes that include different types of vehicles. For example, the same image can have a class(Super category) of “Car” and class (category) of “BMW” or “Mercedes”[Model A], another image can have a class (Super category) of “Motorcycle”, and class of “Honda” or “Yamaha”[Model B].</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/7/e7be955b89608ce7aaf4c77c1ebce124f1777b38.png" data-download-href="https://discuss.pytorch.org/uploads/default/e7be955b89608ce7aaf4c77c1ebce124f1777b38" title="imagem"><img src="https://discuss.pytorch.org/uploads/default/original/3X/e/7/e7be955b89608ce7aaf4c77c1ebce124f1777b38.png" alt="imagem" data-base62-sha1="x46CmBNvJmBogCiaJ8sG0dhPwRa" width="690" height="239" data-dominant-color="FAF9F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">imagem</span><span class="informations">805×279 3.99 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>–<br>
Since the input images and therefore most of the features should be the same, I wish to train one model then freeze it’s feature-extractor, and sort of transfer learn the same model for the categories using the pretrained feature extractor.</p>
<p>During inference, I want to load Model A and do its prediction → Load Model B Head → Swap “Model A” head with “Model B” → Do the prediction of secondary model.</p>
<p>All of this process has the objective of saving computational resources, since the weights of the feature extracting backbone is the same, I only want to save the weights of the head B.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="aspiranteaphd" data-post="1" data-topic="168192">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/ed655f/48.png" class="avatar"> aspiranteaphd:</div>
<blockquote>
<p>Since the input images and therefore most of the features should be the same, I wish to train one model then freeze it’s feature-extractor, and sort of transfer learn the same model for the categories using the pretrained feature extractor.</p>
<p>During inference, I want to load Model A and do its prediction → Load Model B Head → Swap “Model A” head with “Model B” → Do the prediction of secondary model.</p>
</blockquote>
</aside>
<p>The idea sounds valid assuming you could replace the “head” attribute in the original model with your custom modules. Did you try to use this approach and are you seeing any issues?</p>

username: aspiranteaphd
cooked: <aside class="quote no-group" data-username="aspiranteaphd" data-post="1" data-topic="168192">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/ed655f/48.png" class="avatar"> aspiranteaphd:</div>
<blockquote>
<p>Since the input images and therefore most of the features should be the same, I wish to train one model then freeze it’s feature-extractor, and sort of transfer learn the same model for the categories using the pretrained feature extractor.</p>
<p>During inference, I want to load Model A and do its prediction → Load Model B Head → Swap “Model A” head with “Model B” → Do the prediction of secondary model.</p>
</blockquote>
</aside>
<p>Thanks for the fast response, Unfortunately I’m just giving the first steps in deep learning and PyTorch, I have read so much but couldn’t reach a conclusion on a starting point. I think a better example of my problem is shown below.</p>
<p>I have a dataset of traffic scenes that include different brands of cars. For example, the same image can have a ‘BMW’ or a ‘Mercedes’ among other brands. I want to create a model, where each head classifies a ‘car model’ for a specific brand, like the image below.</p>
<p><img src="https://discuss.pytorch.org/images/transparent.png" alt="imagem" data-orig-src="upload://9lktoUrmPpHiD29f1O2PZHkJFuo.png" width="690" height="239"></p>
<p>I kind of want to “future-prof” the model, and train only a specific head when a new car model launches. Also, one of the request was using yolov5.</p>
<p>Best regards.</p>