Transfer learning for multi-label classification from a single-label model

username: KeepingItClassy11
cooked: <p>Greetings!</p>
<p>I’ve had great success with building multi-class, single-label classifiers as described in the <a href="http://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html" rel="nofollow noopener">official PyTorch transfer learning tutorial</a>. I have a couple of use cases that require a multi-label image classifier, and I was wondering whether/how I could use the same pre-trained model (e.g. ResNet-101) to train a multi-label classifier. I understand that I need to use a different loss function, but I’m not sure how I can get the resulting model to generate multi-label predictions.</p>

username: ptrblck
cooked: <p>The loss function should ensure your model predicts multi-label outputs.<br>
Since you most likely have to change the last layer(s) of the model, the classifier has to learn it’s output distributions from scratch.<br>
What are your doubts about it? Maybe I’m missing a point here.</p>

username: KeepingItClassy11
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>, I don’t think you’re missing anything, it’s more likely that I am. <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=5" title=":smile:" class="emoji" alt=":smile:"> My only experience with PyTorch has been fine-tuning a multi-class, single-label classifier on a pre-trained ResNet model as described in the tutorial, i.e. using a single-label dataset, resetting the final layer, and using the CrossEntropyLoss function.</p>
<p>However, I’m not sure about all the changes I need to make to that code to produce a multi-label classification model. Obviously I need to provide a multi-label training set for fine-tuning, and change the loss function to something like MultiLabelMarginLoss, but is there anything else I need to do? I’ve searched for tutorials and code examples specifically for multi-label image classification, but haven’t found any.</p>

username: ptrblck
cooked: <p>You could use <a href="http://pytorch.org/docs/stable/nn.html#torch.nn.BCELoss">BCELoss</a> for multi-label classification.<br>
Just apply a sigmoid on your model’s output and use the BCELoss.<br>
The target should have the same shape as the output in this case.</p>

username: pradpalnis
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> ,<br>
I have a similar kind of uses case as above to use the pre-trained model of a multi-label classifier.<br>
Single Images has to be classified with two labels.</p>
<p>Used below code to modify the last layer for single-label classifier of 10 classes ,<br>
in_features = resnet.fc.in_features<br>
resnet.fc = nn.Linear(in_features, 10 )</p>
<p>How to modify the last layer for two-label classifier each of 10 classes? I’m a newbie to pytorch</p>
<p>in_features = resnet.fc.in_features<br>
resnet.fc = nn.Linear(in_features, ? )</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="pradpalnis" data-post="5" data-topic="16755">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/c6cbf5/48.png" class="avatar"> pradpalnis:</div>
<blockquote>
<p>Single Images has to be classified with two labels.</p>
</blockquote>
</aside>
<p>Does it mean you are working with 10 labels and for each sample two of these labels are active?<br>
If so, your code looks fine:</p>
<pre data-code-wrap="python"><code class="lang-python">in_features = resnet.fc.in_features
resnet.fc = nn.Linear(in_features, 10 )
</code></pre>
<p>Just make sure the target has the same shape as the model’s output (<code>[batch_size, 10]</code>) and contains the corresponding labels (<code>1.</code> for the active classes, <code>0</code> else).</p>
<p>PS: If you are not applying a sigmoid on the output, use <code>nn.BCEWithLogitsLoss</code>.</p>

username: pradpalnis
cooked: <p>output tensor label looks as below ,</p>
<p>tensor([[0., 0., 0., 0., 0., 0., 0., 1., 0., 0.],<br>
[0., 0., 0., 0., 0., 0., 1., 0., 0., 0.]]))</p>

username: ptrblck
cooked: <p>The target looks like you are dealing with a <em>multi-class</em> classification, not a <em>multi-label</em> one, i.e. each sample corresponds to a single class.<br>
If that’s the case, your target should only contain the current class index and should be a <code>LongTensor</code>.<br>
In case you already have the one-hot encoded targets, just call:</p>
<pre><code class="lang-python">target = torch.argmax(target, 1)
</code></pre>
<p>on them and use <code>nn.CrossEntropyLoss</code> (with logit outputs) as your criterion.</p>

username: pradpalnis
cooked: <p>I have confused you by sharing the above target output. I’m dealing with multi-label one. In my case, each sample corresponds to two class.</p>
<p>Input_image (say A) which contains has two objects ( say obj1,obj2).</p>
<p>Example :<br>
Input: image_a<br>
output :[ cat, dog]<br>
i.e., image_a pic has both cat and dog .</p>

username: ptrblck
cooked: <p>In that case the target should contain two active classes (e.g. assuming cat and dog are class0 and class4, respectively):</p>
<pre><code class="lang-python">tensor([[1., 0., 0., 0., 1., 0., 0., 0., 0., 0.]])
</code></pre>
<p>If that’s the case, just ignore my last post and refer to the one before it. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>