Multiple Output from MobileNet V3

ptinn: <p>Hi,</p>
<p>Following the <a href="https://discuss.pytorch.org/t/modify-resnet50-to-give-multiple-outputs/46905">approaches seen</a>, I tried to add a second output to mobilenet_v3 using the following:</p>
<pre><code class="lang-auto">class MBN_v3(torch.nn.Module):
    def __init__(self, num_classes1, num_classes2):
        super(MBN_v3, self).__init__()
        self.model_resnet = mobilenet_v3_small(pretrained=True)
        
        self.model_resnet.fc = torch.nn.Identity()
        self.classifier1 = torch.nn.Linear(1024, num_classes1)
        self.classifier1 = torch.nn.Linear(1024, num_classes2)

    def forward(self, x):
        x = self.model_resnet(x)
        out1 = self.classifier1(x)
        out2 = self.classifier2(x)
        
        return out1, out2

</code></pre>
<p>The self.classifer1 tagging seem to the one causing an error but I am not sure how to interpret the following output (with some omissions):</p>
<pre><code class="lang-auto"> ** On entry to SGEMM  parameter number 10 had an illegal value
Traceback ...
packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs)
 ... 
line 1051, in _call_impl return forward_call(*input, **kwargs) return F.linear(input, self.weight, self.bias)
... 
line 1847, in linear return torch._C._nn.linear(input, weight, bias)
RuntimeError: CUDA error: CUBLAS_STATUS_INVALID_VALUE when calling `cublasSgemm( handle, opa, opb, m, n, k, &amp;alpha, a, lda, b, ldb, &amp;beta, c, ldc)`
</code></pre>
<p>Any insight or suggestion would be highly appreciated!</p>
<p>P</p>

ParGG: <p>Hi P,</p>
<p>I replicated your code and I don’t have the same error. There are a few comments for you though:</p>
<ol>
<li>
<code>pretrain</code> is deprecated, you can use the definition in the code below;</li>
<li>
<code>classifier2</code> is not defined, you have a copy-paste error in the constructor;</li>
<li>the output of <code>model_resnet</code> has 1000 classes instead of 1024, maybe this is also the source of your error but I got a simple mismatch error instead.</li>
</ol>
<pre><code class="lang-python">In [1]: from torchvision.models import mobilenet_v3_small, MobileNet_V3_Small_Weights

In [2]: import torch

In [3]: class MBN_v3(torch.nn.Module):
   ...:     def __init__(self, num_classes1, num_classes2):
   ...:         super(MBN_v3, self).__init__()
   ...:         self.model_resnet = mobilenet_v3_small(weights=MobileNet_V3_Small_Weights.DEFAULT)
   ...: 
   ...:         self.model_resnet.fc = torch.nn.Identity()
   ...:         self.classifier1 = torch.nn.Linear(1000, num_classes1)
   ...:         self.classifier2 = torch.nn.Linear(1000, num_classes2)
   ...: 
   ...:     def forward(self, x):
   ...:         x = self.model_resnet(x)
   ...:         out1 = self.classifier1(x)
   ...:         out2 = self.classifier2(x)
   ...: 
   ...:         return out1, out2
   ...: 

In [4]: model = MBN_v3(10, 20)

In [5]: inp = torch.randint(0, 256, (1, 3, 256, 256), dtype= torch.float32)

In [6]: model(inp)
Out[6]: 
(tensor([[-3.8697, -0.3494,  0.3550, -0.3996,  0.1518,  1.0698, -1.8423,  2.0341,
          -2.8451, -3.5946]], grad_fn=&lt;AddmmBackward0&gt;),
 tensor([[ 2.7698,  0.6458,  1.0684, -0.1247,  1.7675, -1.9195,  0.7464, -2.3890,
          -0.9420, -0.1669,  2.5537, -0.3263, -1.5426,  1.0682,  0.8419, -1.6700,
           0.9832, -1.5306,  0.0394, -1.7743]], grad_fn=&lt;AddmmBackward0&gt;))
</code></pre>

ptinn: <p>Thank you Gian!</p>
<p>The class number of <code>1024</code> was causing the error indeed.  Do the modified classifiers actually become an additional layer added to the original classifier as printed below?   I want to make sure that I understand where the <code>1000</code> is coming from.</p>
<pre><code class="lang-auto">  (classifier): Sequential(
    (0): Linear(in_features=576, out_features=1024, bias=True)
    (1): Hardswish()
    (2): Dropout(p=0.2, inplace=True)
    (3): Linear(in_features=1024, out_features=1000, bias=True
</code></pre>
<p>Additionally, is the<code> MobileNet_V3_Small_Weights</code> version dependent?  It gives me and <code>ImportError</code>.</p>

ParGG: <aside class="quote no-group" data-username="ptinn" data-post="3" data-topic="156331">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptinn/48/45450_2.png" class="avatar"> ptinn:</div>
<blockquote>
<p>Do the modified classifiers actually become an additional layer added to the original classifier as printed below?</p>
</blockquote>
</aside>
<p>The 1000 comes from the last layer in the classifier, as you can see there are 1000 <code>out_features</code>. The 2 layers you add will basically be appended right below that <code>(3) Linear</code>, as two children.</p>
<aside class="quote no-group" data-username="ptinn" data-post="3" data-topic="156331">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptinn/48/45450_2.png" class="avatar"> ptinn:</div>
<blockquote>
<p>Additionally, is the<code> MobileNet_V3_Small_Weights</code> version dependent? It gives me and <code>ImportError</code>.</p>
</blockquote>
</aside>
<p>Yes, you should have v0.13 or later.</p>
<p>PS: In theory, you should replace the last Linear layer in the <code>model_resnet</code> with your two layers.</p>

ptinn: <p>Thanks for the clarification Gian.  Given the classifier has a sequence of 4 components, how does the indexing work to make the replacement work to insert two two layers?   It’s one part I still having trouble understanding.</p>

ParGG: <p>You can replace a layer like this:</p>
<pre><code class="lang-python">model.classifier[3] = NewLayer(...)
</code></pre>
<p>Whilst you can remove it like this according to the implementation:</p>
<pre><code class="lang-python">model.classifier.__delitem__(3)
</code></pre>
<p>Replacing won’t work in your case because you have two heads, what you can do is delete the last layers and then keep your code as it is. You just need to change from 1000 to 1024 input features.</p>