Changing the backbone for FasterRCNN

kiostra: <p>Hello,<br>
I am new to pytorch and I wanted to try different backbones for FasterRCNN. I am following this tutorial: <a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html#defining-your-model" rel="nofollow noopener">https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html#defining-your-model</a><br>
This is my model</p>
<pre><code class="lang-auto">def my_model(num_classes):
    backboneV3= torchvision.models.inception_v3(pretrained=False)
    backboneV3_nofc  = nn.Sequential(*list(backboneV3.children())[:-1])
    backboneV3_nofc.out_channels = 192
    anchor_generator = AnchorGenerator(sizes=((32, 64, 128, 256, 512),),
                                       aspect_ratios=((0.5, 1.0, 2.0),))

    roi_pooler = torchvision.ops.MultiScaleRoIAlign(featmap_names=[0],
                                                    output_size=7,
                                                    sampling_ratio=2)
    modelV3 = FasterRCNN(backboneV3_nofc,
                       num_classes=num_classes,
                       rpn_anchor_generator=anchor_generator,
                       box_roi_pool=roi_pooler)
   
    return modelV3
</code></pre>
<p>I give the data during training exactly as in the tutorial but I always get this error:<br>
<code>RuntimeError: Expected 4-dimensional input for 4-dimensional weight 192 768, but got 2-dimensional input of size [2, 1000] instead</code><br>
What should I change in the model?</p>

mailcorahul: <p>You should ideally pass a tensor of shape <code>B x C x H x W</code>, B=batch_size, C=channel, H=height and W=width.<br>
Assuming the exception above is occurring at the input layer, it looks like you are passing a tensor of shape <code>2 x 1000</code>. Can you verify the shape of your input tensor?</p>

kiostra: <p>So if I do:</p>
<pre><code class="lang-auto">backboneV3= torchvision.models.inception_v3(pretrained=False)
backboneV3.eval()
x = torch.rand(1,3, 299, 299)
backboneV3(x)
</code></pre>
<p>this works fine. But if i try to slice the model in order to add the FasterRCNN it does not work anymore:</p>
<pre><code class="lang-auto">backboneV3_nofc  = nn.Sequential(*list(backboneV3.children())[:-1])
backboneV3_nofc.eval()
backboneV3_nofc(x)
</code></pre>
<p>I feel a bit lost…how should I slice the model?</p>

ptrblck: <p>In the tutorial, the backbone is created using <code>model.features</code>.<br>
This is unfortunately not possible in the current Inception implementation, since some functional calls are performed in the <code>forward</code> method as seen <a href="https://github.com/pytorch/vision/blob/07cbb46aba8569f0fac95667d57421391e6d36e9/torchvision/models/inception.py#L127">here</a>.<br>
These functional calls also make slicing the model and wrapping by <code>nn.Sequential</code> not possible.</p>
<p>A possible workaround would be to return the features, by replacing the last linear layer with <code>nn.Identity</code>:</p>
<pre><code class="lang-python">model = models.inception_v3(aux_logits=False)
model.fc = nn.Identity()
</code></pre>
<p>This would not slice the model in any way, keep the <code>forward</code> definition, and just return the penultimate activations.</p>
<p>Let me know, if this would work.</p>