Modify Layers of Faster RCNN

draaken0: <p>M using the object detection model Faster RCNN Resnet 50 for some project. Now I want to add some additional layers to it.<br>
How can I add additional layers to he existing pre-trained object detection model ?</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from torchvision.ops.misc import FrozenBatchNorm2d
from torch.nn import Conv2d, Sequential, ReLU

class CustomNet(nn.Module):
    def __init__(self):
        super(CustomNet, self).__init__()

        self.l1 = Sequential(
            Conv2d(2048, 512, kernel_size=(1,1), bias = False, stride = (1,1)),
            FrozenBatchNorm2d(512),
            Conv2d(512, 512, kernel_size=(3,3), bias = False, stride = (1,1), padding=(1, 1)),
            FrozenBatchNorm2d(512),
            Conv2d(512, 2048, kernel_size=(1,1), bias = False, stride = (1,1)),
            FrozenBatchNorm2d(2048),
            ReLU(inplace=True) )

    def forward(self, x):
        x = self.l1(x)
        return x

</code></pre>
<p>I want to add the above layer generated to the main model.</p>
<pre><code class="lang-auto">main_model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)
</code></pre>
<p>How can I do that ?</p>

Dwight_Foster: <p>You could do this: Add a the model in <strong>init</strong> like this</p>
<pre><code class="lang-auto">self.main_model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)
</code></pre>
<p>and then in the forward pass you could do this</p>
<pre><code class="lang-auto">x = self.main_model(x)
x = self.l1(x)
</code></pre>

draaken0: <p>Thank you for the response <a class="mention" href="/u/dwight_foster">@Dwight_Foster</a> .<br>
I did that, now I realized that my whole question was wrong. I want to insert the layer in between the existing Sequential layers of FRCNN model.<br>
As FRCNN model have on 4 elements Rransform, Backbone, RPN and ROI and I want to add layers in backbone of the network.</p>

Dwight_Foster: <p>Ok for the backbone you could add a layer by using the .add_module function. For example if you wanted to add a convolution to layer 3 you could do this</p>
<pre><code class="lang-auto">main_model.backbone.body.layer3.add_module("6", nn.Sequential(nn.Conv2d(4,4,5,6)))
</code></pre>

draaken0: <p><a class="mention" href="/u/dwight_foster">@Dwight_Foster</a> I was able to add the layers as you mentioned an it works like a charm. But, I have another question what is Bottleneck in the code below:</p>
<pre><code class="lang-auto">main_model.backbone.body.layer1.add_module("3", nn.Sequential(nn.Conv2d(1024,256,kernel_size=(1,1),stride=(1,1),bias=False)))
</code></pre>
<p>After adding the layer an executing the code below</p>
<pre><code class="lang-auto">main_model.backbone.body.layer1
</code></pre>
<p>I get this output</p>
<pre><code class="lang-auto">Sequential(
  (0): Bottleneck(
    (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn1): FrozenBatchNorm2d(64)
    (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    (bn2): FrozenBatchNorm2d(64)
    (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn3): FrozenBatchNorm2d(256)
    (relu): ReLU(inplace=True)
    (downsample): Sequential(
      (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (1): FrozenBatchNorm2d(256)
    )
  )
  (1): Bottleneck(
    (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn1): FrozenBatchNorm2d(64)
    (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    (bn2): FrozenBatchNorm2d(64)
    (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn3): FrozenBatchNorm2d(256)
    (relu): ReLU(inplace=True)
  )
  (2): Bottleneck(
    (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn1): FrozenBatchNorm2d(64)
    (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    (bn2): FrozenBatchNorm2d(64)
    (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn3): FrozenBatchNorm2d(256)
    (relu): ReLU(inplace=True)
  )
  (3): Sequential(
    (0): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
  )
)
</code></pre>
<p>As you can see it is already a sequential layer and I dont know what is <strong>Bottleneck</strong> mean ?</p>

Dwight_Foster: <p>Glad that it worked. The Bottleneck is just a technique in resnet that they used to lower the number of parameters. A complete explanation can be found <a href="https://paperswithcode.com/method/bottleneck-residual-block#:~:text=create%20a%20bottleneck.-,The%20use%20of%20a%20bottleneck%20reduces%20the%20number%20of%20parameters,%2D50%20and%20ResNet%2D101." rel="noopener nofollow ugc">here</a>.</p>

sai_subramani: <p>hi <a class="mention" href="/u/draaken0">@draaken0</a>  i am trying to add a classification layer in between the pretrained model Faster RCNN Resnet 50. as i am not so expert in PyTorch. i cant able to find how to add the layer! can you able to help me?</p>

sai_subramani: <p>Hi <a class="mention" href="/u/dwight_foster">@Dwight_Foster</a> I am trying to add a Block of layer to Faster RCNN Resnet 50 pretrained model as the model is giving the output of prediction box and the object class, i want to add a parallel branch which consist of sequential layer followed by average pooling,convolution,average pooling and fully connected layer followed by soft max function to identify the object class is affected or not.</p>
<p>Hence the output of the pretrained model can give the object class,object coordinates and affected or not (classify) after adding the new  layer.</p>

Skylar_White: <p>do you get the solution for that? i have simmilar case with you when i want to add some layer for other classification task.</p>