FasterRCNN with custom backbone architecture - Error in Anchor Generator object

v3nkyc0d3z: <p>I am trying to a FasterRCNN based object detection framework with a custom backbone. The backbone is the encoder architecture of Mix Vision Transformer architecture implemented in the Segmentation Models Pytorch library(<a href="https://github.com/qubvel/segmentation_models.pytorch" class="inline-onebox" rel="noopener nofollow ugc">GitHub - qubvel/segmentation_models.pytorch: Segmentation models with pretrained backbones. PyTorch.</a>). The encoder returns 4 features maps of sizes</p>
<pre><code class="lang-auto">torch.Size([1, 64, 320, 180])
torch.Size([1, 128, 160, 90])
torch.Size([1, 320, 80, 45])
torch.Size([1, 512, 40, 23])
</code></pre>
<p>I passed these features through <code>from torchvision.ops.feature_pyramid_network import FeaturePyramidNetwork</code> and transformed them into feature maps of sizes</p>
<pre><code class="lang-auto">torch.Size([1, 256, 320, 180])
torch.Size([1, 256, 160, 90])
torch.Size([1, 256, 80, 45])
torch.Size([1, 256, 40, 23])
</code></pre>
<p>I set the backbone.out_channels parameter as 256, and then used the following code to construct the FasterRCNN model</p>
<pre><code class="lang-auto">backbone_fpn = BackboneWithFPN(backbone, [64,128,320,512], 256)
backbone_fpn.out_channels = 256

anchor_generator = AnchorGenerator(sizes=((16, 32, 64, 128),),
                                   aspect_ratios=((0.5, 1.0, 2.0),))
roi_pooler = torchvision.ops.MultiScaleRoIAlign(featmap_names=['0','1','2','3'],
                                                output_size=7,
                                                sampling_ratio=2)
model = FasterRCNN(
    backbone=backbone_fpn,
    num_classes = 5,
    rpn_anchor_generator=anchor_generator,
    box_roi_pool=roi_pooler
)
</code></pre>
<p>The model object was constructed however, whenever I invoke a feed forward operation I get the following issue.</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "/data_fast/venkatesh/carla/custom_fasterRCNN.py", line 67, in &lt;module&gt;
    pred = model(list(image))
  File "/home/venkatesh/anaconda3/envs/cscapes/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/venkatesh/anaconda3/envs/cscapes/lib/python3.10/site-packages/torchvision/models/detection/generalized_rcnn.py", line 104, in forward
    proposals, proposal_losses = self.rpn(images, features, targets)
  File "/home/venkatesh/anaconda3/envs/cscapes/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/venkatesh/anaconda3/envs/cscapes/lib/python3.10/site-packages/torchvision/models/detection/rpn.py", line 361, in forward
    anchors = self.anchor_generator(images, features)
  File "/home/venkatesh/anaconda3/envs/cscapes/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/venkatesh/anaconda3/envs/cscapes/lib/python3.10/site-packages/torchvision/models/detection/anchor_utils.py", line 127, in forward
    anchors_over_all_feature_maps = self.grid_anchors(grid_sizes, strides)
  File "/home/venkatesh/anaconda3/envs/cscapes/lib/python3.10/site-packages/torchvision/models/detection/anchor_utils.py", line 88, in grid_anchors
    torch._assert(
  File "/home/venkatesh/anaconda3/envs/cscapes/lib/python3.10/site-packages/torch/__init__.py", line 827, in _assert
    assert condition, message
AssertionError: Anchors should be Tuple[Tuple[int]] because each feature map could potentially have different sizes and aspect ratios. There needs to be a match between the number of feature maps passed and the number of sizes / aspect ratios specified.
</code></pre>
<p>Tried to alter the <code>sizes</code> and the <code>aspect_ratio</code> parameters of <code>AnchorGenerator</code> object, nothing worked. Could someone please help me with this issue?</p>

mwcoleman: <p>Did you get this working?<br>
Potentially changing AnchorGenerator sizes to <code>((16,), (32,), (64,), (128,),)</code><br>
will fix the issue?</p>

v3nkyc0d3z: <p>It worked after I made that change! Thank you!</p>