BBox predicition occuring outside image/Bad BBox predictions

LakeOggy: <p>Hello</p>
<p>I’ve been trying to use a Faster RCNN in training a model on multispectral imagery (4 or more colour channels) for identifying vessels within the imagery. I’m currently utilizing a very slightly modified version of a ResNet50 FPN Model as a backbone (I used this particular <a href="https://www.kaggle.com/code/eclipser33/faster-rcnn-with-pytorch?scriptVersionId=40117114&amp;cellId=8" rel="noopener nofollow ugc">bit of code taken</a> from a kaggle notebook as a starting point). However when I’ve been trying to train the model, I’ve been having mixed results even after a high epoch</p>
<p>As an example of what my results look like (blue green red channels represented as red green blue if the colour seems off).</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/a/ca9d556c7733fbf5a4c29425ec39f446bf87730f.png" data-download-href="https://discuss.pytorch.org/uploads/default/ca9d556c7733fbf5a4c29425ec39f446bf87730f" title="Capture"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/a/ca9d556c7733fbf5a4c29425ec39f446bf87730f_2_690x336.png" alt="Capture" data-base62-sha1="sUpxa9peBemGEtu1HYqfPkiS979" width="690" height="336" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/a/ca9d556c7733fbf5a4c29425ec39f446bf87730f_2_690x336.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/a/ca9d556c7733fbf5a4c29425ec39f446bf87730f_2_1035x504.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/c/a/ca9d556c7733fbf5a4c29425ec39f446bf87730f_2_1380x672.png 2x" data-dominant-color="4F5251"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture</span><span class="informations">1557×759 94.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>With the numerical results for this particular result being (Trained on RTX 3090)</p>
<pre><code class="lang-auto">{'boxes': tensor([[0.0000e+00, 7.0580e-03, 4.5089e+00, 3.4324e+00],
        [3.6286e-01, 4.3109e-01, 9.7545e-02, 1.0113e-01],
        [3.8097e-01, 4.6612e-01, 9.9756e-02, 1.0369e-01],
        [3.9624e-01, 4.3783e-01, 9.7002e-02, 1.0255e-01],
        [6.0473e-01, 2.4947e-01, 1.5833e+00, 1.4847e+00],
        [0.0000e+00, 2.9449e-03, 6.8887e+00, 8.5387e-01],
        [8.9358e-01, 9.2008e-01, 4.1534e-01, 3.9130e-01],
        [3.4877e-01, 2.5463e+00, 1.6075e-01, 4.1155e-01]]),
 'labels': tensor([1, 1, 1, 1, 1, 1, 1, 1]), 
'scores': tensor([0.6883, 0.6401, 0.6320, 0.6276, 0.5587, 0.2847, 0.2674, 0.1132])}
</code></pre>
<p>Which, besides not really fiting the image, also seem to be placing boxes beyond the size of the image (which are 128 by 128 pixel images).</p>
<p>My gut is that I’ve done something wrong during the intialization of the FasterRCNN model and the anchors, but I’ve confused myself enough that I’m feeling out of my depth. The code where I create the model looks like:</p>
<pre><code class="lang-auto">
import torchvision
from torchvision.models.detection import FasterRCNN
from torchvision.models.detection.anchor_utils import AnchorGenerator
import ResModelsFPN

def create_model(bands, num_classes=2):
    backbone_v6_fpn = ResModelsFPN.resnet50_fpn_backbone(bands, num_classes)

    anchor_sizes = ((2,), (4,), (8,), (16,), (32,))
    anchor_generator_v4 = AnchorGenerator(sizes=anchor_sizes, aspect_ratios=((0.125, 0.25, 0.5, 1.0, 2.0),) * len(anchor_sizes))

    roi_pooler = torchvision.ops.MultiScaleRoIAlign(featmap_names=['0'], output_size=7, sampling_ratio=2)

    image_std = (0.3, 0.3, 0.3)
    image_mean = (0.4, 0.4, 0.4)

    model = FasterRCNN(backbone_v6_fpn, min_size=128, max_size=128, image_mean=image_mean, image_std=image_std,
                       num_classes=2, rpn_anchor_generator=anchor_generator_v4, box_roi_pool=roi_pooler)

    return model
</code></pre>
<p>Does anyone see anything obviously wrong with this before I start digging deeper/posting more sections of the code. I particularly feel that I may have screwed on on the anchor generation, as this is my first time tackling object detection with pytorch’s faster rcnn.</p>
<p>I appreciate any help anyone is able to give me.</p>