Why does PyTorch Object Detection model NOT learn anything?

username: Laya1
cooked: <p>I have an object detection task with one class of objects (plus the background). I am training a Faster RCNN model and a Retinanet model for the task. The <strong>loss during training does not decrease at all</strong>. At the end of training, the Faster RCNN predicts about 100 bounding boxes, all in the top left corner and overlapping with each other. The Retinanet model produces empty bounding boxes.</p>
<p>Given the non-decreasing loss at the time of training, it is expected that the models will not perform well. But I cannot see a reason <strong>why the models do not learn anything at all</strong>. Here are relevant parts of the code:<br>
<strong>Models:</strong></p>
<pre><code class="lang-auto">model_retinanet = torchvision.models.detection.retinanet_resnet50_fpn(pretrained=False, pretrained_backbone=True, num_classes=2)
model_retinanet.transform = torchvision.models.detection.transform.GeneralizedRCNNTransform(min_size=3040,
                                                                                  max_size=4048,
                                                                                  image_mean=[0.485, 0.456, 0.406],
                                                                                  image_std=[0.229, 0.224, 0.225])

model_fasterrcnn = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)
in_features = model_fasterrcnn.roi_heads.box_predictor.cls_score.in_features
model_fasterrcnn.roi_heads.box_predictor = FastRCNNPredictor(in_features, 2)
model_fasterrcnn.transform = torchvision.models.detection.transform.GeneralizedRCNNTransform(min_size=3040,
                                                                                  max_size=4048,
                                                                                  image_mean=[0.485, 0.456, 0.406],
                                                                                  image_std=[0.229, 0.224, 0.225])
</code></pre>
<p><strong>Training:</strong></p>
<pre><code class="lang-auto">for epoch in range(num_epochs):
        train_one_epoch(model, optimizer, data_loader_train, torch.device('cpu'), epoch, print_freq=10)
</code></pre>
<p>Here <code>model</code> is either <code>model_retinanet</code> or <code>model_fasterrcnn</code>. The function <code>train_one_epoch</code> is the one used in the PyTorch example on <a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html" rel="noopener nofollow ugc">fine tuning object detection model</a>.</p>
<p>PS: <em>I have tried the same learning exercise on MATLAB, and it seems to work. However, MATLAB does not offer me as much flexibility as PyTorch, so I am trying to get the model to learn in PyTorch.</em></p>

username: Dicko
cooked: <p>Hello, did you manage to resolve this issue? I am facing similar problems. Any insights would be much appreciated. Thank you.</p>

username: Laya1
cooked: <p>Yes. I realised that I was doing a custom transform operation, which had a logical mistake. I was resizing the image, but there was a problem with the implementation of resizing of bounding boxes.</p>