Why are my predictions so bad for the fasterrcnn_resnet50_fpn

username: StrafeNDestroy
cooked: <p>Hello, I am using the fastercnn resnet trying to draw some bounding boxes around raccoons. I was able to get the training loop done and also draw the predicted box on some images in the eval state of my saved model. The predictions are pretty bad and I was wondering what I could do to make them better. My training data size is 127 images of raccoons. These are the images with correct boxes taken from the data loader.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/7/a7bc1df53ae62e30da8f2a472ff7f6f39fb29d7a.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/a7bc1df53ae62e30da8f2a472ff7f6f39fb29d7a" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/7/a7bc1df53ae62e30da8f2a472ff7f6f39fb29d7a_2_690x75.jpeg" alt="image" data-base62-sha1="nVQM8ABOzaNDahClztkCrB4pBq2" width="690" height="75" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/7/a7bc1df53ae62e30da8f2a472ff7f6f39fb29d7a_2_690x75.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/a/7/a7bc1df53ae62e30da8f2a472ff7f6f39fb29d7a_2_1035x112.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/a/7/a7bc1df53ae62e30da8f2a472ff7f6f39fb29d7a_2_1380x150.jpeg 2x" data-dominant-color="9E9C8E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1599×176 103 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<hr>
<pre data-code-wrap="python"><code class="lang-python">EPOCHS = 10
BATCH_SIZE = 2

model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True) # loading in model as pretrain, IE defualt weights
N_CLASS = 2 #Number of classes we want our model to predict, our case 3 but the background is a class so +1
INP_FEATURES = model.roi_heads.box_predictor.cls_score.in_features
model.roi_heads.box_predictor = FastRCNNPredictor(INP_FEATURES, N_CLASS)

model.to(device)
params = [p for p in model.parameters() if p.requires_grad]
optimizer = torch.optim.Adam(params) # how our should compute parameters and also how our steps are calculated 
lr_scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer) # dynamically changes the learning rate

# Training Loop
lossHist = LossAverager()
valLossHist = LossAverager()
column_names = ["Epoch", "Train_loss", "Valid_loss", "Error_rate", "Duration"]
df = pd.DataFrame(columns=column_names)
for epoch in tqdm(range(1, EPOCHS)):

    start_time = time()
    model.train()  # setting out model to train
    lossHist.reset()  # resets our values,averages,sums,counts in lossHist
    for images, targets in dl_test:
        images = torch.stack(images).to(device)
        targets = [{k: v.to(device) for k, v in t.items()} for t in targets]  # sending targets to the GPU  
        bs = images.shape[0]
        loss_dict = model(images, targets)  # passing our model a single batch of images with repective targets       
        totalLoss = sum(loss for loss in loss_dict.values())
        lossValue = totalLoss.item()
        lossHist.update(lossValue, bs)
        optimizer.zero_grad()
        totalLoss.backward()
        optimizer.step()
    if lr_scheduler is not None:
        lr_scheduler.step(totalLoss)

    df = df.append({"Epoch": epoch, "Train_loss": lossHist.avg,
                    "Duration": str(datetime.timedelta(seconds=time() - start_time))[2:7],
                    "Valid_loss": valLossHist.avg}, ignore_index=True)
torch.save(model.state_dict(), r'model_weights.pth')
df
</code></pre>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/b/1b2f87184795c1f5b3968403599ad06ca78e5379.png" alt="image" data-base62-sha1="3SuIBdGvHhVD1hxQoCU92aphrdv" width="434" height="303"></p>
<h1><a name="loading-saved-model-to-inference-on-1" class="anchor" href="#loading-saved-model-to-inference-on-1"></a>Loading saved model to inference on</h1>
<pre data-code-wrap="python"><code class="lang-python">model = torchvision.models.detection.fasterrcnn_resnet50_fpn()
N_CLASS = 2 
INP_FEATURES = model.roi_heads.box_predictor.cls_score.in_features
model.roi_heads.box_predictor = FastRCNNPredictor(INP_FEATURES, N_CLASS)
model.load_state_dict(torch.load("model_weights.pth"))
model.eval()
output = model(image)
pprint.pprint(output)
coords = output[0]["boxes"][0].tolist()
# BAD RESULTS
[{'boxes': tensor([[ 133.0887,  325.2898,  977.9802, 1024.0000],
        [ 133.7064,    0.0000,  978.3478,  780.0204],
        [   0.0000,  323.3601,  587.6085, 1024.0000],
        [   0.0000,    0.0000,  791.2852,  518.3591],
        [ 290.4432,   74.9510,  791.7764, 1006.7882],
        [   0.0000,  167.1415,  632.4937,  756.1417],
        [ 537.8644,  261.5962, 1024.0000, 1024.0000]],
       grad_fn=&lt;StackBackward0&gt;),
  'labels': tensor([1, 1, 1, 1, 1, 1, 1]),
  'scores': tensor([0.5783, 0.5775, 0.5772, 0.5772, 0.5763, 0.5753, 0.5752],
       grad_fn=&lt;IndexBackward0&gt;)}]
</code></pre>