KEYPOINT Detection issues

Petrea_Tancau: <p>Hi! I have a big issue in training a Keypoint R-CNN for detection of keypoints.<br>
My dataset is composed of pictures with faces with boxes on eyes and as a keypoint, one per each box, the center of the eye.</p>
<p>the model definition is the one from the torchvision framework:<br>
model = torchvision.models.detection.keypointrcnn_resnet50_fpn(pretrained=False, progress=True, num_classes=2, num_keypoints=1, pretrained_backbone=True)</p>
<p>I am training the network with the coco method:</p>
<h1><a name="construct-an-optimizer-1" class="anchor" href="#construct-an-optimizer-1"></a>construct an optimizer</h1>
<p>params = [p for p in model.parameters() if p.requires_grad]<br>
optimizer = torch.optim.SGD(params, lr=0.0001,<br>
momentum=0.9, weight_decay=0.0005)</p>
<h1><a name="and-a-learning-rate-scheduler-which-decreases-the-learning-rate-by-2" class="anchor" href="#and-a-learning-rate-scheduler-which-decreases-the-learning-rate-by-2"></a>and a learning rate scheduler which decreases the learning rate by</h1>
<h1><a name="h-10x-every-3-epochs-3" class="anchor" href="#h-10x-every-3-epochs-3"></a>10x every 3 epochs</h1>
<p>lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer,<br>
step_size=3,<br>
gamma=0.1)</p>
<p>num_epochs = 7</p>
<h1><a name="h-4" class="anchor" href="#h-4"></a></h1>
<p>for epoch in range(num_epochs):<br>
# train for one epoch, printing every 10 iterations<br>
train_one_epoch(model, optimizer, data_loader, device, epoch, print_freq=30)<br>
lr_scheduler.step()</p>
<p>Now, this is the issue I got every time, I can’t even train for one epoch.<br>
I have already checked that my data is correctly passed and the implementation of the data loader is like it is described in the documentation of torchvision.</p>
<p>The following image is showing the exception:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/5/512fec4420b27dbacfae05c5bbbc96c84ccf4548.png" data-download-href="https://discuss.pytorch.org/uploads/default/512fec4420b27dbacfae05c5bbbc96c84ccf4548" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/5/512fec4420b27dbacfae05c5bbbc96c84ccf4548_2_690x293.png" alt="image" data-base62-sha1="bAdlFodq5gCpAxa66GTeS1scQUE" width="690" height="293" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/5/512fec4420b27dbacfae05c5bbbc96c84ccf4548_2_690x293.png, https://discuss.pytorch.org/uploads/default/optimized/2X/5/512fec4420b27dbacfae05c5bbbc96c84ccf4548_2_1035x439.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/5/512fec4420b27dbacfae05c5bbbc96c84ccf4548_2_1380x586.png 2x" data-dominant-color="F4F7F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1614×687 83.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Does someone already encountered this problem? I thank you very much for helping me.</p>

radeeplearning: <p>I have worked on keypoint detection before, perhaps you could try the following solution to ensure the number of keypoints are changed from 17 to the desired keypoints</p>
<pre><code class="lang-auto">model = torchvision.models.detection.keypointrcnn_resnet50_fpn(pretrained=False, progress=True, num_classes=2, pretrained_backbone=True)

model.roi_heads.keypoint_predictor = models.detection.keypoint_rcnn.KeypointRCNNPredictor(512, 2)

</code></pre>
<p>In my project I required the keypoint detected to be 26 as opposed to the default 17 so I changed the keypoint_predictor output in the roi_head directly.</p>
<p>Hope this helps</p>

Petrea_Tancau: <p>Thank you! I have already tried also this kind of solution but nothing changed so…I decided to write my own training procedure, for this task, based on MSELoss and used as a predictor a Resnext50 with the Linear layer that gives the desired output.</p>

radeeplearning: <p>Ha alright, glad to hear you found a solution</p>