Custom Loss Function - Heatmaps

jovanv95: <p>Hey, I am doing keypoint detecion with heatmaps. Where i have x,y cordinates for each image and i made heatmaps for them. I want to use a MSE loss function, but i cant use it on heatmaps alone since the numbers are very low(-1,1). I wrote a custom loss function that gets the cordinates from each of the heatmpas and then i calculate  the mse with the x and y cordinates. But i get this error:</p>
<p>RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn.</p>
<p>Here is my loss function:</p>
<p>class HourGlassLoss(torch.nn.Module):<br>
def <strong>init</strong>(self,keypoint_number,batch_size):<br>
super(HourGlassLoss, self).<strong>init</strong>()<br>
self.keypoint_number = keypoint_number<br>
self.batch_size = batch_size</p>
<pre><code>def forward(self, output, target):
    predicted = output.clone().detach().requires_grad_(True)
    true = target.clone().detach().requires_grad_(True)
    _, max_indices = torch.max(predicted.view(predicted.size(0), predicted.size(1), -1), dim=2)
    keypoints = torch.stack([max_indices // predicted.size(3), max_indices % predicted.size(3)], dim=2).float()
    keypoints = keypoints[0]
    predicted = keypoints.unsqueeze(0)


    _, max_indices = torch.max(true.view(true.size(0), true.size(1), -1), dim=2)
    keypoints = torch.stack([max_indices // true.size(3), max_indices % true.size(3)], dim=2).float()
    keypoints = keypoints[0]
    true = keypoints.unsqueeze(0)

    loss = torch.mean((predicted - true)**2)

    return loss
</code></pre>

ptrblck: <p>You are explicitly detaching the <code>ouput</code> tensor in:</p>
<pre><code class="lang-python">predicted = output.clone().detach().requires_grad_(True)
</code></pre>
<p>which will cut the computation graph.<br>
Could you explain why you are doing this?</p>

jovanv95: <p>Thanks for replying.  I tried something like this:</p>
<pre><code>def forward(self, predicted, true):
    print(predicted.is_leaf)
    print(true.is_leaf)
    _, max_indices = torch.max(predicted.view(predicted.size(0), predicted.size(1), -1), dim=2)
    keypoints = torch.stack([max_indices // predicted.size(3), max_indices % predicted.size(3)], dim=2).float()
    keypoints = keypoints[0]
    predicted = keypoints.unsqueeze(0)


    _, max_indices = torch.max(true.view(true.size(0), true.size(1), -1), dim=2)
    keypoints = torch.stack([max_indices // true.size(3), max_indices % true.size(3)], dim=2).float()
    keypoints = keypoints[0]
    true = keypoints.unsqueeze(0)

    loss = torch.mean((predicted - true)**2)

    return loss
</code></pre>
<p>When i check to see if the variables are leafs, it says that the model output variable(predicted) is not a leaf. But when i use any other built in pyotrch loss function i dont have the same problem, even though it also says that the output is not a leaf.</p>
<pre><code>def forward(self, predicted, true):

    loss = nn.MSELoss()
    loss = loss(predicted, true)

    return loss
</code></pre>
<p>This works but as said since i am comparing two heatmaps the loss function is really low after the first epoch(0.048).</p>

ptrblck: <p><code>predicted</code> is not supposed to be a leaf tensor since it’s created as the model output, isn’t it?</p>