How to implement weighted mean square error?

mderakhshani: <p>Hello guys, I would like to implement below loss function which is a weighted mean square loss function:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/1X/64fce39f8376e6f940cb95f96aa227a5fb5eb9ff.jpg" data-download-href="https://discuss.pytorch.org/uploads/default/64fce39f8376e6f940cb95f96aa227a5fb5eb9ff" title="photo_۲۰۱۷-۰۵-۰۱_۱۳-۵۳-۲۲.jpg"><img src="https://discuss.pytorch.org/uploads/default/optimized/1X/64fce39f8376e6f940cb95f96aa227a5fb5eb9ff_2_690x397.jpg" width="690" height="397" srcset="https://discuss.pytorch.org/uploads/default/optimized/1X/64fce39f8376e6f940cb95f96aa227a5fb5eb9ff_2_690x397.jpg, https://discuss.pytorch.org/uploads/default/original/1X/64fce39f8376e6f940cb95f96aa227a5fb5eb9ff.jpg 1.5x, https://discuss.pytorch.org/uploads/default/original/1X/64fce39f8376e6f940cb95f96aa227a5fb5eb9ff.jpg 2x" data-dominant-color="F7F8F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">photo_۲۰۱۷-۰۵-۰۱_۱۳-۵۳-۲۲.jpg</span><span class="informations">774×446 25.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
How can I implement such a lost function in pytorch? In another words, Is there any way to use <code>nn.MSELoss</code> to achieve to my mentioned loss function?</p>

fmassa: <p>You can probably use a combination of tensor operations to compute your loss.<br>
For example</p>
<pre><code class="lang-python">def mse_loss(input, target):
    return torch.sum((input - target) ** 2)

def weighted_mse_loss(input, target, weight):
    return torch.sum(weight * (input - target) ** 2)
</code></pre>

mderakhshani: <p>What I have understood from your above snippet is to create a <code>nn.Module</code> and write your above code in its forward function. Am I right? Because I would like to use <code>autograd</code> for <strong>backprop</strong>!</p>

fmassa: <p>You don’t need to write a <code>nn.Module</code> for that, a simple function is enough.<br>
Note that backpropagation is handled by <code>Variables</code>, and <strong>not</strong> by <code>nn.Module</code>.</p>
<p>So this is perfectly fine</p>
<pre><code class="lang-python">def pow(input, n):
    return input ** n

t = Variable(torch.rand(1), requires_grad=True)
t2 = pow(input, 2)
t6 = pow(t2, 3)
t6.backward()
</code></pre>

mderakhshani: <p>Thank you. <img src="//discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:">  I appreciate your response!</p>

whamp: <p>Were you able to solve this? When i try to set this function as my criterion i get an error saying the function requires the 3 inputs which obviously aren’t calculated yet until training. Maybe this is why it needs to go in nn.module ?</p>

whamp: <p>This is the ugly hack i created that works for this problem. 16 outputs with the first output being weighted 8/16ths and the remaining outputs weighted 0.5/15.</p>
<p>I’m sure there’s a better way to do this but if you’re in a hurry this works.</p>
<pre><code class="lang-auto">def weighted_mse_loss(input,target):
    #alpha of 0.5 means half weight goes to first, remaining half split by remaining 15
    weights = Variable(torch.Tensor([0.5,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15,0.5/15])).cuda()  
    pct_var = (input-target)**2
    out = pct_var * weights.expand_as(target)
    loss = out.mean() 
    return loss
</code></pre>

Danial_Maleki: <p>your question is somehow irrelevant.</p>

Naman-ntc: <p>Hi,<br>
It’s not a big deal but why isn’t weighting provided in default implementation since all (atleast most) other loss functions have it?</p>

Fang_Wu: <p>Actually,</p>
<pre><code class="lang-auto">def weighted_mse_loss(input, target, weight):
        return (weight * (input - target) ** 2).mean()
</code></pre>
<p>is slightly better, since the sum is some big. Taking the average is exactly the original way that nn.MSELoss does.</p>

David_Wong: <p>I think it is better divided by the sum of weight instead of taking average cause it is how the weighted cross entropy loss implemented</p>
<pre><code class="lang-auto">def weighted_mse_loss(input, target, weight):
        return (weight * (input - target) ** 2).sum() / weight.sum()
</code></pre>