nn.MSELoss() produces different output compared to a mse_loss function

Pranavan_Theivendira: <p>Hi All,</p>
<p>I am getting the following issue. I am using nn.MSELoss() to calculate the loss as well as another function 2 calculate the loss. When I just check with one sample, it produces two different results. What could be the problem? (I commented the place where I have issue)</p>
<pre><code class="lang-auto">from networks.auto_encoder import AutoEncoder
from layers.encoder import Encoder
from networks.decoder import Decoder
import torch
from dataset.DamadicEvaluatorDataset import DamadicEvaluatorDataset
import torch.nn as nn
from torch.utils.data import DataLoader
from math import sqrt
import matplotlib.pyplot as plt
import torch
from torch.autograd import Variable

def mse_loss(inputs, target):
    return torch.sum((inputs - target) ** 2)

def get_loss_list(d_p, e_p):
    ae_path = 'models/auto-encoder/auto_encoder_f.h5'
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    encoder = Encoder()
    decoder = Decoder()
    model = AutoEncoder(encoder=encoder, decoder=decoder)
    model.load_state_dict(torch.load(ae_path, map_location=torch.device('cpu')))
    model = model.to(device)
    model.eval()
    
    
    batch_size = 128
    ds = DamadicEvaluatorDataset(path_list_scaler=d_p, path_list=e_p)
    data_loader = DataLoader(ds, shuffle=True, batch_size=batch_size)

    criterion = nn.MSELoss()
    loss_t = []
    outs = []

    for i, data in enumerate(data_loader):
        out = model(data.float())
        for index in range(out.size(0)):
            loss = mse_loss(out[index], data[index].float())
            loss_1 = criterion(out[index], data[index].float())
            print(loss, loss_1.item()) # these 2 should be same, but I am getting different values
            loss_t.append(loss)
            outs.append(out[index].squeeze())
    
    return loss_t, outs
</code></pre>
<p>Thanks</p>

Nikronic: <p>Hi,</p>
<p>I think the issue is that you are using <code>sum</code> to reduce the MSE loss.</p>
<aside class="quote no-group" data-username="Pranavan_Theivendira" data-post="1" data-topic="88349">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pranavan_theivendira/48/12366_2.png" class="avatar"> Pranavan_Theivendira:</div>
<blockquote>
<p><code>return torch.sum((inputs - target) ** 2)</code></p>
</blockquote>
</aside>
<p>In <a href="https://pytorch.org/docs/master/generated/torch.nn.MSELoss.html" rel="noopener nofollow ugc">nn.MSELoss</a> (please see the linked documentation), you can specify an argument <code>reduction</code> which by default is <code>mean</code>. So, you may need to explicitly state you want <code>reduction='sum'</code>.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/2/c/2c86e76862e50f308ae9f1d02a7e96ace7a69a58.png" alt="image" data-base62-sha1="6lU2q6Um4j6o2gO5WYDVSHwIorK" width="545" height="183"></p>
<pre><code class="lang-auto">criterion = nn.MSELoss(reduction='sum')
</code></pre>
<p>Bests</p>

Pranavan_Theivendira: <p>Yay. Thanks for your time. That is correct. I fixed the issues.</p>