Regression with Multiple Outputs

vtandra: <p>I have a multiple input and multiple output (MIMO) regression problem. When I use the MSE loss function I see only one MSE. How is Pytorch calculating it ? Does it take the mean of MSE of all the outputs ?</p>

ptrblck: <p>I don’t know how you are passing multiple outputs to <code>nn.MSELoss</code> as only one output tensor and target tensor are expected:</p>
<pre><code class="lang-python">output1 = torch.randn(10, 10, requires_grad=True)
output2 = torch.randn(10, 10, requires_grad=True)
target = torch.randn(10, 10)

# works
loss = criterion(output1, target)

# fails
loss = criterion(output1, output1, target)
# TypeError: forward() takes 3 positional arguments but 4 were given
</code></pre>
<p>Could you share your code showing how the loss is calculated?</p>

vtandra: <p>Here you go :</p>
<h1>
<a name="loss-and-optimizer-1" class="anchor" href="#loss-and-optimizer-1"></a>Loss and optimizer</h1>
<p>criterion = nn.MSELoss()</p>
<p>pred_output1= torch.tensor([[1.0003, 0.9998, 0.9928],<br>
[0.9962, 0.9955, 0.9874],<br>
[0.9944, 0.9940, 0.9851],<br>
[0.9315, 0.9306, 0.9251],<br>
[0.9407, 0.9401, 0.9375],<br>
[0.9877, 0.9868, 0.9778]])<br>
#<span class="hashtag">#torch</span>.Size([6, 3]) —three outputs</p>
<p>target= torch.tensor([[1.0000, 1.0000, 1.0000],<br>
[1.0000, 1.0000, 1.0000],<br>
[1.0000, 1.0000, 1.0000],<br>
[0.9400, 0.9500, 0.9500],<br>
[0.9500, 0.9500, 0.9500],<br>
[0.9800, 0.9800, 0.9800]])</p>
<p>loss = criterion(pred_output1, target)</p>
<p>“print(loss) = tensor(0.0001)”</p>

ptrblck: <p>Thanks for the code. The loss will be computes elementwise as seen here:</p>
<pre><code class="lang-python">criterion = nn.MSELoss()
pred_output1 = torch.randn(6, 3)
target = torch.randn(6, 3)

loss = criterion(pred_output1, target)
print(loss) 
# tensor(1.5746)

loss_manual = ((pred_output1 - target)**2).mean()
print(loss_manual)
# tensor(1.5746)
</code></pre>
<p>By default the <code>mean</code> reduction will be used.</p>

NumericLee: <p>My output vector is 1x8 ,  the 8 elements differ greatly in distribution.</p>
<p>nn.L1Loss(reduction=None) returns a scalar. Is there a way to get back a vector of 8 L1 losses without writing a custom function?</p>

ptrblck: <aside class="quote no-group" data-username="NumericLee" data-post="5" data-topic="175428">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/n/ecccb3/48.png" class="avatar"> NumericLee:</div>
<blockquote>
<p>nn.L1Loss(reduction=None) returns a scalar.</p>
</blockquote>
</aside>
<p>That’s not the case for me:</p>
<pre data-code-wrap="python"><code class="lang-python">output = torch.randn(1, 8, requires_grad=True)
target = torch.randn(1, 8)

criterion = nn.L1Loss(reduction="none")

loss = criterion(output, target)
print(loss)
# tensor([[0.2418, 0.8901, 0.0970, 0.5503, 0.2731, 0.3836, 1.0063, 0.5069]],
#        grad_fn=&lt;AbsBackward0&gt;)
</code></pre>

NumericLee: <aside class="quote no-group" data-username="ptrblck" data-post="6" data-topic="175428">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>nn.L1Loss(reduction="none")</code></p>
</blockquote>
</aside>
<p>There are “none” wiser than you. Thanks</p>