Why I get a model negative loss returned

Harardin: <p>I’m quite new in python and pytorch.</p>
<p>I have perhaps simple problem.<br>
I get negative loss numbers returned.</p>
<p>This is the output:</p>
<pre><code class="lang-auto">Epoch [1/100], Step [10/63], Loss: 0.1118
Epoch [1/100], Step [20/63], Loss: -0.3783
Epoch [1/100], Step [30/63], Loss: -1.4037
Epoch [1/100], Step [40/63], Loss: -3.8306
Epoch [1/100], Step [50/63], Loss: -7.8438
Epoch [1/100], Step [60/63], Loss: -14.6053
Epoch [2/100], Step [10/63], Loss: -27.3804
Epoch [2/100], Step [20/63], Loss: -36.3176
Epoch [2/100], Step [30/63], Loss: -41.2981
etc...
</code></pre>
<p>My datareader from CSV file:</p>
<pre><code class="lang-auto">class CSV_Data(Dataset):
    def __init__(self, path):
        xy = np.loadtxt("data/"+path, delimiter=",", dtype=np.float32, skiprows=1, usecols=(0,1,2,3))
        self.x = torch.tensor(xy[:, 0:2], requires_grad=True)
        self.y = torch.from_numpy(xy[:, 2:])
        self.n_samples = xy.shape[0]
    
    def __getitem__(self, index):
        sample = self.x[index], self.y[index]
        return sample

    def __len__(self):
        # len(dataset)
        return self.n_samples
</code></pre>
<p>My model:</p>
<pre><code class="lang-auto">class RegressionReluNN(nn.Module):
    def __init__(self, input_size, hidden_size):
        super(RegressionReluNN, self).__init__()
        self.linear1 = nn.Linear(in_features=input_size, out_features=hidden_size)
        self.relu = nn.ELU()
        # Or maybe Relu ???
        self.linear2 = nn.Linear(in_features=hidden_size, out_features=2)

    def forward(self, x):
        out = self.linear1(x)
        out = self.relu(out)
        out = self.linear2(out)
        # sigmoid at the end ???
        y_predict = torch.sigmoid(out)
        return y_predict
</code></pre>
<p>Training loop:</p>
<pre><code class="lang-auto"># criterion = nn.CrossEntropyLoss()
criterion = nn.MultiLabelSoftMarginLoss()

n_total_steps = len(train_loader) # This is for test

for epoch in range(num_epochs):
    for i, (open_close, min_max) in enumerate(train_loader):
        # Forward
        outputs = model(open_close)
        # print(outputs)
        # sys.exit()
        # min_max_target = torch.argmax(min_max, 1)

        # print(min_max_target)
        # sys.exit()

        loss = criterion(outputs, min_max)

        # Backward
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        if (i+1) % 10 == 0:
            print(f'Epoch [{epoch+1}/{num_epochs}], Step [{i+1}/{n_total_steps}], Loss: {loss.item():.4f}')

</code></pre>
<p>Data I have for the input and target is:</p>
<pre><code class="lang-auto">input:
[[float32, float32]]
target:
[[float32, float32]]
</code></pre>
<p>I also tied to use CrossEntropuLoss but but got confused with conversion, the actual: <code>min_max_target = torch.argmax(min_max, 1)</code> didn’t get me any positive results.</p>

KFrank: <p>Hi Anton!</p>
<aside class="quote no-group" data-username="Harardin" data-post="1" data-topic="119294" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/harardin/48/39111_2.png" class="avatar"> Harardin:</div>
<blockquote>
<p>I get negative loss numbers returned.</p>
</blockquote>
</aside>
<p>You are probably passing <code>target</code> values that are greater than one into<br>
your <code>MultiLabelSoftMarginLoss</code> loss criterion.</p>
<aside class="quote no-group">
<blockquote>
<pre><code class="lang-auto">criterion = nn.MultiLabelSoftMarginLoss()
...
for epoch in range(num_epochs):
    for i, (open_close, min_max) in enumerate(train_loader):
        outputs = model(open_close)
        ...
        loss = criterion(outputs, min_max)
</code></pre>
</blockquote>
</aside>
<p><code>min_max</code> is read in from some unspecified data file, and then used as<br>
the <code>target</code> for <code>MultiLabelSoftMarginLoss</code>, which expects its <code>target</code><br>
to consist of values that are either <code>0.0</code> or <code>1.0</code>.  If you pass it values<br>
that are greater than <code>1.0</code>, it can return a negative loss.</p>
<p>Best.</p>
<p>K. Frank</p>

Harardin: <p>Thank you for your answer, it rally helped me to understand my mistake.</p>