Negative NLLLoss

hs99: <p>When I use NLLLoss as criterion for my CNN Model I get negative loss as shown :<br>
Is it fine?Does it say something of model performance??</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/0/f/0f3b10f978cac65cd5431f0b391697950cbd184b.png" alt="image" data-base62-sha1="2aJI5dz6TaxFK22peTwbw0nY94v" width="660" height="188"></p>
<p>Is that possible ??Also when I use CrossEntropyLoss I get  +ve values like:<br>
Epoch: 1 	Training Loss: 0.985429 	Validation Loss: 0.980953<br>
Validation loss decreased (inf → 0.980953).  Saving model …<br>
Epoch: 2 	Training Loss: 0.979373 	Validation Loss: 0.978553<br>
Validation loss decreased (0.980953 → 0.978553).  Saving model …<br>
Epoch: 3 	Training Loss: 0.973842 	Validation Loss: 0.977175<br>
Validation loss decreased (0.978553 → 0.977175).  Saving model …<br>
Epoch: 4 	Training Loss: 0.967670 	Validation Loss: 0.973433<br>
Validation loss decreased (0.977175 → 0.973433).  Saving model …<br>
Epoch: 5 	Training Loss: 0.963760 	Validation Loss: 0.971874<br>
Validation loss decreased (0.973433 → 0.971874).  Saving model …</p>
<p>Also…Can u plz help me by telling whether these values of losses indicate how  well is my model…Like I now it depends on other parameters also…but just forloss how good the model is<br>
I am pretty new to all this stuff…Plz help!!</p>

KFrank: <p>Hi hs!</p>
<aside class="quote no-group" data-username="hs99" data-post="1" data-topic="116003" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hs99/48/36177_2.png" class="avatar"> hs99:</div>
<blockquote>
<p>When I use NLLLoss as criterion for my CNN Model I get negative loss as shown :</p>
</blockquote>
</aside>
<p><code>NLLLoss</code> requires its <code>input</code> to be log-probabilities.  To be valid<br>
log-probabilities, these values must be non-positive.  You are passing<br>
<code>NLLLoss</code> positive <code>input</code> values so it is returning negative (and<br>
meaningless) values for the loss.</p>
<p>If you pass the output of your final <code>Linear</code> layer through a <code>LogSoftmax</code><br>
layer, the <em>logits</em> produced by the <code>Linear</code> layer (which can be positive)<br>
will be converted to valid (and non-positive) log-probabilities.</p>
<p>Best.</p>
<p>K. Frank</p>

hs99: <p><a class="mention" href="/u/kfrank">@KFrank</a></p>
<p>I have done that as well.Here is my model code</p>
<p>model_1 = CNN(num_classes=5, hid_size=128).to(device)</p>
<p>The model code is:</p>
<p>class ConvNormPool(nn.Module):<br>
“”“Conv Skip-connection module”“”<br>
def <strong>init</strong>(<br>
self,<br>
input_size,<br>
hidden_size,<br>
kernel_size,<br>
norm_type=‘bachnorm’<br>
):<br>
super().<strong>init</strong>()</p>
<pre><code>    self.kernel_size = kernel_size
    self.conv_1 = nn.Conv1d(
        in_channels=input_size,
        out_channels=hidden_size,
        kernel_size=kernel_size
    )
    self.conv_2 = nn.Conv1d(
        in_channels=hidden_size,
        out_channels=hidden_size,
        kernel_size=kernel_size
    )
    self.conv_3 = nn.Conv1d(
        in_channels=hidden_size,
        out_channels=hidden_size,
        kernel_size=kernel_size
    )
    self.swish_1 = Swish()
    self.swish_2 = Swish()
    self.swish_3 = Swish()
    if norm_type == 'group':
        self.normalization_1 = nn.GroupNorm(
            num_groups=8,
            num_channels=hidden_size
        )
        self.normalization_2 = nn.GroupNorm(
            num_groups=8,
            num_channels=hidden_size
        )
        self.normalization_3 = nn.GroupNorm(
            num_groups=8,
            num_channels=hidden_size
        )
    else:
        self.normalization_1 = nn.BatchNorm1d(num_features=hidden_size)
        self.normalization_2 = nn.BatchNorm1d(num_features=hidden_size)
        self.normalization_3 = nn.BatchNorm1d(num_features=hidden_size)
        
    self.pool = nn.MaxPool1d(kernel_size=2)
    
def forward(self, input):
    #print("INPUT SHAPE SKIP")
    #print(input.shape)
    conv1 = self.conv_1(input)
    x = self.normalization_1(conv1)
    x = self.swish_1(x)
    x = F.pad(x, pad=(self.kernel_size - 1, 0))
    
    x = self.conv_2(x)
    x = self.normalization_2(x)
    x = self.swish_2(x)
    x = F.pad(x, pad=(self.kernel_size - 1, 0))
    
    conv3 = self.conv_3(x)
    x = self.normalization_3(conv1+conv3)
    x = self.swish_3(x)
    x = F.pad(x, pad=(self.kernel_size - 1, 0))   
    
    x = self.pool(x)
    return x
</code></pre>
<hr>
<p>class CNN(nn.Module):<br>
def <strong>init</strong>(<br>
self,<br>
input_size = 1,<br>
hid_size = 256,<br>
kernel_size = 5,<br>
num_classes = 5,<br>
):</p>
<pre><code>    super().__init__()
    
    self.conv1 = ConvNormPool(
        input_size=input_size,
        hidden_size=hid_size,
        kernel_size=kernel_size,
    )
    self.conv2 = ConvNormPool(
        input_size=hid_size,
        hidden_size=hid_size//2,
        kernel_size=kernel_size,
    )
    self.conv3 = ConvNormPool(
        input_size=hid_size//2,
        hidden_size=hid_size//4,
        kernel_size=kernel_size,
    )
    self.avgpool = nn.AdaptiveAvgPool1d((1))
    self.fc = nn.Linear(in_features=hid_size//4, out_features=num_classes)
    
def forward(self, input):
    #print("INPUT SHAPE is ")
    #print(input.shape)
    #print("INPUT SHAPE is ")
    #print(input.shape)
    #input = input.unsqueeze(1)
    #input = input.unsqueeze(0)
    x = self.conv1(input)
    x = self.conv2(x)
    x = self.conv3(x)
    x = self.avgpool(x)        
    # print(x.shape) # num_features * num_channels
    x = x.view(-1, x.size(1) * x.size(2))
    **x = F.softmax(self.fc(x), dim=1)**
    return x
</code></pre>
<p>According to your suggestion, I did thechange in the last part of CNN class …I changed x = F.softmax(self.fc(x), dim=1) to<br>
x = F.log_softmax(self.fc(x), dim=1)<br>
Now I get the NLLLloss Values as:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/4/4/441253b41635a23e4b48c0e8dde9a929d45e36eb.png" alt="image" data-base62-sha1="9IbJsWnPN9Al9yf2LEHPFcKhVeb" width="647" height="72"></p>
<p>Will this Log_softmax layer also work proper for CrossEntropyLoss or I need to change it to simple softmax only??</p>
<p>Kindly pls.tell me if I am wrong…And also where and what code to insert for correction…I am pretty new to pytorch So pls help!!!<br>
I would be very grateful <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=12" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:" loading="lazy" width="20" height="20"><img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=12" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:" loading="lazy" width="20" height="20"><img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=12" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:" loading="lazy" width="20" height="20"></p>

hs99: <aside class="quote no-group" data-username="KFrank" data-post="2" data-topic="116003">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/ecb155/48.png" class="avatar"> KFrank:</div>
<blockquote>
<p>logits</p>
</blockquote>
</aside>
<p>Could you pls help me out about how to go about it??I have put my model code in the reply</p>