Loss remains the same for every epoch

Sourabh: <p>I am doing the following :</p>
<ol>
<li>A model is used as an encoder.</li>
<li>The model predicts some outputs which I then take and convert into a numpy array. (so I also detach them first from the tensor.)</li>
<li>During the course of subsequent calculations on this numpy array , I use a <code>argmax()</code> to finally return me something ( for example something like [[1,4,6,3]]. Lets call them <code>predictions</code>.)</li>
<li>I convert these predictions to tensors and also set the <code>requires_grad</code> paramter to <code>True</code> as:</li>
</ol>
<blockquote>
<p>torch.tensor(predictions, requires_grad=True)</p>
</blockquote>
<ol start="5">
<li>I then try and apply the torch.nn.CrossEntropyLoss on this and my ground truth as :</li>
</ol>
<blockquote>
<p>loss=torch.nn.CrossEntropyLoss(predictions,ground_truth)</p>
</blockquote>
<p>When I run the code, I see the same loss for all the epochs and the model basically does not train.<br>
For example:</p>
<blockquote>
<p>Epoch : 1, Loss : 4.0<br>
Epoch : 2, Loss : 4.0<br>
Epoch : 3, Loss : 4.0<br>
Epoch : 4, Loss : 4.0</p>
</blockquote>
<p>Now , I know that the <code>argmax()</code> is not differentiable and hence should not be used in the loss. However, I try and do some post processing on my outputs (that involves me doing the argmax() operation) before converting it to a tensor and then apply the <code>CrossEntropy</code> loss to it.<br>
If it is still wrong, it there an alternative to using the argmax()?<br>
Could you please help me with this??</p>

J_Johnson: <p>Hello! <code>torch.argmax()</code> is something you may want to use for statistical purposes or during inference to make sense of the model outputs.</p>
<p>But you don’t need it when using CrossEntropyLoss for training. For example:</p>
<pre><code class="lang-auto">loss = nn.CrossEntropyLoss()
model_output = torch.rand((3, 5), requires_grad=True) # float values between 0 and 1
targets = torch.empty(3, dtype=torch.long).random_(5) # integers between 0 and 4

print(model_output, targets)
output = loss(input, target)
output.backward()
</code></pre>

Sourabh: <p>Hello! I understand that <code>argmax</code> is not differentiable and thus should not be used in the loss function. However, I still don’t get why it should be a problem when I am using it as a part of some independent operations and then using the results obtained in a fresh numpy array for calculating the loss. (I am sorry if this is not clear. I can elaborate on it if you want)<br>
I also realize that <code>Cross Entropy</code> deals with <code>logits</code> and thus its probably wrong to apply <code>Cross entropy loss</code> in my use case.<br>
However, even when I apply the <code>L1Loss</code> I still don’t see any change in my loss on an epoch to epoch basis.<br>
Could you help me understand it a little better?<br>
Also do you have a potential alterntative for the argmax function?</p>

J_Johnson: <p>Consider the following model output <code>tensor([[3.4, 0.1, 0.1]])</code>. This means the model is predicting class 0. If we ran <code>argmax</code>, we’d get 0. But suppose class 2 was the correct class. If we tried <code>L1Loss</code> on the class integers, we’d have a loss of 2.0. But then if 1 were the correct class, the loss would be 1.0.</p>
<p>Why should the order of the classes make any difference for the amount of loss? It shouldn’t. If we shuffled the classes, along with the model output, the loss should be the same. But if using <code>L1Loss</code> on <code>argmax</code> values, you’re not going to get any sort of sensible loss.</p>
<p>If you wanted to apply <code>L1Loss</code>, instead, you <em>could</em> change the <strong>targets</strong> to a <code>one_hot</code> vector. I.e. class 2 being <code>tensor([[0.0, 0.0, 1.0]])</code> and calculate loss with respect to the raw model outputs. At least that will give you something more sensible. And this would be agnostic to changes in order.</p>
<p>Although you now would have a new set of issues. A big one being that we want the model to maximize the value at class position 2 and minimize the values everywhere else. As opposed to just targetting 0s and 1.</p>
<p>That’s where <code>CrossEntropyLoss</code> comes in. It handles that for you all behind the scenes. But it requires a vector of class probabilities as the model output.</p>

ptrblck: <aside class="quote no-group" data-username="Sourabh" data-post="3" data-topic="177029">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sourabh/48/45473_2.png" class="avatar"> Sourabh:</div>
<blockquote>
<p>I understand that <code>argmax</code> is not differentiable and thus should not be used in the loss function. However, I still don’t get why it should be a problem</p>
</blockquote>
</aside>
<p>A non-differentiable operation will break the computation graph and will thus not allow you to calculate the gradients for any parameter previously used in the forward pass. Setting the <code>requires_grad</code> attribute of the detached tensor to <code>true</code> won’t help and somehow attach this tensor back to the computation graph.<br>
The used loss function is irrelevant at this point, since the gradients w.r.t. the model parameters won’t even be calculated.</p>

Sourabh: <p>Thanks a lot for the explaination!!</p>

Sourabh: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> I think this behaviour is also the same when I maually detach a tensor??  (which makes sense)<br>
For example, if I have a tensor <code>a</code> and I do something like?</p>
<p><code>a.cpu().detach().numpy()</code></p>
<p>I see the following question relevant and is exactly my usecase : <a href="https://discuss.pytorch.org/t/get-values-from-tensor-without-detaching/138465" class="inline-onebox">Get values from tensor without detaching?</a></p>
<p>However, the question is unanswered. Can you <a class="mention" href="/u/j_johnson">@J_Johnson</a> <a class="mention" href="/u/ptrblck">@ptrblck</a> , please suggest a work around to detaching to so that my computation graph is not altered.</p>

ptrblck: <aside class="quote no-group" data-username="Sourabh" data-post="7" data-topic="177029">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sourabh/48/45473_2.png" class="avatar"> Sourabh:</div>
<blockquote>
<p>please suggest a work around to detaching to so that my computation graph is not altered.</p>
</blockquote>
</aside>
<p>It’s not possible and wouldn’t make sense, since detaching a tensors explicitly cuts the computation graph. If you want to keep the computation graph you shouldn’t call <code>detach()</code> on the tensor.<br>
If you need to use a 3rd party library, such as <code>numpy</code>, you would need to implement the <code>backward</code> function manually via a custom <code>autograd.Function</code>.</p>
<p>For non-differentiable operations you could create a custom <code>autograd.Function</code> returning all zero gradients (or somehow come up with other gradients). However, I wouldn’t know how all-zero gradients would be beneficial.</p>

J_Johnson: <aside class="quote no-group" data-username="Sourabh" data-post="1" data-topic="177029">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sourabh/48/45473_2.png" class="avatar"> Sourabh:</div>
<blockquote>
<p>However, I try and do some post processing on my outputs (that involves me doing the argmax() operation)</p>
</blockquote>
</aside>
<p>Perhaps you can share:</p>
<ol>
<li>What your “post processing” code is(or a reduced version of it);</li>
<li>What your motivation is for using it;</li>
<li>What you hope to accomplish overall.</li>
</ol>

Sourabh: <p>Hello <a class="mention" href="/u/j_johnson">@J_Johnson</a> <a class="mention" href="/u/ptrblck">@ptrblck</a> , I was converting the tensors to numpy array to get some dot products.<br>
As the loss does not change, I then replaced the numpy operations with the <code>torch.mul</code> followed by <code>torch.sum</code>.  I then save it in a numpy array and then convert it into tensors using <code>torch.from_numpy(my_numpy_array)</code>.<br>
I then apply the <code>torch.nn.BCEWithLogitsLoss</code>.<br>
However, I still do not see any changes in the loss. Its the same.</p>
<pre><code class="lang-auto">
input = autograd.Variable(output_after_my_operations, requires_grad=True)      
assert input.shape == self.y.shape   #torch.size([16])
loss = self.criteria_bce(input, self.y)
</code></pre>

J_Johnson: <p>PyTorch can take the <a href="https://pytorch.org/docs/stable/generated/torch.dot.html" rel="noopener nofollow ugc">dot product</a>. By handling it through torch, you can maintain the graph. I see no reason to change to numpy as that will break the graph.</p>
<p>If you want to use autograd, all operations must be handled through PyTorch.</p>

ptrblck: <p>This behavior is still expected as explained multiple times already.<br>
If you detach a tensor from the computation graph (e.g. by transforming it to a numpy array) the previous operations and used parameters will not get any valid gradients computed from the final loss.<br>
Re-creating a tensor via <code>torch.from_numpy</code> or the deprecated <code>Variable</code> will not change this behavior and the computation graph will still be cut.<br>
In my previous post I’ve mentioned already that you should either avoid detaching the tensor from the computation graph by sticking to differentiable PyTorch operations or you would have to implement a custom <code>autograd.Function</code>.</p>