Backpropagate several error values

Fabrice_Auzanneau: <p>Hi<br>
I designed a classification model that I want to train on ‘noisy’ ground truth values. This means that for a given forward propagation, leading to say 10 output values, I’d like to create a ground truth tensor which is not one hot encoded, but made of one most important probability and 9 others that are not equal to zero.</p>
<p>This is quite similar to label smoothing but the values are random instead of being all equal to 0.1<br>
I don’t understand the label smoothing examples that I found (on the net or on the forum), so can anyone help me do that ?</p>
<p>Thanks for your help</p>

KFrank: <p>Hi Fabrice!</p>
<aside class="quote no-group" data-username="Fabrice_Auzanneau" data-post="1" data-topic="201252" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fabrice_auzanneau/48/31486_2.png" class="avatar"> Fabrice_Auzanneau:</div>
<blockquote>
<p>I designed a classification model</p>
</blockquote>
</aside>
<p>Let me assume that you are using <code>CrossEntropyLoss</code> as your loss criterion.</p>
<aside class="quote no-group">
<blockquote>
<p>I’d like to create a ground truth tensor which is not one hot encoded, but made of one most important probability and 9 others that are not equal to zero.</p>
</blockquote>
</aside>
<p><code>CrossEntropyLoss</code> takes two kinds of ground-truth <code>target</code>s.  One consists<br>
of integer class labels, with no class dimension.  The second consist of<br>
floating-point probabilities for each of the classes and has a class dimension.</p>
<p>It looks like you want the latter.  It would have a larger value, closer to one,<br>
for the “correct” class and smaller values, closer to zero, for the “incorrect”<br>
classes.  Those values could be chosen randomly or could be based on<br>
something in your training data that told you that one of the “incorrect”<br>
classes had a probability of, say, <code>0.15</code>, while another had a probability of,<br>
say, <code>0.05</code>.</p>
<p>Best.</p>
<p>K. Frank</p>

Fabrice_Auzanneau: <p>Thanks Frank for your interesting answer.</p>
<p>Should I do something like this:</p>
<pre><code class="lang-auto">loss = nn.CrossEntropyLoss()
pred = model(input)
target = torch.zero_like(pred)
# here I change the target according to what I indicated
output = loss(pred, target)
output.backward()
</code></pre>

KFrank: <p>Hi Fabrice!</p>
<aside class="quote no-group quote-modified" data-username="Fabrice_Auzanneau" data-post="3" data-topic="201252" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fabrice_auzanneau/48/31486_2.png" class="avatar"> Fabrice_Auzanneau:</div>
<blockquote>
<p>Should I do something like this:</p>
<pre><code class="lang-auto"># here I change the target according to what I indicated
output = loss(pred, target)
output.backward()
</code></pre>
</blockquote>
</aside>
<p>Yes (assuming that what you “indicated” makes sense for your use case).</p>
<p>Best.</p>
<p>K. Frank</p>

Fabrice_Auzanneau: <p>Thanks again, I’ll try this very soon!</p>

Fabrice_Auzanneau: <p>It works fine, thanks again for your help!</p>