akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Using `ignore_index` only at test time

username: jennhu
cooked: <p>I am training a language model. I want to ignore a certain class index (<code>-1</code>) in computing loss <strong>only at test time</strong>, but consider all class indices in the loss function during training.</p>
<p>So at test time, I map all instances of the to-be-ignored vocab items to that index (<code>-1</code>), and then pass <code>ignore_index=-1</code> to <code>torch.nn.CrossEntropyLoss</code>. But that gives me this error:</p>
<pre><code class="lang-auto">/opt/conda/conda-bld/pytorch_1565287148058/work/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, 
Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [4,0,0] Assertion `t &gt;=
 0 &amp;&amp; t &lt; n_classes` failed.
</code></pre>
<p>Is this because I am training and testing my model with a different number of classes? How can I fix this? Thank you for your help!</p>

username: ptrblck
cooked: <p>Are you sure you are using the right criterion?<br>
This dummy code snippet seems to work:</p>
<pre><code class="lang-python">criterion = nn.CrossEntropyLoss(ignore_index=-1)
output = torch.randn(10, 10, requires_grad=True)
target = torch.randint(0, 10, (10,))
target[0] = -1
loss = criterion(output, target)
loss.backward()
print(output.grad)
</code></pre>
<p>The gradients at <code>output.grad[0]</code> are all zeros as expected.</p>
<p>If that doesn’t work, could you please post a code snippet so that we could have a look?</p>