Isn't setting default ignore_index=-100 a bit dangerous?

username: Enumaris
cooked: <p>I noticed that for the cross_entropy_loss function there is a ignore_index keyword that, by default, is set to -100. Isn’t this kind of dangerous if people are trying to train a classifier that has more than 100 classes (e.g. classifiers on the imagenet data)? They will have one of their classes randomly ignored won’t they? I tried to trace back this keyword myself through the source code to see where it’s actually used, but I eventually get to a torch._C._nn module that I’m not able to find, so I’m not able to confirm if this is a potential issue or not (seems like it would be hard to make this not a potential issue).</p>

username: sjmielke
cooked: <p>I too wonder about this, looking at the THNN code ( <a href="https://github.com/torch/nn/blob/master/lib/THNN/generic/ClassNLLCriterion.c" rel="nofollow noopener">https://github.com/torch/nn/blob/master/lib/THNN/generic/ClassNLLCriterion.c</a> ; no idea about what happens in other backends) it seems like it literally compares the two signed integers (current class and ignored_index) in C, so any negative number should just never match.<br>
Of course, I could be wrong, so an official answer or note in the docs about negative numbers not begin interpreted as in Python but as in C might be useful <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: Sam-gege
cooked: <p>seems this is the actual class that will be ignored, rather than the index of that class. so if your class is from 0 to num_class-1, then any negative ignore_index is equivalent, I’m too wondering why it is set to this particular value rather than a simple None</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="sjmielke" data-post="2" data-topic="10861">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sjmielke/48/3706_2.png" class="avatar"> sjmielke:</div>
<blockquote>
<p>it seems like it literally compares the two signed integers (current class and ignored_index) in C, so any negative number should just never match.</p>
</blockquote>
</aside>
<p>I’m a bit late, but I don’t fully understand this claim. Why would two signed integers never match for negative values?</p>
<aside class="quote no-group" data-username="Sam-gege" data-post="3" data-topic="10861">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sam-gege/48/42338_2.png" class="avatar"> Sam-gege:</div>
<blockquote>
<p>I’m too wondering why it is set to this particular value rather than a simple None</p>
</blockquote>
</aside>
<p>I guess the default value might have been a convenient “invalid” <code>long</code> to use without overloading the argument list to <code>long</code>, and a <code>None</code> object etc.<br>
My guess is that it’s thus just for legacy and beckwards compatibility reasons and if this argument would have been added nowadays, better overloading might have been used.</p>