torch.nn.CTCLoss returns infinity for some inputs

sugeedarou: <p>Hi,<br>
I tried to replace CrossEntropyLoss with CTCLoss but I often get infinity outputs,<br>
setting zero_infinity=True just makes it output 0 forever.<br>
Any idea why?</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn

probs = torch.tensor(
[[[-3.6852, -3.8380, -3.8917, -3.8420, -4.1217, -3.5202, -3.8134,
    -3.3443, -4.5487, -3.8015, -3.9781, -3.0194, -4.0216, -3.5204,
    -4.2357, -4.2557, -4.6121, -3.9420, -3.7618, -4.0152, -3.7686,
    -3.2913, -3.7141, -3.9715, -3.3414, -3.7910, -3.5413, -3.7887,
    -3.9627, -3.7912, -4.6102, -3.8574, -4.7015, -4.1991, -3.4495,
    -3.6784, -3.7860, -4.4842, -3.9561, -4.3670, -4.5784, -4.1805,
    -4.8021, -4.1785, -3.1910, -4.4310, -4.5877, -4.8743, -4.7679]],

[[-3.6433, -3.8820, -3.8138, -3.8630, -4.0570, -3.4049, -3.8250,
    -3.4900, -4.6103, -3.7115, -4.0392, -3.1994, -4.0751, -3.4773,
    -4.1737, -4.2480, -4.7001, -3.9189, -3.7473, -3.9203, -3.8479,
    -3.3608, -3.5682, -3.9201, -3.3088, -3.8354, -3.6256, -3.6313,
    -3.9220, -3.7211, -4.6523, -3.8543, -4.6962, -4.3603, -3.3139,
    -3.5954, -3.7370, -4.4784, -3.9313, -4.4535, -4.6581, -4.1376,
    -4.8446, -4.2231, -3.2970, -4.3959, -4.6230, -4.8661, -4.8886]],

[[-3.6278, -3.9100, -3.7469, -3.8844, -3.9681, -3.4005, -3.8425,
    -3.6862, -4.4955, -3.6545, -4.0343, -3.4717, -4.1044, -3.5075,
    -4.0795, -4.1596, -4.6185, -3.8613, -3.7557, -3.8153, -3.9310,
    -3.5221, -3.4801, -3.8535, -3.3727, -3.8728, -3.7805, -3.5169,
    -3.8751, -3.6887, -4.5521, -3.8429, -4.5521, -4.4218, -3.2836,
    -3.5790, -3.6708, -4.3754, -3.9049, -4.3983, -4.6034, -4.0614,
    -4.7089, -4.1932, -3.4841, -4.2479, -4.5258, -4.6913, -4.8192]]])
labels = torch.tensor([[11, 24, 24]])
lengths = torch.tensor([3])

criterion = nn.CTCLoss(blank=48, reduction='none')
lossv = criterion(probs, labels, lengths, lengths)
print(lossv)
</code></pre>
<p>probs were aquired from GRU output using:<br>
<code>probs = F.log_softmax(outputs, dim=2).transpose(0, 1)</code><br>
the model should classify 48 classes and outputs 49 classes</p>
<p>Thanks in advance!</p>

Unity05: <p>Hi,</p>
<p>CTCLoss returns inf if the input length is too short. It appears that for the PyTorch CTCLoss, the input lengths are too short if input_len = target_len, actually that’s also not the primary use case for CTCLoss. I don’t know whether that’s intentional or a bug of PyTorch’s implementation, as this limitation does not make sense to me.</p>
<p>Regards,<br>
Unity05</p>

sugeedarou: <p>Thank you very much, that’s it!<br>
It seems like the PyTorch Documentation is wrong as it states “…which limits the length of the target sequence such that it must be ≤ the input length”<br>
<a href="https://pytorch.org/docs/stable/generated/torch.nn.CTCLoss.html" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.nn.CTCLoss.html</a></p>

Unity05: <p>Indeed, however I still believe that the implementation is wrong as what the PyTorch Documentation states makes perfect sense to me.<br>
Do you know any user here who is to be pinged in such cases?</p>

Unity05: <p>Hi again,</p>
<p>I’ve found out that that is not a bug or a documentation issue.<br>
It’s due to your special example:<br>
your labels are <code>[11, 24, 24]</code>. Therefore especially the last two are the same. So why does the problem occur, when the lengths are equal? Well, for the input to represent the same label twice in a row, there <strong>must</strong> be a blank in between those two, as that’s how CTC works, what is impossible to achieve with equal input lengths and target lengths.</p>
<p>Regards,<br>
Unity05</p>

sugeedarou: <p>Ah I see, so I need to insert a blank between them (and double the inputs).<br>
Thank you again! <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=9" title=":smile:" class="emoji" alt=":smile:"></p>

Unity05: <p>I’m not sure if I’ve understood you right, but your labels are fine, they just don’t work with an input of same length. <img src="https://discuss.pytorch.org/images/emoji/apple/grinning_face_with_smiling_eyes.png?v=9" title=":grinning_face_with_smiling_eyes:" class="emoji" alt=":grinning_face_with_smiling_eyes:"></p>

sugeedarou: <p>Yeah you’re right again, blanks must be in the outputs of the GRU not the labels.<br>
First time CTC user <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=9" title=":sweat_smile:" class="emoji" alt=":sweat_smile:"></p>

Unity05: <p>Nice, always something to learn there is. <img src="https://discuss.pytorch.org/images/emoji/apple/grinning_face_with_smiling_eyes.png?v=9" title=":grinning_face_with_smiling_eyes:" class="emoji" alt=":grinning_face_with_smiling_eyes:"></p>