nn.CTCLoss returns inf

username: Arsham_mor
cooked: <p>I tried to train CTC model based on LibriSpeech dataset,<br>
but CTCLoss always return inf or 0 if I use zero_infinity=True.</p>
<p>here is my decoder:</p>
<pre><code class="lang-auto">class ConvDecoder(nn.Module):
    def __init__(self, in_channels, vocab_size):
        super().__init__()
        self.decoder = nn.Conv1d(
            in_channels=in_channels, out_channels=vocab_size, kernel_size=1)

    def forward(self, x):
        return self.decoder(x)
</code></pre>
<ul>
<li>
<p>It gets tensor of size <code>(batch, channels, width)</code> and return <code>(batch, 28, width)</code> where 28 is my vocab size, my vocab labels start from 1 and reserved label 0 for blank id.</p>
</li>
<li>
<p>Then I reshape model output to <code>(width, batch, 28)</code>, here is exact code:</p>
</li>
</ul>
<pre><code class="lang-auto">logits = self.forward(inputs)
batch_size, channels, sequence = logits.size()
logits = logits.view((sequence, batch_size, channels))
probs = nn.functional.log_softmax(logits, dim=-1)
</code></pre>
<ul>
<li>And finally CTC loss:</li>
</ul>
<pre><code class="lang-auto">loss = self.loss(probs, outputs, input_lengths, output_lengths)
</code></pre>
<p>how I instantiate loss:<br>
<code>self.loss = nn.CTCLoss(blank=0)</code></p>
<p>and this is my vocab dictionary:</p>
<pre><code class="lang-auto">{'a': 1,
 'b': 2,
 'c': 3,
 'd': 4,
 'e': 5,
 'f': 6,
 'g': 7,
 'h': 8,
 'i': 9,
 'j': 10,
 'k': 11,
 'l': 12,
 'm': 13,
 'n': 14,
 'o': 15,
 'p': 16,
 'q': 17,
 'r': 18,
 's': 19,
 't': 20,
 'u': 21,
 'v': 22,
 'w': 23,
 'x': 24,
 'y': 25,
 'z': 26,
 ' ': 27,
 '[unk]': 28}
</code></pre>
<p>indexes start from 1 but didn’t use any character for blank in my vocab.</p>
<h4>
<a name="i-tired-several-examples-of-ctc-models-and-they-worked-just-cant-figure-out-what-ive-done-wrong-1" class="anchor" href="#i-tired-several-examples-of-ctc-models-and-they-worked-just-cant-figure-out-what-ive-done-wrong-1"></a>I tired several examples of CTC models and they worked, just can’t figure out what I’ve done wrong.</h4>
<p>here is train logs:</p>
<pre><code class="lang-auto">Epoch 0:   0%|          | 2/28539 [00:04&lt;17:53:05,  2.26s/it, loss=inf, v_num=36]
output shape: torch.Size([732, 1, 28])

Epoch 0:   0%|          | 3/28539 [00:05&lt;15:16:36,  1.93s/it, loss=inf, v_num=36]
output shape: torch.Size([1508, 1, 28])

Epoch 0:   0%|          | 4/28539 [00:08&lt;16:28:45,  2.08s/it, loss=inf, v_num=36]
output shape: torch.Size([549, 1, 28])
</code></pre>
<p>note that output shape refers to final tensor that get into <code>self.loss</code>, set batch size to 1 for simplicity.</p>
<p>Thank you.</p>

username: tom
cooked: <aside class="quote no-group" data-username="Arsham_mor" data-post="1" data-topic="126687">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/arsham_mor/48/38888_2.png" class="avatar"> Arsham_mor:</div>
<blockquote>
<pre><code class="lang-auto">batch_size, channels, sequence = logits.size()
logits = logits.view((sequence, batch_size, channels))
</code></pre>
</blockquote>
</aside>
<p>You almost certainly want permute here and not view.</p>
<p>A loss of inf means your input sequence is too short to be aligned to your target sequence (ie the data has likelihood 0 given the model - CTC loss is a negative log likelihood after all).</p>
<p>Best regards</p>
<p>Thomas</p>

username: Arsham_mor
cooked: <p>Thanks <a class="mention" href="/u/tom">@tom</a></p>
<p>I debug my audio preprocessor and found it returns <code>tensor(1)</code> for all audio sample as input length, it shame I totally missed checking input_lengths … well, couldn’t find it without your guid.</p>

username: tom
cooked: <p>Cool, glad you solved it!</p>