Negative CTC loss

paarandika: <p>I’m using a CNN+LSTM model with CTC loss for an OCR task. I use the <code>null</code> character as the target value for empty images without any text. When the target is <code>null</code> character and the model correctly predicts <code>null</code> character, CTC loss produce a negative value. Is this the expected behaviour of CTC loss? If so, how should I indicate empty images?<br>
Thanks.</p>

tomassosorio: <p>I would try to use space instead of blank character, I am assuming you are referring to null as the blank which is a special character to deal with double or triple character problem.</p>

Muhammad_Ali: <p>I am trying to use CRNN, model to give me Text-Perceptual-Loss, to be used for Text Super Resolution.<br>
I am using pyotrch’s CTC loss,</p>
<pre><code class="lang-auto">criterion = nn.CTCLoss(blank=0)

def encode_text_batch(text_batch):
    
    text_batch_targets_lens = [len(text) for text in text_batch]
    text_batch_targets_lens = torch.IntTensor(text_batch_targets_lens)
    
    text_batch_concat = "".join(text_batch)
    text_batch_targets = []

    for c in text_batch_concat:
      try:
        if (c=='"' ):
          id = char2id["'"]

        else: 
          id =char2id[c]
          
        text_batch_targets.append(id)
      except:
        text_batch_targets.append(0)
                          

    text_batch_targets = torch.IntTensor(text_batch_targets)
    #print(text_batch_targets)
    return text_batch_targets, text_batch_targets_lens
def compute_loss(text_batch, text_batch_logits):
    """
    text_batch: list of strings of length equal to batch size
    text_batch_logits: Tensor of size([T, batch_size, num_classes])
    """
    print(text_batch_logits.shape)
    text_batch_logps = F.log_softmax(text_batch_logits, 2) # [T, batch_size, num_classes] 
    #print(text_batch_logps.shape) 
    #print(text_batch_logps.size(0))
    text_batch_logps_lens = torch.full(size=(text_batch_logps.size(1),), 
                                       fill_value=text_batch_logps.size(0), 
                                       dtype=torch.int32).to(device) # [batch_size] 
    #print(text_batch_logps_lens) 
    #print(text_batch_logps.shape)
    #print(text_batch_logps_lens) 
    text_batch_targets, text_batch_targets_lens = encode_text_batch(text_batch)
    #print(text_batch_targets,text_batch_targets_lens)
    #print(text_batch_targets_lens)
    loss = criterion(text_batch_logps, text_batch_targets, text_batch_logps_lens, text_batch_targets_lens)

    return loss
</code></pre>
<p>But, like for image-example I am sharing, for “SIT”, if it is predicting SI, loss is less, AS COMPARED TO WHEN IT PREDICTS "SIT<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/2/624e63d4065681d1dbb8d7b868f8ad67cfa1304b.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/624e63d4065681d1dbb8d7b868f8ad67cfa1304b" title="Screenshot 2021-03-05 161633"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/2/624e63d4065681d1dbb8d7b868f8ad67cfa1304b_2_596x500.jpeg" alt="Screenshot 2021-03-05 161633" data-base62-sha1="e1EJZv6APSO2E2WrNLzREdxC1ez" width="596" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/2/624e63d4065681d1dbb8d7b868f8ad67cfa1304b_2_596x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/6/2/624e63d4065681d1dbb8d7b868f8ad67cfa1304b_2_894x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/2/624e63d4065681d1dbb8d7b868f8ad67cfa1304b.jpeg 2x" data-dominant-color="EFEFEF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2021-03-05 161633</span><span class="informations">937×786 113 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

paarandika: <p>I have faced the same issue and did not get an answer.</p><aside class="quote quote-modified" data-post="1" data-topic="82902">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/paarandika/48/20245_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/ctc-loss-unexpected-behaviour/82902">CTC Loss Unexpected Behaviour</a> <a class="badge-category__wrapper " href="/c/nlp/8"><span data-category-id="8" data-drop-close="true" class="badge-category " title="Topics related to Natural Language Processing"><span class="badge-category__name">nlp</span></span></a>
  </div>
  <blockquote>
    I have used the following code to test the behaviour of CTC loss. 
def get_char_maps (vocab):
    char_to_index={}
    index_to_char={}
    cnt=0
    for c in vocab:
        char_to_index[c]=cnt
        index_to_char[cnt]=c
        cnt+=1
    vocab_size=cnt
    return (char_to_index, index_to_char, vocab_size)

loss_function = CTCLoss()
char_to_index, index_to_char, vocab_size = get_char_maps(['~', 'a', 'b', 'c', 'd', 'e', 'f', 'g', ' '])

empty_char = '~'
label = 'abc'
seq_len = 20
predict = 'a…
  </blockquote>
</aside>


tom: <p>As the example of <a class="mention" href="/u/paarandika">@paarandika</a> was easier to reproduce with the code he offered, I answered there. The gist of it is that you have to be a bit careful about “the prediction” here. CTC loss measures the cumulative probability of <em>all possible alignments</em>. If the individually most probable alignment is matching the targets one, that does not say much about that the sum of all possible alignments is gives a lot of probability mass to the targets. This is particularly true for cases where the predictions are somewhat longer than the target sequences (i.e. you have a lot of ε). Keep in mind that the loss is the negative loss likelihood of the targets under the predictions: A loss of 1.39 means ~25% likelihood for the targets, a loss of 2.35 means ~10% likelihood for the targets. This is very far from what you would expect from, say, a vanilla n-class classification problem, but the universe of alignments is rather large (if you have predictions of sequence length N with C characters (excluding ε), there are sum_{k=1…N}(C^k) &gt;= C^N possible predictions, some of them (hopefully) map to the target sequence (else you get infinite loss, people do ask about this on the forum).</p>
<p>Best regards</p>
<p>Thomas</p>