Pytorch CTCLoss Error

shrutee: <p>When I try to use  <code>torch.nn.CTCLoss</code> , with  <code>reduction='none'</code> , I get the following error.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/7/a7cbe6eeae2f6e2502df19b965e6827d086a26ae.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/a7cbe6eeae2f6e2502df19b965e6827d086a26ae" title="39%20PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/7/a7cbe6eeae2f6e2502df19b965e6827d086a26ae_2_666x500.jpeg" alt="39%20PM" data-base62-sha1="nWoAW5doiG2XHnKMxu5wwGlPhZI" width="666" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/7/a7cbe6eeae2f6e2502df19b965e6827d086a26ae_2_666x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/a/7/a7cbe6eeae2f6e2502df19b965e6827d086a26ae_2_999x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/a/7/a7cbe6eeae2f6e2502df19b965e6827d086a26ae.jpeg 2x" data-dominant-color="767672"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">39%20PM</span><span class="informations">1280×960 169 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Now, there is no way I can pass  <code>size_average</code>  and  <code>reduce</code>  as parameters to pytorch CTC Loss. I am assuming that these values must be ambiguous right now, hence the error.<br>
Any suggestions?</p>
<p>Do we have anyway to pass these parameters through CTCLoss so that they are not ambiguous?</p>
<p>My CTCLoss looks something like this:</p>
<p><code>torch.nn.CTCLoss(reduction='sum', zero_infinity=True, **kwargs)</code></p>

ptrblck: <p>Could you post a code snippet to reproduce this issue?<br>
Based on the screenshot you’ve posted it looks like <code>reduce</code> and <code>size_average</code> is used, which are the legacy arguments, so which PyTorch version are you using?</p>
<p>PS: You can add code snippets directly by wrapping them into three backticks ``` <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

shrutee: <p>Hey, pytorch version is 1.3.1</p>
<p>Code - <a href="https://gist.github.com/MittalShruti/735aa0f7902de27ca7dafeff4c88d6d1#file-debug-model2-build-stt-asr-ipynb" rel="nofollow noopener">asr ctc</a></p>

ptrblck: <p>Unfortunately, the code is not executable. Could you create dummy inputs, so that we could debug this issue?<br>
Using your default values (<code>reduction='none', zero_infinity=True</code>), this dummy code snippet works:</p>
<pre><code class="lang-python">ctc = torch.nn.CTCLoss(reduction='none', zero_infinity=True)

T = 50      # Input sequence length
C = 20      # Number of classes (including blank)
N = 16      # Batch size
S = 30      # Target sequence length of longest target in batch
S_min = 10  # Minimum target length, for demonstration purposes
# Initialize random batch of input vectors, for *size = (T,N,C)
input = torch.randn(T, N, C).log_softmax(2).detach().requires_grad_()
# Initialize random batch of targets (0 = blank, 1:C = classes)
target = torch.randint(low=1, high=C, size=(N, S), dtype=torch.long)
input_lengths = torch.full(size=(N,), fill_value=T, dtype=torch.long)
target_lengths = torch.randint(low=S_min, high=S, size=(N,), dtype=torch.long)

loss = ctc(input, target, input_lengths, target_lengths)
</code></pre>

shrutee: <p>I have created a small dataset.<br>
Code - <a href="https://gist.github.com/MittalShruti/d7ae4380ffd7518b424b03bae6ac51af#file-debug-asr-ipynb" rel="nofollow noopener">debug asr</a><br>
Code before ‘<strong>Model</strong>’ is just creating the databunch, so can run it as it is.</p>

shrutee: <p>Update: CTC works fine. There’s some issue with how I have defined the model.</p>