Repository: pytorch
Issue: torch.nn.functional.ctc_loss with invalid input produce NaN or infinity gradient, while the batch entries are fine
Labels: module: nn,module: loss,module: error checking,triaged,Stale

### 🐛 Describe the bug

I searched a bug in my code and found the following strange behavior of the ctc loss.
I made a small toy example:
```python
import torch
import numpy as np

for s in [
    [0],
    [1],
    [0, 1],
    [0, 0],
    [1, 0],
    [1, 1],
]:
    T, N, C, S = 143, 2, 30, 33
    probs_shape = (143, 2, 30)

    log_probs = torch.arange(T * N * C, dtype=torch.float32).reshape(T, N, C).log_softmax(2).detach().requires_grad_()
    targets = torch.full((N, S), 1, dtype=torch.long)

    targets[0, 5] = 35  # <----- Bug on my side

    input_lengths = torch.full([N], T, dtype=torch.long)
    target_lengths = torch.tensor([15, 10])

    loss = torch.nn.functional.ctc_loss(log_probs[:, s], targets[s], input_lengths[s], target_lengths[s])
    loss.backward()

    print(s, loss, torch.std(log_probs.grad))
```
that produces
```python
[0] tensor(240.8816, grad_fn=<MeanBackward0>) tensor(0.0103)
[1] tensor(404.3048, grad_fn=<MeanBackward0>) tensor(0.0146)
[0, 1] tensor(322.5932, grad_fn=<MeanBackward0>) tensor(nan)  # <------ Why is only this NaN?
[0, 0] tensor(240.8816, grad_fn=<MeanBackward0>) tensor(0.0078)
[1, 0] tensor(322.5932, grad_fn=<MeanBackward0>) tensor(0.0080)
[1, 1] tensor(404.3048, grad_fn=<MeanBackward0>) tensor(0.0146)
```
on my machine. I also tested it on other machines, where it also produced this strange behavior.
I had to search for the right values in the examples to show the bug (e.g. target_lengths).

Now let me explain:
In my code was a bug and a `35` was a target, while `C` (`C = number of characters in alphabet including blank`) was smaller.
I observed a NaN gradient and while investigating, I reduced the minibatch size and the NaN gradient disappeared.

In the example above, I try to show this strange behavior.
The first example has one wrong label and the second is fine.
Calling `ctc_loss` and `backward` on the individual examples (e.g. `[0]`) works, but it fails, when evaluating both in the order `[0, 1]`, while the order `[1, 0]` works. Also duplicating the first example (`[0, 0]`) works.

It wound be nice to get a consistent behavior of `ctc_loss`, because it is difficult to find the bug, when it only happens for special combinations of the input with a particular order.

I found that @t-vi answered question in the past to CTC.


### Versions


```
Collecting environment information...
PyTorch version: 1.11.0.dev20211025+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.3 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: version 3.14.0
Libc version: glibc-2.31

Python version: 3.8 (64-bit runtime)
Python platform: Linux-5.4.0-91-generic-x86_64-with-glibc2.10
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.3
[pip3] numpydoc==1.1.0
[pip3] padertorch==0.0.0
[pip3] torch==1.11.0.dev20211025+cpu
[pip3] torch-complex==0.1.2
[pip3] torchaudio==0.11.0.dev20211025+cpu
[pip3] torchsummary==1.5.1
[pip3] torchvision==0.12.0.dev20211025+cpu
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.3.0           h06a4308_520  
[conda] mkl-include               2020.1                      217  
[conda] mkl-service               2.4.0            py38h7f8727e_0  
[conda] mkl_fft                   1.3.0            py38h42c9631_2  
[conda] mkl_random                1.2.2            py38h51133e4_0  
[conda] numpy                     1.20.3           py38hf144106_0  
[conda] numpy-base                1.20.3           py38h74d4b33_0  
[conda] numpydoc                  0.9.2                    pypi_0    pypi
[conda] padertorch                0.0.0                     dev_0    <develop>
[conda] torch                     1.11.0.dev20211025+cpu          pypi_0    pypi
[conda] torch-complex             0.1.2                     dev_0    <develop>
[conda] torchaudio                0.11.0.dev20211025+cpu          pypi_0    pypi
[conda] torchsummary              1.5.1                    pypi_0    pypi
[conda] torchvision               0.12.0.dev20211025+cpu          pypi_0    pypi
```




cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345