Repository: pytorch
Issue: Feature Request: deterministic CUDA torch.nn.CTCLoss
Labels: feature,module: nn,module: loss,triaged,module: determinism

Updated again.

The inclusion of the CTC loss function in pytorch is great. However, the loss may be deterministic or non-deterministic under some conditions. In typical speech training scenarios, it could actually happen that for some batches the CUDNN algorithm is used (which can be set to deterministic or non-deterministic modes), and for some batches the CUDA algorithm is used (which is always non-deterministic).

| Device | `USE_CUDNN` | `torch.backends.cudnn.deterministic` | Code | Deterministic |
|---|---|---|---|---|
| CPU | - | -| [ATen/native/LossCTC.cpp](https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/LossCTC.cpp)  | Yes | 
| GPU | False | - | [ATen/native/cuda/LossCTC.cu](https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/cuda/LossCTC.cu) | No |
| GPU | True | True  |  [ATen/native/cudnn/LossCTC.cpp](https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/cudnn/LossCTC.cpp) | Yes |
| GPU | True | False | [ATen/native/cudnn/LossCTC.cpp](https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/cudnn/LossCTC.cpp) | No|

- `USE_CUDNN` is a boolean variable re-generated for every batch [here](https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/LossCTC.cpp#L336-L355)

I have attached a code sample for reproducing the non-deterministic loss function. Even though the difference in this code sample is small, it gets larger in training of speech recognition models. To the best of my knowledge, warp-ctc is deterministic on GPU (e.g.via [Sean Naren's pytorch binding](https://github.com/SeanNaren/warp-ctc)).

Would it be possible to introduce a deterministic mode in the CUDA version? To the best of my knowledge, this needs an alternative for `atomicAdd` in [`LossCTC.cu`](https://github.com/pytorch/pytorch/blob/29f096cc70cf9cc1a317ae7107228215b7dde60b/aten/src/ATen/native/cuda/LossCTC.cu#L409-L412).

Output:
```
7402
Trial 0 ::: Device cuda ::: Iter 99 ::: net_csum -1.2945041656 ::: log_prob_checksum -500026.3125000000 ::: loss 10.0990858078 ::: inc_loss 1045.5471239090
Trial 1 ::: Device cuda ::: Iter 99 ::: net_csum -1.2944996357 ::: log_prob_checksum -500026.2500000000 ::: loss 10.0990848541 ::: inc_loss 1045.5470838547
Trial 2 ::: Device cuda ::: Iter 99 ::: net_csum -1.2945439816 ::: log_prob_checksum -500026.2500000000 ::: loss 10.0990819931 ::: inc_loss 1045.5470523834
Trial 0 ::: Device cpu  ::: Iter 99 ::: net_csum 2.3939034939 ::: log_prob_checksum -499604.8750000000 ::: loss 10.1452789307 ::: inc_loss 1050.8508319855
Trial 1 ::: Device cpu  ::: Iter 99 ::: net_csum 2.3939034939 ::: log_prob_checksum -499604.8750000000 ::: loss 10.1452789307 ::: inc_loss 1050.8508319855
Trial 2 ::: Device cpu  ::: Iter 99 ::: net_csum 2.3939034939 ::: log_prob_checksum -499604.8750000000 ::: loss 10.1452789307 ::: inc_loss 1050.8508319855
```
Code:
```
import torch
import torch.nn as nn

def train_with_ctc(trial, device):
    
    # Random seed
    torch.manual_seed(0)

    # Data
    torch.manual_seed(0)
    inp=torch.randn(500,16,10, device=device)    
    targets = torch.randint(1, 20, (sum(range(100,116)),), dtype=torch.int)
    input_lengths = torch.full((16,), 500, dtype=torch.long)
    target_lengths = torch.arange(100,116, dtype=torch.long)
    
    # Network
    net=nn.Sequential(nn.Linear(10,20), torch.nn.LogSoftmax(2))
    net.to(device)
    optimizer=torch.optim.Adam(net.parameters())
    ctc_loss = nn.CTCLoss(blank=19) # use cuDNN: blank=0, use CUDA: blank !=0 (only true when other parameters are not changed)
        
    # Training loop
    inc_loss=0
    for it in range(100):
        optimizer.zero_grad()
        log_probs=net(inp)
        loss = ctc_loss(log_probs, targets, input_lengths, target_lengths)
        loss.backward()
        optimizer.step()
        inc_loss+=float(loss)

    # Print
    torch.cuda.synchronize()
    net_csum=torch.nn.utils.parameters_to_vector(net.parameters()).sum()
    print('Trial {} ::: Device {:4} ::: Iter {} ::: net_csum {:.10f} ::: log_prob_checksum {:.10f} ::: loss {:.10f} ::: inc_loss {:.10f}'.format(trial, device, it, net_csum,log_probs.sum(), float(loss), inc_loss))

if __name__ == '__main__':

    # Get cuDNN version
    print(torch.backends.cudnn.version())
    
    # Toggle cuDNN backend    
    # torch.backends.cudnn.deterministic = True # change to True and blank label to 0 to toggle CUDNN_CTC_LOSS_ALGO_DETERMINISTIC / CUDNN_CTC_LOSS_ALGO_NON_DETERMINISTIC in pytorch/aten/src/ATen/native/cudnn/LossCTC.cpp
    # torch.backends.cudnn.benchmark = False
    
    # Train with CTC
    for device in ['cuda', 'cpu']:
        for trial in range(3):
            train_with_ctc(trial, device)
        
```

cc @albanD @mruberry @jbschlosser @walterddr @kurtamohler