Repository: pytorch
Issue: `torch.nn.LSTM` using cuDNN fails with sequences longer than 65,535
Labels: module: cudnn,module: nn,module: rnn,module: cuda,triaged,module: 64-bit

### 🐛 Describe the bug

I'm having trouble with LSTM models running on GPU when the inputted sequence gets too long. Here's an example:

```python
import torch


def test_case(sequence_length: int):
    # device = "cpu"  # SUCCESS
    device = "cuda"  # FAIL
    print("Attempt...")
    # Not important what values are chosen for these
    batch_size = 2
    input_size = 4
    x = torch.randn(sequence_length, batch_size, input_size).to(device)
    model = torch.nn.LSTM(num_layers=3, input_size=input_size, hidden_size=128).to(
        device
    )
    model(x)


test_case(sequence_length=65535)  # Succeeds
try:
    test_case(sequence_length=65536)
    print("SUCCESS!")
except RuntimeError as e:
    print("FAIL!")
    print(e)
```

Outputs:
```
Attempt...
Attempt...
FAIL!
cuDNN error: CUDNN_STATUS_NOT_SUPPORTED. This error may appear if you passed in a non-contiguous input.
```

Unfortunately, I haven't dug in to find the root cause.

### Versions

```
Collecting environment information...
PyTorch version: 2.4.0
Is debug build: False
CUDA used to build PyTorch: 12.4
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Pro
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:40:08) [MSC v.1938 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.22631-SP0
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3070 Ti
Nvidia driver version: 560.70
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture=9
CurrentClockSpeed=3600
DeviceID=CPU0
Family=198
L2CacheSize=12288
L2CacheSpeed=
Manufacturer=GenuineIntel
MaxClockSpeed=3600
Name=12th Gen Intel(R) Core(TM) i7-12700KF
ProcessorType=3
Revision=

Versions of relevant libraries:
[pip3] flake8==7.0.0
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.26.4
[pip3] onnx==1.16.0
[pip3] onnxruntime==1.17.3
[pip3] pytorch-lightning==2.2.4
[pip3] torch==2.4.0
[pip3] torchaudio==2.4.0
[pip3] torchmetrics==1.4.0.post0
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.4.0                 pypi_0    pypi
[conda] mkl-service               2.4.0           py310h2bbff1b_1
[conda] numpy                     1.26.4          py310hf667824_0    conda-forge
[conda] pytorch                   2.4.0           py3.10_cuda12.4_cudnn9_0    pytorch
[conda] pytorch-cuda              12.4                 h3fd98bf_6    pytorch
[conda] pytorch-lightning         2.2.4                    pypi_0    pypi
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torchaudio                2.4.0                    pypi_0    pypi
[conda] torchmetrics              1.4.0.post0              pypi_0    pypi
```

cc @csarofeen @ptrblck @xwang233 @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki @msaroufim