Repository: pytorch
Issue: Pytorch2.0 doesn't support compiling GRU and RNN model
Labels: feature,low priority,triaged,oncall: pt2,module: dynamo

### 🐛 Describe the bug

I am trying to boost model with compiling model in PyTorch2.0. However, GRU modules are not abled to be compiled. 

Following is the MWE:
```python
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self):
        super().__init__()
        self.gru = nn.GRU(16, 16, batch_first=True)
    
    def forward(self, x):
        return self.gru(x)

def test():
    device = torch.device('cuda')
    model = Model().to(device)
    model = torch.compile(model)
    x = torch.rand(1024, 20, 16).to(device)
    model(x)
```

The error message was saying:
```plain
Traceback (most recent call last):
  File "example.py", line 23, in <module>
    test()
  File "example.py", line 20, in test
    model(x)
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1482, in _call_impl
    return forward_call(*args, **kwargs)
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 83, in forward
    return self.dynamo_ctx(self._orig_mod.forward)(*args, **kwargs)
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 212, in _fn
    return fn(*args, **kwargs)
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 333, in catch_errors
    return callback(frame, cache_size, hooks)
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 480, in _convert_frame
    result = inner_convert(frame, cache_size, hooks)
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 103, in _fn
    return fn(*args, **kwargs)
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 88, in time_wrapper
    r = func(*args, **kwargs)
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 339, in _convert_frame_assert
    return _compile(
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 400, in _compile
    out_code = transform_code_object(code, transform)
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 341, in transform_code_object
    transformations(instructions, code_options)
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 387, in transform
    tracer.run()
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1684, in run
    super().run()
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 538, in run
    and self.step()
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 501, in step
    getattr(self, inst.opname)(inst)
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1750, in RETURN_VALUE
    self.output.compile_subgraph(self)
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/output_graph.py", line 553, in compile_subgraph
    self.compile_and_call_fx_graph(tx, pass2.graph_output_vars(), root)
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/output_graph.py", line 600, in compile_and_call_fx_graph
    compiled_fn = self.call_user_compiler(gm)
  File "/miniconda3/envs/pytorch2/lib/python3.8/site-packages/torch/_dynamo/output_graph.py", line 681, in call_user_compiler
    raise BackendCompilerFailed(self.compiler_fn, e) from e
torch._dynamo.exc.BackendCompilerFailed: debug_wrapper raised NotImplementedError: could not find kernel for aten._cudnn_rnn_flatten_weight.default at dispatch key DispatchKey.Meta
```

The code run smoothly if I use nn.LSTM, but got the same error if use nn.RNN.

### Versions

```plain
PyTorch version: 2.0.0.dev20221227+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: Could not collect
CMake version: version 3.25.0
Libc version: glibc-2.35

Python version: 3.8.15 (default, Nov 24 2022, 15:19:38)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.79.1-microsoft-standard-WSL2-x86_64-with-glibc2.17
Is CUDA available: True
CUDA runtime version: 11.7.64
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA TITAN RTX
Nvidia driver version: 522.25
cuDNN version: Probably one of the following:
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn.so.8.7.0
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.7.0
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.7.0
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.7.0
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.7.0
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.7.0
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.7.0
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.24.1
[pip3] torch==2.0.0.dev20221227+cu117
[pip3] torchaudio==2.0.0.dev20221222+cu117
[pip3] torchtriton==2.0.0+0d7e753227
[pip3] torchvision==0.15.0.dev20221222+cu117
[conda] numpy                     1.24.1                   pypi_0    pypi
[conda] torch                     2.0.0.dev20221227+cu117          pypi_0    pypi
[conda] torchaudio                2.0.0.dev20221222+cu117          pypi_0    pypi
[conda] torchtriton               2.0.0+0d7e753227          pypi_0    pypi
[conda] torchvision               0.15.0.dev20221222+cu117          pypi_0    pypi
```

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @soumith @ngimel @mlazos @yanboliang @chunyuan-w @Xia-Weiwen @desertfire