Errors using torch.compile() on wav2vec2 model
Labels: triaged, ezyang's list, oncall: pt2, module: dynamic shapes, module: inductor, module: dynamo

### 🐛 Describe the bug

I've been experimenting with enabling torch.compile() for the wav2vec2 model in torchaudio, and ran into a few issues with it. My code can be found [here](https://github.com/pytorch/audio/pull/2941). Here's a summary of the issues I found, with reproduction instructions for each:

### 1) Errors with `inference_mode()`
To reproduce this, change the `no_grad()` to `inference_mode()` in `examples/asr/librispeech_ctc_decoder/inference.py`, and run it:
```
python examples/asr/librispeech_ctc_decoder/inference.py --librispeech_path /PATH/TO/LIBRISPEECH --batch_size 2 --compile
```
It produces the error "RuntimeError: Inference tensors do not track version counter.", but continues to run. Full stack trace is [here](https://gist.githubusercontent.com/kalakris/5764ba0d412c8e2d250f9c14ada3d4f9/raw/3b7ee3151fab49dc300dcdc631f88c65a6bbed0f/inference_mode.txt).

### 2) Errors with `dynamic=True`
Ideally, we'd want to run wav2vec2 with dynamic tensor sizes per call. But I wasn't able to do that. To reproduce, uncomment line 47 of `examples/asr/librispeech_ctc_decoder/inference.py`, comment out line 48, and run it:
```
python examples/asr/librispeech_ctc_decoder/inference.py --librispeech_path /PATH/TO/LIBRISPEECH --batch_size 2 --compile
```
Stack trace is [here](https://gist.githubusercontent.com/kalakris/5764ba0d412c8e2d250f9c14ada3d4f9/raw/3b7ee3151fab49dc300dcdc631f88c65a6bbed0f/dynamic_equals_true.txt).

### 3) Errors when passing the `lengths` parameter to the model
When batching inputs to this model, we need to pass in the lengths of each sample as well. Unfortunately this performs some operations which seems to break with `torch.compile()`. To reproduce, uncomment line 93 of `examples/asr/librispeech_ctc_decoder/inference.py`, comment out line 94, and run it:
```
python examples/asr/librispeech_ctc_decoder/inference.py --librispeech_path /PATH/TO/LIBRISPEECH --batch_size 2 --compile
```
Stack trace is [here](https://gist.githubusercontent.com/kalakris/5764ba0d412c8e2d250f9c14ada3d4f9/raw/3b7ee3151fab49dc300dcdc631f88c65a6bbed0f/with_lengths_parameter.txt).

### Versions

PyTorch version: 2.0.0.dev20221216
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.10.8 (main, Nov 24 2022, 14:13:03) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.4.0-124-generic-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: 11.6.112
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: Tesla V100-SXM2-32GB
Nvidia driver version: 470.141.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.4
[pip3] torch==2.0.0.dev20221216
[pip3] torchaudio==0.14.0.dev20221216
[pip3] torchvision==0.15.0.dev20221216
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.4.0           h06a4308_640
[conda] mkl-service               2.4.0           py310h7f8727e_0
[conda] mkl_fft                   1.3.1           py310hd6ae3a3_0
[conda] mkl_random                1.2.2           py310h00e6091_0
[conda] numpy                     1.23.4          py310hd5efca6_0
[conda] numpy-base                1.23.4          py310h8e6c178_0
[conda] pytorch                   2.0.0.dev20221216 py3.10_cuda11.6_cudnn8.3.2_0    pytorch-nightly
[conda] pytorch-cuda              11.6                 h867d48c_2    pytorch-nightly
[conda] pytorch-mutex             1.0                        cuda    pytorch-nightly
[conda] torchaudio                0.14.0.dev20221216     py310_cu116    pytorch-nightly
[conda] torchtriton               2.0.0+0d7e753227           py310    pytorch-nightly
[conda] torchvision               0.15.0.dev20221216     py310_cu116    pytorch-nightly

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @mlazos @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire @mthrok