`@torch.compile` fails with `InternalTorchDynamoError` on torch-nightly
Labels: triaged, oncall: pt2

### 🐛 Describe the bug

I'm running a modified Stable Diffusion code with `@torch.compile` [applied on one of the functions](https://github.com/sgrigory/stablediffusion2/blob/115130e4a9c8562225f24e6929575906752c79f9/ldm/models/diffusion/plms.py#L187). The compilation breaks with `TypeError: object of type 'tuple_iterator' has no len()` (full logs below)

The error goes away if [here](https://github.com/sgrigory/stablediffusion2/blob/115130e4a9c8562225f24e6929575906752c79f9/ldm/modules/diffusionmodules/util.py#L113-L114) I replace
```
args = tuple(inputs) + tuple(params)
return CheckpointFunction.apply(func, len(inputs), *args)
```
with
```
return CheckpointFunction.apply(func, len(inputs), *inputs, *params)
```
so this is probably something about how tuples are treated.
Note:
- The checkpointing function in question memorizes results of the forward pass for the backward pass, no saving on disk
- The error doesn't go away after setting `torch._inductor.config.triton.cudagraphs = False`
- The full code is [here](https://github.com/sgrigory/stablediffusion2/tree/115130e4a9c8562225f24e6929575906752c79f9)

<details>
  <summary>Show full error log</summary>

```
$ TORCHDYNAMO_REPRO_AFTER="dynamo" with-proxy python stable-diffusion/scripts/txt2img.py --prompt "A photo" --seed 187 --plms --config stable-diffusion/configs/stable-diffusion/v2-inference_native_mha.yaml --ckpt /tmp/model_native_mha.ckpt --n_iter 1 --n_samples 1
---------------------------------------------------------

Global seed set to 187
Loading model from /tmp/model_native_mha.ckpt
Global Step: 875000
No module 'xformers'. Proceeding without it.
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 865.99 M params.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Creating invisible watermark encoder (see https://github.com/ShieldMnt/invisible-watermark)...
Sampling:   0%|                                                                                                                                             | 0/1 [00:00<?, ?it/sData shape for PLMS sampling is (1, 4, 64, 64)                                                                                                               | 0/1 [00:00<?, ?it/s]
Running PLMS Sampling with 50 timesteps
PLMS Sampler:   0%|                                                                                                                                        | 0/50 [00:03<?, ?it/s]
data:   0%|                                                                                                                                                 | 0/1 [00:03<?, ?it/s]
Sampling:   0%|                                                                                                                                             | 0/1 [00:03<?, ?it/s]
Traceback (most recent call last):
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 440, in _compile
    check_fn = CheckFunctionManager(
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/_dynamo/guards.py", line 518, in init
    guard.create(local_builder, global_builder)
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/_guards.py", line 163, in create
    return self.create_fn(self.source.select(local_builder, global_builder), self)
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/_dynamo/guards.py", line 298, in LIST_LENGTH
    code.append(f"len({ref}) == {len(value)}")
TypeError: object of type 'tuple_iterator' has no len()

Set torch._dynamo.config.verbose=True for more information


You can suppress this exception and fall back to eager by setting:
    torch._dynamo.config.suppress_errors = True


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "stable-diffusion/scripts/txt2img.py", line 325, in <module>
    main(opt)
  File "stable-diffusion/scripts/txt2img.py", line 275, in main
    samples, _ = sampler.sample(S=opt.steps,
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 34, in decorate_context
    return func(*args, **kwargs)
  File "/data/sandcastle/boxes/fbsource/stable-diffusion/ldm/models/diffusion/plms.py", line 99, in sample
    samples, intermediates = self.plms_sampling(conditioning, size,
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 34, in decorate_context
    return func(*args, **kwargs)
  File "/data/sandcastle/boxes/fbsource/stable-diffusion/ldm/models/diffusion/plms.py", line 167, in plms_sampling
    outs = self.p_sample_plms(img, cond, ts, index=index, use_original_steps=ddim_use_original_steps,
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 212, in _fn
    return fn(*args, **kwargs)
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 34, in decorate_context
    return func(*args, **kwargs)
  File "/data/sandcastle/boxes/fbsource/stable-diffusion/ldm/models/diffusion/plms.py", line 232, in p_sample_plms
    e_t = get_model_output(x, t)
  File "/data/sandcastle/boxes/fbsource/stable-diffusion/ldm/models/diffusion/plms.py", line 202, in get_model_output
    e_t_uncond, e_t = self.model.apply_model(x_in, t_in, c_in).chunk(2)
  File "/data/sandcastle/boxes/fbsource/stable-diffusion/ldm/models/diffusion/ddpm.py", line 858, in apply_model
    x_recon = self.model(x_noisy, t, **cond)
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1482, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/sandcastle/boxes/fbsource/stable-diffusion/ldm/models/diffusion/ddpm.py", line 1329, in forward
    out = self.diffusion_model(x, t, context=cc)
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1482, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/sandcastle/boxes/fbsource/stable-diffusion/ldm/modules/diffusionmodules/openaimodel.py", line 778, in forward
    h = module(h, emb, context)
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1482, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/sandcastle/boxes/fbsource/stable-diffusion/ldm/modules/diffusionmodules/openaimodel.py", line 82, in forward
    x = layer(x, emb)
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1482, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/sandcastle/boxes/fbsource/stable-diffusion/ldm/modules/diffusionmodules/openaimodel.py", line 249, in forward
    return checkpoint(
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 333, in catch_errors
    return callback(frame, cache_size, hooks)
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 480, in _convert_frame
    result = inner_convert(frame, cache_size, hooks)
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 103, in _fn
    return fn(*args, **kwargs)
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 88, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 339, in _convert_frame_assert
    return _compile(
  File "/home/grigorysizov/anaconda3/envs/ldm/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 470, in _compile
    raise InternalTorchDynamoError() from e
torch._dynamo.exc.InternalTorchDynamoError
```
</details>

@msaroufim 

### Versions
```
PyTorch version: 2.0.0.dev20230102+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A


OS: CentOS Stream 8 (x86_64)
GCC version: (GCC) 8.5.0 20210514 (Red Hat 8.5.0-17)
Clang version: Could not collect
CMake version: version 3.25.0
Libc version: glibc-2.28


Python version: 3.8.5 (default, Sep  4 2020, 07:30:14)  [GCC 7.3.0] (64-bit runtime)
Python platform: Linux-5.12.0-0_fbk9_clang_7032_gb6cf154957d9-x86_64-with-glibc2.10
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: 
GPU 0: Tesla V100-SXM2-16GB
GPU 1: Tesla V100-SXM2-16GB
GPU 2: Tesla V100-SXM2-16GB
GPU 3: Tesla V100-SXM2-16GB
GPU 4: Tesla V100-SXM2-16GB
GPU 5: Tesla V100-SXM2-16GB
GPU 6: Tesla V100-SXM2-16GB
GPU 7: Tesla V100-SXM2-16GB


Nvidia driver version: 470.57.02
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True


Versions of relevant libraries:
[pip3] numpy==1.23.1
[pip3] open-clip-torch==2.0.2
[pip3] pytorch-lightning==1.4.2
[pip3] pytorch-triton==2.0.0+0d7e753227
[pip3] torch==2.0.0.dev20230102+cu117
[pip3] torchmetrics==0.6.0
[pip3] torchvision==0.15.0.dev20230102+cu117
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               11.4.1               h8ab8bb3_9    nvidia
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py38h7f8727e_0  
[conda] mkl_fft                   1.3.1            py38hd3c417c_0  
[conda] mkl_random                1.2.2            py38h51133e4_0  
[conda] numpy                     1.23.1           py38h6c91a56_0  
[conda] numpy-base                1.23.1           py38ha15fc14_0  
[conda] open-clip-torch           2.0.2                    pypi_0    pypi
[conda] pytorch-lightning         1.4.2                    pypi_0    pypi
[conda] pytorch-triton            2.0.0+0d7e753227          pypi_0    pypi
[conda] torch                     2.0.0.dev20230102+cu117          pypi_0    pypi
[conda] torchmetrics              0.6.0                    pypi_0    pypi
[conda] torchvision               0.15.0.dev20230102+cu117          pypi_0    pypi
```

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh