Repository: pytorch
Issue: torch._inductor.config.save_args=True and dynamic shapes do not work together
Labels: triaged,oncall: pt2,module: dynamic shapes,module: inductor

### EDIT: Minimal Repro:

```
import torch
from torch._inductor import config

config.save_args = True

@torch.compile(dynamic=True)
def foo(x):
    output = torch.zeros_like(x)
    n = output.numel()
    return output

foo(torch.randn(4))
```


### 🐛 Describe the bug

Testing out `torch.compile` with user-defined triton kernels by reproducing this [test](https://github.com/pytorch/pytorch/blob/83cb6a75ade05a7d9cd1e94aac24ef39cc66088a/test/dynamo/test_functions.py#L1870-L1898).

When I run with `dynamic=True`, `full_graph=True`, I get:
```
torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
RuntimeError: Cannot call numel() on tensor with symbolic sizes/strides
```



### Error logs

```
2023-12-06 22:56:09,218 - torch._inductor.debug - WARNING -  debug.py:406 - model___9 debug trace: /notebooks/pytorch/pytorch/user-defined-ops/logs_dynamic/debug/torch_compile_debug/run_2023_12_06_22_56_08_426697-pid_51542/torchinductor/model___9.0
2023-12-06 22:56:09,224 - torch._dynamo.eval_frame - DEBUG -  eval_frame.py:314 - Unsetting top-level compile config hash: a826c0fbcb87cbcb5d3006af100f41e2
Traceback (most recent call last):
  File "/notebooks/pytorch/pytorch/user-defined-ops/user-defined.py", line 86, in <module>
    compiled_func(t_view)
  File "/notebooks/pytorch/pytorch/torch/_dynamo/eval_frame.py", line 489, in _fn
    return fn(*args, **kwargs)
  File "/notebooks/pytorch/pytorch/torch/_dynamo/eval_frame.py", line 655, in catch_errors
    return callback(frame, cache_entry, hooks, frame_state)
  File "/notebooks/pytorch/pytorch/torch/_dynamo/convert_frame.py", line 383, in _convert_frame_assert
    compiled_product = _compile(
  File "/notebooks/pytorch/pytorch/torch/_dynamo/convert_frame.py", line 646, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/notebooks/pytorch/pytorch/torch/_dynamo/utils.py", line 244, in time_wrapper
    r = func(*args, **kwargs)
  File "/notebooks/pytorch/pytorch/torch/_dynamo/convert_frame.py", line 562, in compile_inner
    out_code = transform_code_object(code, transform)
  File "/notebooks/pytorch/pytorch/torch/_dynamo/bytecode_transformation.py", line 1033, in transform_code_object
    transformations(instructions, code_options)
  File "/notebooks/pytorch/pytorch/torch/_dynamo/convert_frame.py", line 151, in _fn
    return fn(*args, **kwargs)
  File "/notebooks/pytorch/pytorch/torch/_dynamo/convert_frame.py", line 527, in transform
    tracer.run()
  File "/notebooks/pytorch/pytorch/torch/_dynamo/symbolic_convert.py", line 2123, in run
    super().run()
  File "/notebooks/pytorch/pytorch/torch/_dynamo/symbolic_convert.py", line 818, in run
    and self.step()
  File "/notebooks/pytorch/pytorch/torch/_dynamo/symbolic_convert.py", line 781, in step
    getattr(self, inst.opname)(inst)
  File "/notebooks/pytorch/pytorch/torch/_dynamo/symbolic_convert.py", line 2238, in RETURN_VALUE
    self.output.compile_subgraph(
  File "/notebooks/pytorch/pytorch/torch/_dynamo/output_graph.py", line 938, in compile_subgraph
    self.compile_and_call_fx_graph(tx, pass2.graph_output_vars(), root)
  File "/notebooks/conda/envs/pt-source/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/notebooks/pytorch/pytorch/torch/_dynamo/output_graph.py", line 1080, in compile_and_call_fx_graph
    compiled_fn = self.call_user_compiler(gm)
  File "/notebooks/pytorch/pytorch/torch/_dynamo/utils.py", line 244, in time_wrapper
    r = func(*args, **kwargs)
  File "/notebooks/pytorch/pytorch/torch/_dynamo/output_graph.py", line 1152, in call_user_compiler
    raise BackendCompilerFailed(self.compiler_fn, e).with_traceback(
  File "/notebooks/pytorch/pytorch/torch/_dynamo/output_graph.py", line 1133, in call_user_compiler
    compiled_fn = compiler_fn(gm, self.example_inputs())
  File "/notebooks/pytorch/pytorch/torch/_dynamo/repro/after_dynamo.py", line 117, in debug_wrapper
    compiled_gm = compiler_fn(gm, example_inputs)
  File "/notebooks/pytorch/pytorch/torch/_dynamo/repro/after_dynamo.py", line 117, in debug_wrapper
    compiled_gm = compiler_fn(gm, example_inputs)
  File "/notebooks/pytorch/pytorch/torch/__init__.py", line 1657, in __call__
    return compile_fx(model_, inputs_, config_patches=self.config)
  File "/notebooks/pytorch/pytorch/torch/_inductor/compile_fx.py", line 986, in compile_fx
    return compile_fx(
  File "/notebooks/pytorch/pytorch/torch/_inductor/compile_fx.py", line 1168, in compile_fx
    return aot_autograd(
  File "/notebooks/pytorch/pytorch/torch/_dynamo/backends/common.py", line 55, in compiler_fn
    cg = aot_module_simplified(gm, example_inputs, **kwargs)
  File "/notebooks/pytorch/pytorch/torch/_functorch/aot_autograd.py", line 5015, in aot_module_simplified
    compiled_fn = create_aot_dispatcher_function(
  File "/notebooks/pytorch/pytorch/torch/_dynamo/utils.py", line 244, in time_wrapper
    r = func(*args, **kwargs)
  File "/notebooks/pytorch/pytorch/torch/_functorch/aot_autograd.py", line 4555, in create_aot_dispatcher_function
    compiled_fn = compiler_fn(flat_fn, fake_flat_args, aot_config, fw_metadata=fw_metadata)
  File "/notebooks/pytorch/pytorch/torch/_functorch/aot_autograd.py", line 2874, in aot_wrapper_dedupe
    return compiler_fn(flat_fn, leaf_flat_args, aot_config, fw_metadata=fw_metadata)
  File "/notebooks/pytorch/pytorch/torch/_functorch/aot_autograd.py", line 3060, in aot_wrapper_synthetic_base
    return compiler_fn(flat_fn, flat_args, aot_config, fw_metadata=fw_metadata)
  File "/notebooks/pytorch/pytorch/torch/_functorch/aot_autograd.py", line 2161, in aot_dispatch_base
    compiled_fw = compiler(fw_module, updated_flat_args)
  File "/notebooks/pytorch/pytorch/torch/_dynamo/utils.py", line 244, in time_wrapper
    r = func(*args, **kwargs)
  File "/notebooks/pytorch/pytorch/torch/_inductor/compile_fx.py", line 1100, in fw_compiler_base
    return inner_compile(
  File "/notebooks/pytorch/pytorch/torch/_dynamo/repro/after_aot.py", line 83, in debug_wrapper
    inner_compiled_fn = compiler_fn(gm, example_inputs)
  File "/notebooks/pytorch/pytorch/torch/_inductor/debug.py", line 303, in inner
    return fn(*args, **kwargs)
  File "/notebooks/conda/envs/pt-source/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/notebooks/pytorch/pytorch/torch/_inductor/compile_fx.py", line 279, in compile_fx_inner
    save_args_for_compile_fx_inner(
  File "/notebooks/pytorch/pytorch/torch/_inductor/debug.py", line 520, in save_args_for_compile_fx_inner
    pickle.dump((args_to_save, kwargs_to_save), f)
  File "/notebooks/pytorch/pytorch/torch/_tensor.py", line 219, in __reduce_ex__
    func, args = self._reduce_ex_internal(proto)
  File "/notebooks/pytorch/pytorch/torch/_tensor.py", line 381, in _reduce_ex_internal
    self.data_ptr() == 0
torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
RuntimeError: Cannot call numel() on tensor with symbolic sizes/strides
Exception raised from throw_cannot_call_with_symbolic at /notebooks/pytorch/pytorch/c10/core/TensorImpl.cpp:298 (most recent call first):
frame #0: <unknown function> + 0xa610f (0x7f98edf4a10f in /notebooks/pytorch/pytorch/torch/lib/libc10.so)
frame #1: <unknown function> + 0xa5ce6 (0x7f98edf49ce6 in /notebooks/pytorch/pytorch/torch/lib/libc10.so)
frame #2: <unknown function> + 0xa5975 (0x7f98edf49975 in /notebooks/pytorch/pytorch/torch/lib/libc10.so)
frame #3: <unknown function> + 0xa6f22 (0x7f98edf4af22 in /notebooks/pytorch/pytorch/torch/lib/libc10.so)
frame #4: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x44 (0x7f98edf484d2 in /notebooks/pytorch/pytorch/torch/lib/libc10.so)
frame #5: <unknown function> + 0x5c4d0 (0x7f98edf004d0 in /notebooks/pytorch/pytorch/torch/lib/libc10.so)
frame #6: c10::TensorImpl::throw_cannot_call_with_symbolic(char const*) const + 0x94 (0x7f98edf0b0f0 in /notebooks/pytorch/pytorch/torch/lib/libc10.so)
frame #7: <unknown function> + 0x6d25b (0x7f98edf1125b in /notebooks/pytorch/pytorch/torch/lib/libc10.so)
frame #8: c10::TensorImpl::numel_custom() const + 0x6d (0x7f98edf0b987 in /notebooks/pytorch/pytorch/torch/lib/libc10.so)
frame #9: <unknown function> + 0x515067 (0x7f98ec595067 in /notebooks/pytorch/pytorch/torch/lib/libtorch_python.so)
frame #10: <unknown function> + 0x515be6 (0x7f98ec595be6 in /notebooks/pytorch/pytorch/torch/lib/libtorch_python.so)
frame #11: <unknown function> + 0x52e813 (0x7f98ec5ae813 in /notebooks/pytorch/pytorch/torch/lib/libtorch_python.so)
frame #12: <unknown function> + 0x515a72 (0x7f98ec595a72 in /notebooks/pytorch/pytorch/torch/lib/libtorch_python.so)
frame #13: <unknown function> + 0x517064 (0x7f98ec597064 in /notebooks/pytorch/pytorch/torch/lib/libtorch_python.so)
frame #14: <unknown function> + 0x517082 (0x7f98ec597082 in /notebooks/pytorch/pytorch/torch/lib/libtorch_python.so)
frame #15: <unknown function> + 0x445b4f (0x7f98ec4c5b4f in /notebooks/pytorch/pytorch/torch/lib/libtorch_python.so)
frame #16: python() [0x4ff2f4]
<omitting python frames>
frame #20: python() [0x507eae]
frame #21: <unknown function> + 0xb3ce (0x7f98ede443ce in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #22: <unknown function> + 0x993b (0x7f98ede4293b in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #23: <unknown function> + 0xa591 (0x7f98ede43591 in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #24: <unknown function> + 0xb95a (0x7f98ede4495a in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #25: <unknown function> + 0x997a (0x7f98ede4297a in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #26: <unknown function> + 0xa1b8 (0x7f98ede431b8 in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #27: <unknown function> + 0xa591 (0x7f98ede43591 in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #28: <unknown function> + 0xb95a (0x7f98ede4495a in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #29: <unknown function> + 0x997a (0x7f98ede4297a in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #30: <unknown function> + 0xa591 (0x7f98ede43591 in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #31: <unknown function> + 0xb95a (0x7f98ede4495a in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #32: <unknown function> + 0x997a (0x7f98ede4297a in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #33: <unknown function> + 0xa41b (0x7f98ede4341b in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #34: <unknown function> + 0xb95a (0x7f98ede4495a in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #35: <unknown function> + 0x997a (0x7f98ede4297a in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #36: <unknown function> + 0x9df1 (0x7f98ede42df1 in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #37: <unknown function> + 0x973e (0x7f98ede4273e in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #38: <unknown function> + 0x973e (0x7f98ede4273e in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #39: <unknown function> + 0x15ca5 (0x7f98ede4eca5 in /notebooks/conda/envs/pt-source/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so)
frame #40: python() [0x4f4abb]
frame #52: python() [0x5c5298]
frame #57: python() [0x5c5298]
```

### Minified repro

Here's the script I running, essentially copying key sections from [test/dynamo/test_functions.py](https://github.com/pytorch/pytorch/blob/83cb6a75ade05a7d9cd1e94aac24ef39cc66088a/test/dynamo/test_functions.py#L1870-L1898)
```
import os
import logging

os.environ["TORCHINDUCTOR_MAX_AUTOTUNE"] = "1"
os.environ["TORCH_COMPILE_DEBUG"] = "1"
os.environ["TORCHINDUCTOR_SAVE_ARGS"] = "1"

import triton
import triton.language as tl

import torch

torch._inductor.config.cpp_wrapper = False
torch._inductor.config.triton.store_cubin = True
torch._inductor.config.trace.enabled = True

torch._logging.set_logs(all=logging.DEBUG)

@triton.jit
def mul2_kernel(
    in_ptr0,
    out_ptr,
    n_elements,
    BLOCK_SIZE: "tl.constexpr",
):
    pid = tl.program_id(axis=0)
    block_start = pid * BLOCK_SIZE
    offsets = block_start + tl.arange(0, BLOCK_SIZE)
    mask = offsets < n_elements
    x = tl.load(in_ptr0 + offsets, mask=mask)
    output = 2 * x
    tl.store(out_ptr + offsets, output, mask=mask)

def call_triton_take_view(x: torch.Tensor):
    output = torch.zeros_like(x)
    n_elements = output.numel()
    grid = lambda meta: (triton.cdiv(n_elements, meta["BLOCK_SIZE"]),)
    mul2_kernel[grid](x, output, n_elements, BLOCK_SIZE=16)
    return output

def call_triton_return_view(x: torch.Tensor):
    output = torch.zeros_like(x)
    n_elements = output.numel()
    grid = lambda meta: (triton.cdiv(n_elements, meta["BLOCK_SIZE"]),)
    mul2_kernel[grid](x, output, n_elements, BLOCK_SIZE=16)
    return output.view(4, 4)

t = torch.rand(4, 4, device="cuda")
t_view = t.view(16)

backend = "inductor"
dynamic = True

with torch.no_grad():
    compiled_func = torch.compile(
        call_triton_take_view, backend=backend, fullgraph=True, dynamic=dynamic
    )

compiled_func(t_view)
```

### Versions

Collecting environment information...
PyTorch version: 2.2.0a0+gitf128616
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.2 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0
Clang version: Could not collect
CMake version: version 3.27.7
Libc version: glibc-2.35

Python version: 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.19.0-45-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA RTX A6000
Nvidia driver version: 525.116.04
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.9.2
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.9.2
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.9.2
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.9.2
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.9.2
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.9.2
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.9.2
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Address sizes:                   46 bits physical, 48 bits virtual
Byte Order:                      Little Endian
CPU(s):                          8
On-line CPU(s) list:             0-7
Vendor ID:                       GenuineIntel
Model name:                      Intel(R) Xeon(R) Gold 5315Y CPU @ 3.20GHz
CPU family:                      6
Model:                           106
Thread(s) per core:              1
Core(s) per socket:              8
Socket(s):                       1
Stepping:                        6
BogoMIPS:                        6400.19
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush acpi mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rtm rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves umip pku ospke gfni vaes vpclmulqdq rdpid md_clear flush_l1d
Hypervisor vendor:               Xen
Virtualization type:             full
L1d cache:                       384 KiB (8 instances)
L1i cache:                       256 KiB (8 instances)
L2 cache:                        10 MiB (8 instances)
L3 cache:                        96 MiB (8 instances)
NUMA node(s):                    1
NUMA node0 CPU(s):               0-7
Vulnerability Itlb multihit:     KVM: Mitigation: VMX unsupported
Vulnerability L1tf:              Mitigation; PTE Inversion
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Mmio stale data:   Mitigation; Clear CPU buffers; SMT Host state unknown
Vulnerability Retbleed:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Mitigation; Clear CPU buffers; SMT Host state unknown

Versions of relevant libraries:
[pip3] numpy==1.26.2 (/notebooks/conda/envs/pt-source/lib/python3.10/site-packages)
[pip3] optree==0.10.0 (/notebooks/conda/envs/pt-source/lib/python3.10/site-packages)
[pip3] pytorch-triton==2.1.0+6e4932cda8 (/notebooks/conda/envs/pt-source/lib/python3.10/site-packages)
[pip3] torch==2.2.0a0+gitf128616 (/notebooks/pytorch/pytorch)
[conda] magma-cuda118             2.6.1                         1    pytorch
[conda] mkl                       2023.1.0         h213fc3f_46344  
[conda] mkl-include               2023.1.0         h06a4308_46344  
[conda] numpy                     1.26.2                   pypi_0    pypi
[conda] optree                    0.10.0                   pypi_0    pypi
[conda] pytorch-triton            2.1.0+6e4932cda8          pypi_0    pypi
[conda] torch                     2.2.0a0+gitf128616           dev_0    <develop>


cc @ezyang @anijain2305 @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @msaroufim @wconstab @bdhirsh @zou3519