NameError: name 's1' is not defined
Labels: needs reproduction, triaged, oncall: pt2, module: dynamic shapes, module: inductor

### 🐛 Describe the bug

I am running a DDP job on 8 GPUs. torch.compile fails with the following error.
Single GPU compilation runs fine.

### Error logs

```
    File "/usr/local/lib/python3.8/dist-packages/torch/fx/graph_module.py", line 284, in __call__
    raise e
  File "/usr/local/lib/python3.8/dist-packages/torch/fx/graph_module.py", line 274, in __call__
    return super(self.cls, obj).__call__(*args, **kwargs)  # type: ignore[misc]
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "<eval_with_key>.191", line 11, in forward
    submod_1 = self.compiled_submod_1(getitem, getitem_1, l_self_pos_enc_class_pe);  getitem = getitem_1 = l_self_pos_enc_class_pe = None
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/_dynamo/backends/distributed.py", line 335, in forward
    x = self.submod(*args)
  File "/usr/local/lib/python3.8/dist-packages/torch/_dynamo/eval_frame.py", line 328, in _fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/_dynamo/external_utils.py", line 17, in inner
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/_functorch/aot_autograd.py", line 3905, in forward
    return compiled_fn(full_args)
  File "/usr/local/lib/python3.8/dist-packages/torch/_functorch/aot_autograd.py", line 1482, in g
    return f(*args)
  File "/usr/local/lib/python3.8/dist-packages/torch/_functorch/aot_autograd.py", line 2527, in runtime_wrapper
    all_outs = call_func_with_args(
  File "/usr/local/lib/python3.8/dist-packages/torch/_functorch/aot_autograd.py", line 1506, in call_func_with_args
    out = normalize_as_list(f(args))
  File "/usr/local/lib/python3.8/dist-packages/torch/_functorch/aot_autograd.py", line 1482, in g
    return f(*args)
  File "/usr/local/lib/python3.8/dist-packages/torch/autograd/function.py", line 539, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/usr/local/lib/python3.8/dist-packages/torch/_functorch/aot_autograd.py", line 3010, in forward
    fw_outs = call_func_with_args(
  File "/usr/local/lib/python3.8/dist-packages/torch/_functorch/aot_autograd.py", line 1506, in call_func_with_args
    out = normalize_as_list(f(args))
  File "/usr/local/lib/python3.8/dist-packages/torch/_inductor/codecache.py", line 374, in __call__
    return self.get_current_callable()(inputs)
  File "/usr/local/lib/python3.8/dist-packages/torch/_inductor/compile_fx.py", line 628, in run
    return model(new_inputs)
  File "/usr/local/lib/python3.8/dist-packages/torch/_inductor/codecache.py", line 401, in _run_from_cache
    return compiled_graph.compiled_artifact(inputs)
  File "/tmp/torchinductor_root/qe/cqecvn34iq32lm3fql55scrqfzzt5hcnj4g2gnpyqjgnpdbaf527.py", line 161, in call
    assert_size_stride(primals_3, (8, (s1 // 6), 64*(s2 // 6)), (64*(s1 // 6)*(s2 // 6), 64*(s2 // 6), 1))
NameError: name 's1' is not defined
```

### Minified repro
TORCHDYNAMO_REPRO_AFTER="aot"
doesn't seem to do anything

TORCHDYNAMO_REPRO_AFTER="dynamo" generates minifier_launcher.py which fails:
```
  File "/usr/local/lib/python3.8/dist-packages/torch/_dynamo/output_graph.py", line 957, in compile_and_call_fx_graph
    compiled_fn = self.call_user_compiler(gm)
  File "/usr/local/lib/python3.8/dist-packages/torch/_dynamo/utils.py", line 189, in time_wrapper
    r = func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/torch/_dynamo/output_graph.py", line 1024, in call_user_compiler
    raise BackendCompilerFailed(self.compiler_fn, e).with_traceback(
  File "/usr/local/lib/python3.8/dist-packages/torch/_dynamo/output_graph.py", line 1009, in call_user_compiler
    compiled_fn = compiler_fn(gm, self.example_inputs())
  File "/usr/local/lib/python3.8/dist-packages/torch/_dynamo/repro/after_dynamo.py", line 117, in debug_wrapper
    compiled_gm = compiler_fn(gm, example_inputs)
  File "/usr/local/lib/python3.8/dist-packages/torch/_dynamo/repro/after_dynamo.py", line 288, in dynamo_minifier_backend
    minifier(
  File "/usr/local/lib/python3.8/dist-packages/torch/_functorch/fx_minifier.py", line 189, in minifier
    raise RuntimeError("Input graph did not fail the tester")
torch._dynamo.exc.BackendCompilerFailed: backend='debug_wrapper' raised:
RuntimeError: Input graph did not fail the tester
```

### Versions

Versions of relevant libraries:
[pip3] numpy==1.24.4
[pip3] pytorch-triton==2.1.0+e6216047b8
[pip3] torch==2.1.0.dev20230825+cu118
[pip3] torchaudio==2.1.0.dev20230825+cu118
[pip3] torchdata==0.7.0.dev20230825
[pip3] torchtext==0.16.0.dev20230826+rocm5.6
[pip3] torchvision==0.16.0.dev20230826+cu118
[pip3] triton==2.0.0

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov