Repository: pytorch
Issue: llama model failed for dynamic shape path
Labels: needs reproduction,triaged,oncall: pt2,module: dynamic shapes

### 🐛 Describe the bug

When running the dynamic shape path for the Llama model, there report an assert error:

```
ERROR:common:Backend dynamo failed in warmup()
Traceback (most recent call last):
  File "/home/xiaobing/pytorch-offical/benchmarks/dynamo/common.py", line 2226, in warmup
    fn(model, example_inputs)
  File "/home/xiaobing/pytorch-offical/torch/_dynamo/eval_frame.py", line 310, in _fn
    return fn(*args, **kwargs)
  File "/home/xiaobing/pytorch-offical/torch/_dynamo/eval_frame.py", line 470, in catch_errors
    return callback(frame, cache_size, hooks, frame_state)
  File "/home/xiaobing/pytorch-offical/torch/_dynamo/convert_frame.py", line 559, in _convert_frame
    result = inner_convert(frame, cache_size, hooks, frame_state)
  File "/home/xiaobing/pytorch-offical/torch/_dynamo/convert_frame.py", line 130, in _fn
    return fn(*args, **kwargs)
  File "/home/xiaobing/pytorch-offical/torch/_dynamo/convert_frame.py", line 378, in _convert_frame_assert
    return _compile(
  File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 194, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/xiaobing/pytorch-offical/torch/_dynamo/convert_frame.py", line 506, in _compile
    check_fn = CheckFunctionManager(
  File "/home/xiaobing/pytorch-offical/torch/_dynamo/guards.py", line 887, in __init__
    guard.create(local_builder, global_builder)
  File "/home/xiaobing/pytorch-offical/torch/_guards.py", line 216, in create
    return self.create_fn(self.source.select(local_builder, global_builder), self)
  File "/home/xiaobing/pytorch-offical/torch/_dynamo/guards.py", line 564, in SHAPE_ENV
    guards = output_graph.shape_env.produce_guards(
  File "/home/xiaobing/pytorch-offical/torch/fx/experimental/symbolic_shapes.py", line 2809, in produce_guards
    raise ConstraintViolationError(f"Constraints violated!\n{err}")
torch.fx.experimental.symbolic_shapes.ConstraintViolationError: Constraints violated!
  1. Could not validate constraint RelaxedUnspecConstraint(L['inputs'][0].size()[0]) as L['inputs'][0].size()[0] was inferred to be constant (32).  For more information about why it is constant, run with TORCH_LOGS=dynamic

```

### Error logs

_No response_

### Minified repro

```
python -m torch.backends.xeon.run_cpu --node_id 0 benchmarks/dynamo/torchbench.py --performance --float32 -dcpu --inference -n5 --inductor --timeout 9000 --dynamic-shapes --dynamic-batch-only --only llama

```

### Versions

Collecting environment information...
PyTorch version: 2.1.0a0+gitdcacff5
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.5 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.22.1
Libc version: glibc-2.31

Python version: 3.9.16 (main, Mar  8 2023, 14:00:05)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.13.0-30-generic-x86_64-with-glibc2.31
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] bert-pytorch==0.0.1a4
[pip3] clip-anytorch==2.5.2
[pip3] CoCa-pytorch==0.0.7
[pip3] dalle2-pytorch==1.12.4
[pip3] ema-pytorch==0.2.2
[pip3] functorch==1.14.0a0+b71aa0b
[pip3] intel-extension-for-pytorch==2.1.0+git785443c
[pip3] mypy==1.4.1
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.24.3
[pip3] open-clip-torch==2.20.0
[pip3] pytorch-transformers==1.2.0
[pip3] pytorch-triton==2.1.0+9e3e10c5ed
[pip3] pytorch-warmup==0.1.1
[pip3] rotary-embedding-torch==0.2.1
[pip3] torch==2.1.0a0+git0391d8d
[pip3] torch-fidelity==0.3.0
[pip3] torch-scatter==2.1.1+pt20cpu
[pip3] torch-sparse==0.6.17+pt20cpu
[pip3] torch-struct==0.5
[pip3] torchaudio==2.1.0a0+d5b2996
[pip3] torchdata==0.7.0a0+f2bfd3d
[pip3] torchmetrics==0.11.4
[pip3] torchrec-nightly==2023.3.23
[pip3] torchtext==0.16.0a0+60bea66
[pip3] torchvision==0.16.0a0+d24db8c
[pip3] vector-quantize-pytorch==1.1.2
[conda] bert-pytorch              0.0.1a4                   dev_0    <develop>
[conda] clip-anytorch             2.5.2                    pypi_0    pypi
[conda] coca-pytorch              0.0.7                    pypi_0    pypi
[conda] dalle2-pytorch            1.12.4                   pypi_0    pypi
[conda] ema-pytorch               0.2.2                    pypi_0    pypi
[conda] functorch                 1.14.0a0+b71aa0b          pypi_0    pypi
[conda] intel-extension-for-pytorch 2.1.0+git785443c          pypi_0    pypi
[conda] mkl                       2023.0.0         h6d00ec8_25399
[conda] mkl-include               2023.1.0                 pypi_0    pypi
[conda] mkl-static                2023.1.0                 pypi_0    pypi
[conda] numpy                     1.24.3                   pypi_0    pypi
[conda] open-clip-torch           2.20.0                   pypi_0    pypi
[conda] pytorch-transformers      1.2.0                    pypi_0    pypi
[conda] pytorch-triton            2.1.0+9e3e10c5ed          pypi_0    pypi
[conda] pytorch-warmup            0.1.1                    pypi_0    pypi
[conda] rotary-embedding-torch    0.2.1                    pypi_0    pypi
[conda] torch                     2.1.0a0+git0391d8d           dev_0    <develop>
[conda] torch-fidelity            0.3.0                    pypi_0    pypi
[conda] torch-scatter             2.1.1+pt20cpu            pypi_0    pypi
[conda] torch-sparse              0.6.17+pt20cpu           pypi_0    pypi
[conda] torch-struct              0.5                      pypi_0    pypi
[conda] torchaudio                2.1.0a0+d5b2996          pypi_0    pypi
[conda] torchdata                 0.7.0a0+f2bfd3d          pypi_0    pypi
[conda] torchmetrics              0.11.4                   pypi_0    pypi
[conda] torchrec-nightly          2023.3.23                pypi_0    pypi
[conda] torchtext                 0.16.0a0+60bea66          pypi_0    pypi
[conda] torchvision               0.16.0a0+d24db8c          pypi_0    pypi
[conda] vector-quantize-pytorch   1.1.2                    pypi_0    pypi


cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng @ipiszy