Repository: pytorch
Issue: dataclasses.replace not supported by dynamo
Labels: good first issue,triaged,oncall: pt2,module: dynamo,vllm-compile

### 🐛 Describe the bug

The `dataclasses.replace` function appears to be unimplemented in dynamo.   In this particular case it is used from the xformers (0.0.27.post2) package in `xformers/ops/fmha/cutlass.py:259`.  I'm using the pytorch 2.5 nightly (2.4 fails in a different spot).

It can be reproduced with the following steps:

1. Save to `bug.py`
```
import torch
from vllm import LLM, SamplingParams

# Sample prompts.                                                                                                                 
prompts = [
    "Hello, my name is",
    "The president of the United States is",
    "The capital of France is",
    "The future of AI is",
]
# Create a sampling params object.                                                                                                
sampling_params = SamplingParams(temperature=0.8, top_p=0.95)

# Create an LLM.                                                                                                                  
llm = LLM(model="bigcode/tiny_starcoder_py", enforce_eager=True, dtype=torch.float32)

# Generate texts from the prompts. The output is a list of RequestOutput objects                                                  
# that contain the prompt, generated text, and other information.                                                                 
outputs = llm.generate(prompts, sampling_params)
# Print the outputs.                                                                                                              
for output in outputs:
    prompt = output.prompt
    generated_text = output.outputs[0].text
    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")
```

2. Download and build VLLM version/hash: 0.6.1.post2, `b05f5c923`
3. Run the following command.

```
VLLM_TEST_DYNAMO_GRAPH_CAPTURE=1 VLLM_TEST_DYNAMO_FULLGRAPH_CAPTURE=1 python3 bug.py
```

Stacktrace:
```
E           torch._dynamo.exc.Unsupported: Error in model execution (input dumped to /tmp/err_execute_model_input_20240924-020006.pkl): 'skip function replace in file /usr/lib/python3.10/dataclasses.py'
E
E           from user code:
E              File "/home/bnell/nm-vllm-new/vllm/model_executor/models/gpt_bigcode.py", line 284, in forward
E               hidden_states = self.transformer(input_ids, positions, kv_caches,
E             File "/home/bnell/nm-vllm-new/vllm/model_executor/models/gpt_bigcode.py", line 229, in forward
E               hidden_states = layer(hidden_states, kv_caches[i], attn_metadata)
E             File "/home/bnell/nm-vllm-new/vllm/model_executor/models/gpt_bigcode.py", line 173, in forward
E               attn_output = self.attn(
E             File "/home/bnell/nm-vllm-new/vllm/model_executor/models/gpt_bigcode.py", line 110, in forward
E               attn_output = self.attn(q, k, v, kv_cache, attn_metadata)
E             File "/home/bnell/nm-vllm-new/vllm/attention/layer.py", line 98, in forward
E               return self.impl.forward(query,
E             File "/home/bnell/nm-vllm-new/vllm/attention/backends/xformers.py", line 595, in forward
E               out = self._run_memory_efficient_xformers_forward(
E             File "/home/bnell/nm-vllm-new/vllm/attention/backends/xformers.py", line 739, in _run_memory_efficient_xformers_forward
E               out = xops.memory_efficient_attention_forward(
E             File "/home/bnell/pt24/lib/python3.10/site-packages/xformers/ops/fmha/__init__.py", line 304, in memory_efficient_attention_forward
E               return _memory_efficient_attention_forward(
E             File "/home/bnell/pt24/lib/python3.10/site-packages/xformers/ops/fmha/__init__.py", line 418, in _memory_efficient_attention_forward
E               out, *_ = op.apply(inp, needs_gradient=False)
E             File "/home/bnell/pt24/lib/python3.10/site-packages/xformers/ops/fmha/cutlass.py", line 259, in apply
E               replace(inp, query=query, key=key, value=value, attn_bias=bias),
E
E           Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
E
E
E           You can suppress this exception and fall back to eager by setting:
E               import torch._dynamo
E               torch._dynamo.config.suppress_errors = True
```

### Versions

```
Collecting environment information...
PyTorch version: 2.5.0+cu124
Is debug build: False
CUDA used to build PyTorch: 12.4
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.4 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: version 3.30.1
Libc version: glibc-2.35

Python version: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] (64-bit runtime)
Python platform: Linux-6.5.0-35-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 12.5.82
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: 
GPU 0: NVIDIA H100 80GB HBM3
GPU 1: NVIDIA H100 80GB HBM3
GPU 2: NVIDIA H100 80GB HBM3
GPU 3: NVIDIA H100 80GB HBM3
GPU 4: NVIDIA H100 80GB HBM3
GPU 5: NVIDIA H100 80GB HBM3
GPU 6: NVIDIA H100 80GB HBM3
GPU 7: NVIDIA H100 80GB HBM3
  
Nvidia driver version: 555.42.02
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      46 bits physical, 57 bits virtual
Byte Order:                         Little Endian
CPU(s):                             128
On-line CPU(s) list:                0-127
Vendor ID:                          GenuineIntel
Model name:                         Intel(R) Xeon(R) Platinum 8462Y+
CPU family:                         6
Model:                              143
Thread(s) per core:                 2
Core(s) per socket:                 32
Socket(s):                          2
Stepping:                           8
CPU max MHz:                        4100.0000
CPU min MHz:                        800.0000
BogoMIPS:                           5600.00
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx \
fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_ts\
c cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4\
_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cat_l\
2 cdp_l3 invpcid_single intel_ppin cdp_l2 ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase\
 tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx51\
2cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local split_lock_detect a\
vx_vnni avx512_bf16 wbnoinvd dtherm ida arat pln pts hfi vnmi avx512vbmi umip pku ospke waitpkg avx512_vbmi2 gfni vaes vpclmulqdq\
 avx512_vnni avx512_bitalg tme avx512_vpopcntdq la57 rdpid bus_lock_detect cldemote movdiri movdir64b enqcmd fsrm md_clear serial\
ize tsxldtrk pconfig arch_lbr ibt amx_bf16 avx512_fp16 amx_tile amx_int8 flush_l1d arch_capabilities
Virtualization:                     VT-x
L1d cache:                          3 MiB (64 instances)
L1i cache:                          2 MiB (64 instances)
L2 cache:                           128 MiB (64 instances)
L3 cache:                           120 MiB (2 instances)
NUMA node(s):                       2
NUMA node0 CPU(s):                  0-31,64-95
NUMA node1 CPU(s):                  32-63,96-127
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Enhanced / Automatic IBRS; IBPB conditional; RSB filling; PBRSB-eIBRS SW sequence\
; BHI BHI_DIS_S
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Versions of relevant libraries:
[pip3] flashinfer==0.0.9+cu121torch2.3
[pip3] mypy==1.9.0
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.26.4
[pip3] onnx==1.14.1
[pip3] onnxruntime==1.18.1
[pip3] pytorch-triton==3.1.0+5fe38ffd73
[pip3] torch==2.5.0+cu124
[pip3] torchaudio==2.5.0.dev20240919+cu121
[pip3] torchvision==0.20.0.dev20240919+cu121
[pip3] triton==3.0.0
[conda] Could not collect
```

cc @ezyang @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @amjames @rec @zou3519