Repository: pytorch
Issue: [compiled autograd][aot autograd] accumulate grad (on param with non empty grad) mutates inputs and prevents cudagraph
Labels: triaged,module: cuda graphs,oncall: pt2,module: aotdispatch,module: compiled autograd,module: pt2-dispatcher

### 🐛 Describe the bug

For common use cases, I think users would apply the optimizer each run which would clear the grads, and not run into this problem. 

In polyfill, we decompose accumulate grad: https://github.com/pytorch/pytorch/blob/main/torch/_dynamo/polyfill.py#L43-L48

graph when there is a pre-existing grad
```python
DEBUG:torch._dynamo.output_graph.__graph_code:TRACED GRAPH                                                                                                                                                                                                                           
 ===== __compiled_fn_5 =====                                                                                                                                                                                                                                                         
 /home/xmfan/pytorch/torch/fx/_lazy_graph_module.py class GraphModule(torch.nn.Module):                                                                                                                                                                                              
    def forward(self, L_inputs_ : list, L_sizes_0_: "Sym(s0)"):                                                                                                                                                                                                                      
        ...
        # File: /home/xmfan/pytorch/torch/_dynamo/polyfill.py:44 in accumulate_grad, code: new_grad = torch.clone(new_grad)
        new_grad_1: "f32[1, 10]" = torch.clone(t_2);  t_2 = None
         
        # File: /home/xmfan/pytorch/torch/_dynamo/polyfill.py:48 in accumulate_grad, code: x.grad.add_(new_grad)
        add__1: "f32[1, 10]" = getitem_3.add_(new_grad_1);  getitem_3 = new_grad_1 = None
        return ()
```

Then AOT Autograd tries to functionalize it:
```python
 ===== Forward graph 2 =====
 /home/xmfan/pytorch/torch/fx/_lazy_graph_module.py class <lambda>(torch.nn.Module):
    def forward(self, arg0_1: "f32[]", arg1_1: "f32[s0, 10]", arg2_1: "f32[1, 10]", arg3_1: "f32[1, 10]", arg4_1: "f32[1]", arg5_1: "f32[1]", arg6_1: "Sym(s0)"):
        ...
        add_1: "f32[1, 10]" = torch.ops.aten.add.Tensor(arg3_1, clone_1);  clone_1 = None
        copy_: "f32[1, 10]" = torch.ops.aten.copy_.default(arg3_1, add_1);  arg3_1 = add_1 = None
       ...
        return ()
```

But keeps copy_ in the graph! This is problematic if we want to cudagraph this graph, since there are still mutations on inputs (on parameters in this case)

I'm a bit conflicted on which approach we should take here. If we move the copy_ outside of the graph, we'll be able to cudagraph fine, but this will hurt our future plans of fusing backward + optimizer.

### Versions

main

cc @mcarilli @ezyang @eellison @peterbell10 @msaroufim @bdhirsh @anijain2305 @chauhang @zou3519