Dynamo generates invalid frame when graph-breaking due to opacus_cifar10 hooks
Labels: triaged, bug, module: dynamo, release notes: dynamo

### 🐛 Describe the bug

When running torchbench opacus_cifar10 today in CI, hooks are silently ignored on modules such as `torch.nn.modules.conv.Conv2d` since this is 'allowed' in graph and hooks aren't traced by dynamo.  Some of the hook behavior might actually be burned into the AOT trace, and this is at best shaky or more likely incorrect (I didn't read carefully what the hooks do but it's not trivial) - [capture_activations_hook](https://github.com/pytorch/opacus/blob/main/opacus/grad_sample/grad_sample_module.py#L267), [capture_backprops_hook](https://github.com/pytorch/opacus/blob/main/opacus/grad_sample/grad_sample_module.py#L290)

There is a related gh issue (#96722) where dynamo skips hooks on 'allowed' modules.  PR #97184 fixes this by graph-breaking on allowed_modules that have hooks.  See the PR desc for more discussion on alternative fixes.

After graph-breaking on allowed modules with hooks, opacus fails with what looks like an invalid dynamo frame generated.

Checkout: `ghstack checkout https://github.com/pytorch/pytorch/pull/97184`
Repro:
`TORCH_COMPILE_DEBUG=1 python benchmarks/dynamo/torchbench.py --only opacus_cifar10 --performance --backend aot_eager > opacus.log 2>&1`

Log: https://gist.github.com/wconstab/50e752cc4a6db010549c2f5cdfff67ac

**Notable excerpts from log:**

This graphbreak is expected, due to the change in #97184

```
[2023-03-27 23:34:30,451] torch._dynamo.output_graph: [DEBUG] COMPILING GRAPH due to GraphCompileReason(reason="Can't support hooks on 'allowed' modules (<class 'torch.nn.modules.conv.Conv2d'>) becuase allowed modules don't get traced through.",
```

Not sure why getattr fails here, unless it's becuase the hook didn't modify the dynamo's copy of the module and the hook is actually mutating the module via setattr.  (The hook does actually store stuff on the module) 

```
[2023-03-27 23:34:30,456] torch._dynamo.symbolic_convert: [DEBUG] FAILED INLINING <code object __getattr__ at 0x7ff33221e3a0, file "/scratch/whc/work/pytorch/torch/nn/modules/module.py", line 1601>
[2023-03-27 23:34:30,456] torch._dynamo.output_graph: [DEBUG] restore_graphstate: removed 0 nodes
[2023-03-27 23:34:30,457] torch._dynamo.symbolic_convert: [DEBUG] step triggered compile
```

?
```
torch._dynamo.exc.Unsupported: class property GradSampleModule getset_descriptor
[2023-03-27 23:34:30,457] torch._dynamo.output_graph: [DEBUG] restore_graphstate: removed 0 nodes
[2023-03-27 23:34:30,457] torch._dynamo.output_graph: [DEBUG] COMPILING GRAPH due to GraphCompileReason(reason='step_unsupported', user_stack=[<FrameSummary file /scratch/whc/work/py38/lib/python3.8/site-packages/opacus/grad_sample/gsm_base.py, line 71 in __getattr__>])
ERROR:common:Backend dynamo failed in warmup()
```

And finally we bottom out with a stackframe that's invalid.
```
  File "/scratch/whc/work/pytorch/torch/_dynamo/bytecode_analysis.py", line 185, in stacksize_analysis
    assert low >= 0
AssertionError: 
```



### Versions

PR (see above)

cc @soumith @voznesenskym @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire