Repository: pytorch
Issue: Distinguish immutable/mutable fake tensor mode
Labels: triaged,module: fakeTensor,module: dynamo

### 🐛 Describe the bug

@voznesenskym has been trying to get metadata mutation working in PT2 (see https://github.com/pytorch/pytorch/pull/113605) and one of the problems is that (1) we encode nontrivial information in the initial fakeification of tensors (e.g., which dimensions are dynamic, symbolic shapes constraints that arose from Dynamo tracing), but then we (2) destroy the fake tensors when we do metadata mutation, when we need these same initial fake tensors when we end up running the backend. In general, there is an awkwardness where fake tensor to real tensor memoization allows you to consistently map a real tensor to a single fake tensor we created for it... but that fake tensor may be inconsistent with the state of the real tensor (because you mutated its metadata). 

There are a few different ways of resolving this problem, but here is one that involves extending the API of fake tensor mode in (what I think) is a logical way. We now distinguish between two types of fake tensor mode:

* An immutable fake tensor mode generates fake tensors whose metadata cannot be mutated. If you attempt to mutate them (e.g., using `set_` or `t_`, etc.) we raise an error (new capability). When mapping from real tensors to fake tensors, you prefer to perform this mapping with an immutable fake tensor mode.
* A mutable fake tensor mode relaxes this restriction, allowing for metadata mutation on fake tensors. When performing fake tensor propagation, you prefer to use a mutable fake tensor mode.

Crucially, a mutable fake tensor mode can be created from an immutable fake tensor mode, (lazily) deep cloning and preserving sharing all the associated fake tensors. In fact, we will persistently chain a mutable fake tensor to an immutable fake tensor mode, so that we can incrementally fakeify more real tensors and have them reflect in both modes.

Dynamo makes use of the fake tensor modes in the following way:

* At initialization time, it allocates both an immutable fake tensor mode (to record the mapping between real tensor and fake tensor), and a mutable fake tensor mode which it registers as the ambient fake tensor mode for Dynamo tracing.
* To fakeify a tensor, we call the mutable fake tensor mode as is. This involves a memo table miss, so instead of directly creating a new fake tensor, it forwards the fakeification to the underlying immutable fake tensor mode. That will return a new fake tensor, which we then deep copy preserving aliasing into the mutable fake tensor mode and return. Allowing calls to the fake tensor mode minimizes code changes needed in Dynamo.
* When we reach the backend, or any situation where we need to reset the state of fake tensors to their initial state, we chain a new mutable fake tensor mode to the original immutable fake tensor mode. The new mutable fake tensor mode will preserve the initial fakeification properties, because we are performing a deep copy.

cc @eellison @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng 

### Versions

main