gopikrsmscs/torch-issues huggingface dataset:
[dynamo] Better determinism of `ConfigModule` by walking using pytree

### 🚀 The feature, motivation and pitch

https://github.com/pytorch/pytorch/pull/111318

Currently, validation only occurs at the root. However, we should walk the pytree of each object to ensure types are respected.

In particular, we can do conversions of unfriendly types
- function objects into `"f{__module__}{__name__}"` strings
- sort sets and walk their objects to make them deterministic. 
    - There isn't a canonical order, however; sets have `__hash__`, but this hash is based on `id` which is non-deterministic. So sorting by hash is non-deterministic. Further, not all objects implement `>` operator. 
    - In other words, there is no feasible way to make deterministic representations of sets.
[dynamo] Expand _nonvar_fields names
Labels: module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111749

This should be a small compile time optimization, since we won't need to
walk these fields in apply().

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Allow to specify specific files for debug info
Labels: topic: not user facing

Building with `USE_CUSTOM_DEBINFO=torch/csrc/Module.cpp python setup.py develop` for example will provide debug info only for this file.
This allows to enable debug symbols very fast from a non-debug build by doing a clean then develop (as long as you have ccache) and avoid very large binaries that take a very long time to load in gdb.
New swap function
Labels: module: dynamo

This PR is proposing a new approach to solve the nn/optim only linked by python object identity problem.
The idea is to have a function that can swap the content of two Tensors t1 and t2 while preserving all the old references.
This would allow us to swap the `model.weight` with a new Tensor (can be any subclass of Tensor and any TensorImpl (xla, sparse, nested tensorimpl would work)). The use within nn will be done in a follow up.

This is done by swapping the whole content of the PyObject and then putting back the fields associated with external references (refcount, gc tracking and weakrefs).
Note that we have to properly handle all the cases where there is memory used before the public pointer PyObject* and where the PyObject is bigger due to dict/weakref being inlined (older CPython version) or due to slots.

The main limitation of this approach is that the number of slots need to match for the objects being swapped and thus limit usage of slots in subclasses.

Draft right now to see what @colesbury thinks about doing this?

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
[dynamo] add repro for functorch/fx interop issue (`allow_in_graph`)
Labels: open source, topic: not user facing, module: dynamo

Fixes https://github.com/pytorch/pytorch/issues/109025 by adding repro


cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
[dynamo]: `nn.Module` recursively set `training` mode via `train` and `eval`
Labels: open source, topic: not user facing, module: dynamo, ciflow/inductor

Fixes https://github.com/pytorch/pytorch/issues/109885

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Add CSR tensor with non-contiguous values support to CuSparseSpMatCsrDescriptor
Labels: module: sparse, open source, release notes: sparse, topic: new features

Fixes https://github.com/pytorch/pytorch/issues/111574

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111742



cc @alexsamardzic @nikitaved @cpuhrsch @amjames @bhosmer
[dynamo] `{*}Tensor.__init__` from list of ndarray as `torch.stack(List[FakeTensor])`
Labels: open source, module: dynamo, ciflow/inductor

Follow up to https://github.com/pytorch/pytorch/pull/111665

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng @lezcano
GPU computation is not equivalent

### 🐛 Describe the bug

GPU computation is not equivalent, but it is equivalent on CPU. Why? And how can I avoid this?
```python
import torch
import torch.nn as nn

hidden_states = torch.randn([4, 2048, 512])
v_proj = nn.Linear(512, 128, bias=False)
value_states = v_proj(hidden_states)

h1, h2 = torch.chunk(hidden_states, 2, dim=0)
v1 = v_proj(h1)

assert h1.equal(hidden_states[:2])
print(v1[0,0,0].item())
print(value_states[0,0,0].item())
assert v1.equal(value_states[:2])

hidden_states = torch.randn([4, 2048, 512]).cuda()
v_proj = nn.Linear(512, 128, bias=False).cuda()
value_states = v_proj(hidden_states)

h1, h2 = torch.chunk(hidden_states, 2, dim=0)
v1 = v_proj(h1)

assert h1.equal(hidden_states[:2])
print(v1[0,0,0].item())
print(value_states[0,0,0].item())
assert v1.equal(value_states[:2])
```
running results
```python
0.429298460483551
0.429298460483551
0.3757566213607788
0.37575680017471313
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
```

### Versions

```python
PyTorch version: 2.1.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Pro
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.13 (tags/v3.9.13:6de2ca5, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.22635-SP0
Is CUDA available: True
CUDA runtime version: 12.1.105
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3080 Ti Laptop GPU
Nvidia driver version: 531.14
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture=9
CurrentClockSpeed=2300
DeviceID=CPU0
Family=207
L2CacheSize=14336
L2CacheSpeed=
Manufacturer=GenuineIntel
MaxClockSpeed=2300
Name=12th Gen Intel(R) Core(TM) i9-12900HX
ProcessorType=3
Revision=

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.23.3
[pip3] torch==2.1.0+cu121
[pip3] torch-tb-profiler==0.4.3
[pip3] torchaudio==2.1.0+cu121
[pip3] torchvision==0.16.0+cu121
[pip3] torchviz==0.0.2
[conda] Could not collect
```
grad is inf/nan when using torch.amp

### 🐛 Describe the bug

Below is a very simple for using torch.amp, but the gradients are inf/nan.
```python
import torch
from torch.cuda.amp import GradScaler
from torch import optim
scaler = GradScaler()
a = torch.randn(2, 2, requires_grad=True, device="cuda")
b = torch.randn(2, 2, requires_grad=True, device="cuda")
optimizer = optim.Adam([a, b], lr=0.1)
with torch.autocast(device_type='cuda'):
    c = a @ b
    loss = c.sum()
scaler.scale(loss).backward()
scaler.unscale_(optimizer)
print(a.grad)
```
running results:
```python
tensor([[-inf, nan],
        [-inf, nan]], device='cuda:0')
```

### Versions

```python
PyTorch version: 2.1.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Pro
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.13 (tags/v3.9.13:6de2ca5, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.22635-SP0
Is CUDA available: True
CUDA runtime version: 12.1.105
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3080 Ti Laptop GPU
Nvidia driver version: 531.14
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture=9
CurrentClockSpeed=2300
DeviceID=CPU0
Family=207
L2CacheSize=14336
L2CacheSpeed=
Manufacturer=GenuineIntel
MaxClockSpeed=2300
Name=12th Gen Intel(R) Core(TM) i9-12900HX
ProcessorType=3
Revision=

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.23.3
[pip3] torch==2.1.0+cu121
[pip3] torch-tb-profiler==0.4.3
[pip3] torchaudio==2.1.0+cu121
[pip3] torchvision==0.16.0+cu121
[pip3] torchviz==0.0.2
[conda] Could not collect
```
[dynamo] Implement `set.__contains__` for `Tensor` as object match of `FakeTensor` 
Labels: open source, topic: not user facing, module: dynamo, ciflow/inductor

Fixes https://github.com/pytorch/pytorch/issues/111556

Dynamo implementation of `set.__contains__` previously used `__eq__` match.

But this is wrong when `__eq__` match does not imply `__hash__` match, as is the case for `torch.Tensor`, leading to inconsistent results. See: https://github.com/pytorch/pytorch/issues/111542

Hence implement as Tensor object match i.e. proxy node `'example_value'` FakeTensor match.


cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Support calling __torch_function__ attribute access
Labels: module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111737
* #111731
* #111730



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Implementation of Lion Optimizer.

### 🚀 The feature, motivation and pitch

Lion Optimizer is becoming a great alterative to AdamW and Adam Optimizer. It is more efficient as it does not use second order moments and instead uses sign operations in order to update the weights. This saves on memory and decreases training time. In  some cases it is better than Adam and AdamW as given in the paper.
The original paper for this is : https://arxiv.org/pdf/2302.06675.pdf

The RFCS PR for this is: https://github.com/pytorch/rfcs/pull/60

### Alternatives

_No response_

### Additional context

_No response_
Is the index_add_ function differentiable?

### 🚀 The feature, motivation and pitch

        verts_normals = torch.zeros_like(cornea_vertex)
        vertices_faces = cornea_vertex[face_index]

        faces_normals = torch.cross(
            vertices_faces[:, 2] - vertices_faces[:, 1],
            vertices_faces[:, 0] - vertices_faces[:, 1],
            dim=-1,
        )
        unit_faces_normals = safe_normalize(faces_normals)
        verts_normals.index_add_(0, face_index[:, 0], unit_faces_normals)
        verts_normals.index_add_(0, face_index[:, 1], unit_faces_normals)
        verts_normals.index_add_(0, face_index[:, 2], unit_faces_normals)

### Alternatives

_No response_

### Additional context

_No response_
```[tasklist]
### Tasks
```
Support tracing base torch_function impl
Labels: module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #111737
* __->__ #111731
* #111730



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
TensorWithTFOverride inheritance from TensorVariable
Labels: module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #111737
* #111731
* __->__ #111730



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Not Implemented Issue

### 🚀 The feature, motivation and pitch

NotImplementedError: The operator 'aten::_unique2' is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

### Alternatives

Please add this

### Additional context

_No response_
[dynamo] Remove VariableTracker.propagate
Labels: module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111726
* #111725
* #111415
* #111614
* #111717
* #111306



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
[dynamo] Remove VariableTracker.add_options
Labels: module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #111726
* __->__ #111725
* #111415
* #111614
* #111717
* #111306



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Add cudagraph_mark_step_begin in torch.compiler, reference in error message
Labels: module: inductor, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111722

cc @chauhang

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler
Constrain sdpa to fx strides
Labels: module: inductor, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111721

Fix for https://github.com/pytorch/pytorch/issues/109607. sdpa requires last dimension strides to be 1. Add constraint so that we run the op with the strides we observed in tracing.


cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler
add dry run metrics to td strategies
Labels: topic: not user facing

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111719

Creates a decorator in order to emit metrics for dry runs on target determination strategies. @ZainRizvi does this seem reasonable?

<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at c90a623</samp>

Add a new module `dry_run.py` to support dry run mode for test dependency strategies. This mode allows testing the logic and performance of different strategies without actually running the tests. This can help improve testing efficiency and quality.
[dynamo] Lazily construct symbolic_locals
Labels: module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #111726
* #111725
* #111415
* #111614
* __->__ #111717
* #111306



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Cannot pip install torch 2.0.1

### 🐛 Describe the bug

I was trying to follow the instruction on the [webpage](https://pytorch.org/get-started/previous-versions/) to install torch 2.0.1 using pip. 

```
# ROCM 5.4.2 (Linux only)
pip install torch==2.0.1+rocm5.4.2 torchvision==0.15.2+rocm5.4.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/rocm5.4.2
# CUDA 11.7
pip install torch==2.0.1+cu117 torchvision==0.15.2+cu117 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu117
# CUDA 11.8
pip install torch==2.0.1+cu118 torchvision==0.15.2+cu118 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
# CPU only
pip install torch==2.0.1+cpu torchvision==0.15.2+cpu torchaudio==2.0.2 --index-url https://download
```

But it would throw an error e.g. for installing 2.0.1+cu117 
```
ERROR: Could not find a version that satisfies the requirement torch==2.0.1+cu117 (from versions: 1.13.0+cu117, 1.13.1+cu117)
ERROR: No matching distribution found for torch==2.0.1+cu117
```
Commands for other versions above throw similar errors.

### Versions

Attempt to install 2.0.1
[Export] Don't serialize missing args with default value
Labels: fb-exported, topic: not user facing, module: inductor, ciflow/inductor, module: export

Summary: Per https://docs.google.com/document/d/1FzWm-sHYwmRi3x_g036kOxd99KaYquUsA-L5JwOn8ys/edit

I wonder if this would break executorch? @larryliu0820 
I see exir/serialize.py using export's GraphModuleSerializer. 

Test Plan: Existing CIs

Differential Revision: D50519217




cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @avikchaudhuri @gmagogsfm @zhxchen17 @tugsbayasgalan
[dynamo] generic `is_` type shortcut is not appropriately guarded
Labels: bug, oncall: pt2, module: dynamo

### 🐛 Describe the bug


This hack
https://github.com/pytorch/pytorch/blob/5a2f97dee80ca27b732e12b61359d6e475a9c03b/torch/_dynamo/variables/builtin.py#L1310
in https://github.com/pytorch/pytorch/pull/104840

is too strong.

### Use-Cases
Support for tracing `is_` when there's type mismatch: https://github.com/pytorch/pytorch/issues/109504
Part of the way to: https://github.com/pytorch/pytorch/issues/111550

### Solution
Perhaps installing unalias check and guarding on as python constant might be good enough to solve generic is_ check without resorting to hacks like this.


### Repro
```python
import collections

def fn(x, y, z):
    z += 1
    return x is y, z

x = collections.OrderedDict({1: 2})
y = {1: 2}
z = torch.tensor([1])

opt_fn = torch.compile(fn, backend="eager", fullgraph=True)

assert opt_fn(x, y, z) == fn(x, y, z)  # Compile with x is y == False
assert opt_fn(x, x, z) == fn(x, x, z)  # Does not recompile as input types are not guarded
```

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng 

### Versions

main

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519
Re-enable some embedded bag tests
Labels: topic: not user facing

They were temporary disabled in 2019 by  https://github.com/pytorch/pytorch/pull/26599

May be it has been fixed already...

<!--
copilot:poem
-->
### <samp>🤖 Generated by Copilot at 1e49d84</samp>

> _`TestEmbeddingNN`_
> _CUDA tests restored_
> _Bug fixed in autumn breeze_
[aotinductor] 14k models: CppCompileError: C++ compile error
Labels: triaged, oncall: pt2

```
25 errors like: CppCompileError: C++ compile error (example ./generated/test_krrish94_nerf_pytorch.py:SinThetaByTheta # pytest ./generated/test_krrish94_nerf_pytorch.py -k test_001)
```

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519
`fbgemm` update causes failures in `test_embedding.py`
Labels: high priority, triage review, module: regression, module: third_party

### 🐛 Describe the bug

```
% python3 test/nn/test_embedding.py -k test_EmbeddingBag_per_sample_weights_and_new_offsets_cpu_int32_int32_bfloat16
...
AssertionError: Tensor-likes are not close!

Mismatched elements: 4 / 10 (40.0%)
Greatest absolute difference: 9.1875 at index (3, 1) (up to 0.1 allowed)
Greatest relative difference: 0.57421875 at index (3, 1) (up to 0 allowed)
```

Reverting https://github.com/pytorch/FBGEMM/pull/1851 fixes the problem

### Versions

Nightly

cc @ezyang @gchanan @zou3519 @kadeng
lintrunner job time keeps growing
Labels: triaged, module: devx

For example:
Sep 29 https://hud.pytorch.org/pytorch/pytorch/commit/bc047ec906d8e1730e2ccd8192cef3c3467d75d1 - 18 mins
Oct 06 https://hud.pytorch.org/pytorch/pytorch/commit/65d40a72c4ff3cf5218dffda8b5da60ea2163890 - 22 mins
Today, Oct 20 https://hud.pytorch.org/pytorch/pytorch/commit/303c54dbd9921d78ed01116547c063b450338c74 - 26 mins

If we want to reduce the time, we need to investigate what's taking so long.
Two possible candidates are ruff and clangtidy.

It would be nice to have time split by linter in the lintrunner job logs.


cc @ZainRizvi @huydhn @clee2000 @PaliC @malfet
DISABLED test_meta_outplace_fft_ifft_cpu_uint8 (__main__.TestMetaCPU)
Labels: triaged, module: flaky-tests, skipped, module: primTorch, oncall: pt2

Platforms: dynamo

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_meta_outplace_fft_ifft_cpu_uint8&suite=TestMetaCPU) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17905842710).

Over the past 3 hours, it has been determined flaky in 12 workflow(s) with 36 failures and 12 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_meta_outplace_fft_ifft_cpu_uint8`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_meta.py`

cc @ezyang @mruberry @Lezcano @peterbell10 @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519
Add more flexibility on print / output console

### 🚀 The feature, motivation and pitch

For a large number of debug usage, printing tensors on console is clearly usefull. The current C++ API output fix the `std::cout` precision on float to 4 decimals. 

> float can hold up to 7 decimal digits accurately while double can hold up to 15

The only 'user' parameter of `torch::print()` is the `int64_t linesize = 80`.

### Alternatives

I see at least two major options that can be usefull for common usage:

1. set the number of decimal (see `std::set_precision`) for floating-point numbers.
2. set the scientific output optional, not automatic (see `std::fixed` / `std::scientific`) 

### Additional context

I currently output (`std::cout << tensor`) two differents 8x8 Float Tensors for comparison:

```
 0.2360  0.0258  0.1689  0.1564 -0.2261  0.0567  0.1844  0.3033
 0.2940 -0.2500 -0.0653 -0.0805  0.2112 -0.1635  0.2915  0.3023
 0.2912  0.0944  0.1377 -0.1824 -0.1882 -0.2844  0.0189 -0.2718
-0.2812 -0.0292 -0.3035 -0.0724  0.1665 -0.2391  0.0724 -0.1974
-0.2716  0.1460 -0.3044  0.1312  0.2848  0.1549  0.2815  0.1874
 0.0980 -0.1967  0.1135  0.2974 -0.1395  0.2800 -0.2298  0.2627
 0.2153  0.1423  0.2779  0.0157 -0.3499  0.1718  0.2147  0.2121
 0.2856  0.2004  0.0951 -0.0757 -0.3016  0.0643 -0.2685 -0.1260
[ CUDAFloatType{8,8} ]

 0.2360  0.0258  0.1689  0.1564 -0.2261  0.0567  0.1844  0.3033
 0.2940 -0.2500 -0.0653 -0.0805  0.2112 -0.1635  0.2915  0.3023
 0.2912  0.0944  0.1377 -0.1824 -0.1882 -0.2844  0.0189 -0.2718
-0.2812 -0.0292 -0.3035 -0.0724  0.1665 -0.2391  0.0724 -0.1973
-0.2716  0.1460 -0.3044  0.1312  0.2848  0.1549  0.2815  0.1874
 0.0980 -0.1967  0.1135  0.2974 -0.1395  0.2800 -0.2298  0.2627
 0.2153  0.1423  0.2779  0.0157 -0.3499  0.1718  0.2147  0.2121
 0.2856  0.2004  0.0951 -0.0757 -0.3016  0.0643 -0.2685 -0.1260
[ CUDAFloatType{8,8} ]
```

But the `torch::allclose(actual, expected, rtol, atol);` (where `rtol = atol = 1e-5`) give me a `false`.

This `false` is probably `true`, but console output don't help for a quick check.

!!! Thank you for Torch !!!
Runnings SentenceTransformer encoding step causes Docker containers on Mac (Silicon) to crash with code 139

### 🐛 Describe the bug

Hi! Hopefully there isn't a similar issue already open. I couldn't find one after a search through the issues list. Feel free to mark as duplicate/close if it already exists.

I've created this repository with a minimal setup to reproduce the error: https://github.com/sabaimran/repro-torch-bug. You just have to clone it and run `docker-compose up` to see the error. Basically it runs the script below in a minimal Docker container:

```python
import torch
from langchain.embeddings import HuggingFaceEmbeddings

class EmbeddingsModel:
    def __init__(self):
        self.model_name = "sentence-transformers/multi-qa-MiniLM-L6-cos-v1"
        encode_kwargs = {"normalize_embeddings": True}

        if torch.cuda.is_available():
            # Use CUDA GPU
            device = torch.device("cuda:0")
        elif torch.backends.mps.is_available():
            # Use Apple M1 Metal Acceleration
            device = torch.device("mps")
        else:
            device = torch.device("cpu")

        self.device = device
        model_kwargs = {"device": device}
        self.embeddings_model = HuggingFaceEmbeddings(
            model_name=self.model_name, encode_kwargs=encode_kwargs, model_kwargs=model_kwargs
        )

    def embed_documents(self, docs: List[str]):
        logger.info(f"Using device: {self.device} to embed {len(docs)} documents")
        return self.embeddings_model.embed_documents(docs)

model = EmbeddingsModel()
embeddings = model.embed_documents(["this is a document", "so is this"])
print(f"Created embeddings of length {len(embeddings)}")
```

If you run this code inside of a Docker container (with the appropriate dependencies), it will fail with exit code 139.

Pinning the `torch` package to `2.0.1` circumvents the error. See this other relevant issue: https://github.com/docker/for-mac/issues/7016

### Versions

Collecting environment information...
PyTorch version: 2.1.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.2.1 (arm64)
GCC version: Could not collect
Clang version: 14.0.3 (clang-1403.0.22.14.1)
CMake version: version 3.26.4
Libc version: N/A

Python version: 3.11.4 (main, Jul 10 2023, 18:52:37) [Clang 14.0.3 (clang-1403.0.22.14.1)] (64-bit runtime)
Python platform: macOS-13.2.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M2 Pro

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.26.1
[pip3] torch==2.1.0
[pip3] torchvision==0.16.0
[conda] Could not collect
[Release/2.1.1][ONNX] Fix aten::new_zeros due to TorchScript behavior change on Pytorch 2.1 Fix #110935
Labels: module: onnx, open source, release notes: onnx

Original PR: https://github.com/pytorch/pytorch/pull/110956

Fixes https://github.com/pytorch/pytorch/issues/110597

Summary:

* Generic code: The torch._C.Value.node().mustBeNone() is encapsulated into the high-level API JitScalarType.from_value ; _is_none was also extended to allow either None or torch._C.Value.node.mustBeNone(), so users don't manually call into TorchScript API when implementing operators
* Specific to new_zeros (and ops of  *_like and new_*): When checking dtype, we always must use  _is_none, which will call proposed by https://github.com/pytorch/pytorch/pull/110935
[export] 14k models: AssertionError: graph-captured input # 2, of type <class 'torch.nn.parameter.Parameter'>, is not among original inputs of types
Labels: triaged, oncall: pt2, module: export

167 errors like: AssertionError: graph-captured input # 2, of type <class 'torch.nn.parameter.Parameter'>, is not among original inputs of types: (<class 'torch.Tensor'>) (example ./generated/test_XPixelGroup_BasicSR.py:SPADEResnetBlock # pytest ./generated/test_XPixelGroup_BasicSR.py -k test_030)

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519 @avikchaudhuri @gmagogsfm @zhxchen17 @tugsbayasgalan
DISABLED test_sigmoid (__main__.TestQuantizedOps)
Labels: oncall: quantization, triaged, module: macos, skipped

Platforms: mac, macos

This test was disabled because it is failing on main branch ([recent examples](http://torch-ci.com/failure/test_quantization.py%3A%3ATestQuantizedOps%3A%3Atest_sigmoid)).

This test is failing on MacOS x86 https://hud.pytorch.org/pytorch/pytorch/commit/ca7d084ff9b67675cfff0d175ea6b96fcedc4950

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel @malfet @albanD
[aotinductor] 14k models: TypeError: make_boxed_func..g() missing 1 required positional argument: 'args' 
Labels: triaged, oncall: pt2

347 errors like: TypeError: make_boxed_func..g() missing 1 required positional argument: 'args' (example ./generated/test_ludwig_ai_ludwig.py:SequenceReducer # pytest ./generated/test_ludwig_ai_ludwig.py -k test_015)

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519
[Quantization] Add a test for QAT + PTQ selective quantization in
Labels: release notes: quantization

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111689

xnnpack quantizer

Summary:
For some workflows you want to quantize some parts of the model via qat
and then continue eager mode training. After training, you want to
export the whole model and perform PTQ on the rest.

Test Plan:
test added

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D50510480](https://our.internmc.facebook.com/intern/diff/D50510480)
[Release/2.1.1][DCP] Remove _shard_tensor() call in load_sharded_optimizer_state_dict in optimizer.py #111096

Cherry pick into 2.1.1

[original PR: #111096](https://github.com/pytorch/pytorch/pull/111096)
_shard_tensor() calls into dist.all_gather_object() and this is causing optimizer state dict loading to be super slow. Workaround: call FSDP._shard_utils._create_chunk_sharded_tensor() to construct ShardedTensor without any communication.
Disable dynamo when running generated opcheck tests
Labels: fb-exported

Summary: Use `TORCHDYNAMO_DISABLE=1` when running generated opcheck tests. Enable some `fbgemm::pack_segments` tests that errored out (with error `RuntimeError: expected int but got s0*s1**2`) because dynamo was being run in the opcheck tests.

Test Plan: `parsh -v --build-flags mode/dev-nosan //deeplearning/fbgemm/fbgemm_gpu:sparse_ops_test` then `run_tests("test_pack_segments")`

Differential Revision: D50508958
[BE]: ruff apply rule PLW1510 to find silent subprocess errors
Labels: open source, better-engineering, NNC, release notes: jit, module: dynamo, ciflow/inductor

Opts in to check=True or check=False to ensure nonzero exit codes are propogated

cc @EikanWang @jgong5 @voznesenskym @penguinwu @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
[Release/2.1.1] [Test][ShardedTensor] Add test for corner case for chunk sharding spec #109626
Labels: topic: not user facing

Cherry pick https://github.com/pytorch/pytorch/pull/109626 into release/2.1.1

This adds a test case to cover the corner case of empty shards when creating ShardedTensor.
Original fix contributed by a user.
https://github.com/pytorch/pytorch/pull/108915
Cherry-pick PR for the fix above: https://github.com/pytorch/pytorch/pull/108915
AOT Inductor Does not Work with minifier
Labels: ciflow/inductor

### 🐛 Describe the bug

Because AOT Inductor attaches parameters to the GraphModule, it does not currently work with minifier. 

>   File "/opt/dlami/nvme/eellison/work/pytorch/torch/_dynamo/repro/after_aot.py", line 444, in repro_common
    assert not any(mod.named_parameters())

### Versions

master
[2.1.1] Update NCCL to 2.18.6 for upstream bugfix
Labels: open source, topic: not user facing

This updates NCCL in PyTorch 2.1 with one tiny bugfix in from this commit: https://github.com/NVIDIA/nccl/commit/4365458757e4107ecbf629b2fd6e0e19a5d237c2 It's a minor bugfix release, otherwise everything is exactly the same as the release currently in PyTorch. We already updated to 2.19 upstream.
[export] self.buffer += 1 raises error
Labels: triaged, module: export

```
import torch

class Mod(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.register_buffer("foo", torch.ones(2, 3))

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        self.foo += x
        return self.foo

torch.export(Mod(), (torch.ones(2, 3),))
```
produces
```
Mutating module attribute foo during export.

from user code:
   File "/tmp/ipykernel_578241/3307013751.py", line 9, in forward
    self.foo += x

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
```

Changing `self.foo += x` to the equivalent `self.foo.add_(x)` works as expected.

The motivation behind disallowing attribute mutation makes sense. However, buffers should be mutable, and dynamo should be smart enough to recognize that `+=` desugars to `add_` when done on a tensor.

cc @avikchaudhuri @gmagogsfm @zhxchen17 @tugsbayasgalan
[quant][bc-breaking] Remove deprecated QConfigDynamic
Labels: release notes: quantization

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111673

Summary: QConfigDynamic was deprecated in PyTorch 1.12. It has
continued to cause confusion to users who wish to use dynamic
quantization. This commit removes this deprecated API and
requires users to use QConfig instead.

BC-breaking before:
```
qconfig = QConfigDynamic(
    activation=default_dynamic_quant_observer,
    weight=default_weight_observer,
)
```

BC-breaking after:
```
qconfig = QConfig(
    activation=default_dynamic_quant_observer,
    weight=default_weight_observer,
)
```

Test Plan:
python test/test_quantization.py

Reviewers: jerryzh168

Subscribers: jerryzh168, supriyar
Buffer overflow not prevented on MPS devices

### 🐛 Describe the bug

When indexing using an indexing tensor (or list), it is possible to read or write outside the valid range of the tensor.

Minimal example:

```
import torch

x = torch.arange(4, device=torch.device("mps"))
y = x[:2]
y[torch.tensor([3])] = -1
x[3]
```

This code should raise an IndexError and leave x unchanged, but it instead gives -1.

In this example, the overflow reaches a known memory location, but perhaps in general it can reach arbitrary memory on the GPU.

### Versions

Collecting environment information...
PyTorch version: 2.1.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.5.2 (arm64)
GCC version: Could not collect
Clang version: 15.0.0 (clang-1500.0.40.1)
CMake version: version 3.27.7
Libc version: N/A

Python version: 3.10.8 (main, Nov 24 2022, 08:08:27) [Clang 14.0.6 ] (64-bit runtime)
Python platform: macOS-13.5.2-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M2 Pro

Versions of relevant libraries:
[pip3] flake8==6.1.0
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.26.1
[pip3] torch==2.1.0
[conda] numpy                     1.26.1                   pypi_0    pypi
[conda] torch                     2.1.0                    pypi_0    pypi
[Release/2.1] Introduce is_big_gpu condition for test_max_autotune
Labels: open source, topic: not user facing, module: inductor

Fixes https://github.com/pytorch/pytorch/issues/111527

Other test files that rely on max_autotune mode being enabled already conditionalise the UT suite on this condition (e.g. test_select_algorithm). Proposing to add this condition for test_max_autotune.

Currently we are observing failures on these UTs on the ROCm runners but using MI200+ these tests will pass again context: https://github.com/pytorch/pytorch/pull/111381#issuecomment-1768048732

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler
torch.onnx.errors.UnsupportedOperatorError: Exporting the operator 'aten::binary_cross_entropy' to ONNX opset version 14 is not supported.
Labels: module: onnx

### 🚀 The feature, motivation and pitch

Unable to export ONNX model from https://github.com/xue-pai/FuxiCTR/tree/main/model_zoo/AFM. While exporting onnx it throws torch.onnx.errors.UnsupportedOperatorError: Exporting the operator 'aten::binary_cross_entropy' 

### Alternatives

_No response_

### Additional context

_No response_
[dynamo] Fix guard for ndarray calling `torch.as_tensor(None)`
Labels: open source, topic: not user facing, module: dynamo, ciflow/inductor

Fixes https://github.com/pytorch/pytorch/issues/111662

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng @lezcano
[dynamo] Tracking: object identity

### 🚀 The feature, motivation and pitch

This covers many things:
1. Tensor Identity
2. User objects identity - usual objects, enums, builtins??

Use cases
- [ ] https://github.com/pytorch/pytorch/issues/111550
- [ ] https://github.com/pytorch/pytorch/issues/111556

Tensor Aliasing Methods and Obstacles
- [x] https://github.com/pytorch/pytorch/issues/111585
- [x] https://github.com/pytorch/pytorch/issues/111649
- [ ] https://github.com/pytorch/pytorch/issues/111544

Overall Obstacles and Discussion
- [x] https://github.com/pytorch/pytorch/issues/111542
- [ ] https://github.com/pytorch/pytorch/issues/111562 - not sure if we want to implement non-aliasing for general objects
torch.dynamo (caching?) issues with `Optional[np.ndarray]` arguments
Labels: module: numpy, module: dynamo

### 🐛 Describe the bug

```
$ cat nonz.py 
import torch
import numpy as np

def fn(x=None):
    if x is None:
        x = np.ones(3)
    return x**2

opt_fn = torch.compile(fn)

x = np.zeros((2, 2))
print(opt_fn(x))
print(opt_fn())
```

fails with

```
$ python nonz.py 
[[0. 0.]
 [0. 0.]]
ERROR RUNNING GUARDS fn nonz.py:9
lambda L, **___kwargs_ignored:
  ___guarded_code.valid and
  ___check_global_state() and
  hasattr(__as_tensor(L['x']), '_dynamo_dynamic_indices') == False and
  utils_device.CURRENT_DEVICE == None and
  ___skip_backend_check() or ___current_backend() == ___lookup_backend(139895339872512) and
  ___check_tensors(__as_tensor(L['x']), tensor_check_names=tensor_check_names)
Traceback (most recent call last):
  File "nonz.py", line 20, in <module>
    print(opt_fn())
  File "/home/ev-br/repos/pytorch/torch/_dynamo/eval_frame.py", line 410, in _fn
    return fn(*args, **kwargs)
  File "<string>", line 7, in guard
RuntimeError: Could not infer dtype of NoneType
```

Curiously, exchanging the order of calls, i.e. making it 

```
print(opt_fn())
print(opt_fn(x))
```

works fine and produces the correct result. Removing the numpy from the equation --- chaging all `np.` to `torch.` also works fine and produces the correct result.

### Versions

main

cc @mruberry @rgommers @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Higher-level custom op API, V3

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111661
* #111660
* #111310
* #111659
* #111380

This PR introduces:
- a FunctonalBaseOp class.
- To define a new custom op, a user subclasses FunctionalBaseOp, adds
  their device-specific implementations (by adding static methods to
  their subclass), and adds an abstract impl by overriding the
  `abstract` staticmethod.
- Under the hood, we take the class and all its methods and call the
  corresponding torch.library.{define, impl} APIs to register it.

The class-based approach more closely resembles PyTorch C++ codegen;
we get all of the information about the operator up front in the class
and we can do things with this information, like add an "autograd not
implemented" kernel if the user didn't specify an autograd kernel (NB:
this functionality is not in this PR).

Please see the docstrings for example usages.
Change torch.library.impl to accept a device string

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #111661
* #111660
* #111310
* __->__ #111659
* #111380

torch.library.impl now accepts a device string (e.g. "cpu", "cuda"). It
still accepts DispatchKey strings, but we no longer document this, because
using arbitrary DispatchKeys is more for the power users.

We map the device string to a DispatchKey and then register the impl for
said DispatchKey. A user may also specify multiple device strings at once
or specify "types=default" to get a CompositeExplicitAutograd registration.

Test Plan:
- new tests
[aotinductor] Update test utility to use AOTIModelRunner
Labels: module: inductor, module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111657

Summary: Use AOTIModelRunner provided by libtorch instead of the custom written RAIIModelContainer for testing. This change also makes running AOTInductor benchmarks on CPU possbile.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler
WIP Adding 512 to xblock size config
Labels: open source, module: inductor, ciflow/inductor

Try to see perf improvements with adding 512 into xblock size:

inductor-A100-perf-nightly :
- https://github.com/pytorch/pytorch/actions/runs/6589467661


cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler
DISABLED test_meta_outplace_fft_ifft_cpu_int64 (__main__.TestMetaCPU)
Labels: triaged, module: flaky-tests, skipped, module: primTorch, oncall: pt2

Platforms: dynamo

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_meta_outplace_fft_ifft_cpu_int64&suite=TestMetaCPU) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17893852781).

Over the past 3 hours, it has been determined flaky in 2 workflow(s) with 6 failures and 2 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_meta_outplace_fft_ifft_cpu_int64`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_meta.py`

cc @ezyang @mruberry @Lezcano @peterbell10 @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519
FSDP CPU Offload + fp16 + sharded grad scaler crash / hang
Labels: oncall: distributed, triaged, module: fsdp

### 🐛 Describe the bug

I get the following when running the above combination:

```
ERROR:aiplatform.error_reporting.error_reporting:Exception Found: Could not run 'ate     n::_amp_foreach_non_finite_check_and_unscale_' with arguments from the 'CPU' backend     . This could be because the operator doesn't exist for this backend, or was omitted      during the selective/custom build process (if using custom build). If you are a Face     book employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for p     ossible resolutions. 'aten::_amp_foreach_non_finite_check_and_unscale_' is only avai     lable for these backends: [CUDA, Meta, BackendSelect, Python, FuncTorchDynamicLayerB     ackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, Aut     ogradOther, AutogradCPU, AutogradCUDA, AutogradHIP, AutogradXLA, AutogradMPS, Autogr     adIPU, AutogradXPU, AutogradHPU, AutogradVE, AutogradLazy, AutogradMTIA, AutogradPri     vateUse1, AutogradPrivateUse2, AutogradPrivateUse3, AutogradMeta, AutogradNestedTens     or, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, BatchedNestedTensor, FuncTo     rchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDy     namicLayerFrontMode, PreDispatch, PythonDispatcher].
```

in my case, the error seems to be reported but the job doesn't crash and just hangs, which is interesting and might be related to other collectives going on?

### Versions

main

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu @fegin
[dynamo] higher-order ops do not preserve `FakeTensor` for in-place ops
Labels: triaged, module: fakeTensor, module: functorch, module: dynamo

### 🐛 Describe the bug

```python
def fn(z):
    x = z.clone()
    y = torch.vmap(torch.Tensor.acos_)(x)
    # y's fake tensor is not x's fake tensor in terms of pyobjects
    return y is x

fn_opt = torch.compile(backend="eager", fullgraph=True, dynamic=True)(fn)

z = torch.ones(4, 1)

self.assertEqual(fn(z), fn_opt(z))
```

### Solution
Not sure if this is a bug. But ideally, we should not expect x's fake tensor to be different from y's, as then it can be a method to preserve object identity throughout recursive calls to FX.

One possible way to do this might be to reuse the FakeTensor from the inputs when doing fx tracing for higher order ops.
 
However, if it is unavoidable (e.g. as a result of fx tracing requirements), another solution is simply to propagate the storage of the original fake tensor to the higher order op fake tensor.

### Versions

main

cc @eellison @zou3519 @Chillee @samdow @kshitij12345 @janeyx99 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
DISABLED test_nested_tensor_chunk_cpu_float16 (__main__.TestNestedTensorDeviceTypeCPU)
Labels: triaged, module: flaky-tests, module: nestedtensor, skipped, oncall: pt2

Platforms: dynamo

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_nested_tensor_chunk_cpu_float16&suite=TestNestedTensorDeviceTypeCPU) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17886253401).

Over the past 3 hours, it has been determined flaky in 5 workflow(s) with 15 failures and 5 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_nested_tensor_chunk_cpu_float16`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_nestedtensor.py`

cc @cpuhrsch @jbschlosser @bhosmer @drisspg @soulitzer @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519