gopikrsmscs/torch-issues huggingface dataset:
[discussion] move-semantics for tensors
Labels: triaged, module: python frontend

### 🚀 The feature, motivation and pitch

Sometimes we want to kind of pass the tensor to a function entirely (so that it can safely do inplace modifications without worrying to disturb other consumers).

One way it could work is that after a new_tensor_ref = tensor.move() is called, the previous tensor reference becomes somewhat invalid and would throw exception on any usage.

There could also be some `assert tensor.is_single_owner()` which could ensure that there is only one reference to the tensor.

Of course, there would be ways to circumvent this (e.g. by recording manually data_ptr), but it could be a way to increase expressiveness, give compiler more hints for compilation, and let the user express  clearer ownership semantics and have some guards.

### Alternatives

_No response_

### Additional context

_No response_

cc @albanD
Lacking commutativity of `tensor.expand` and `tensor.flatten`
Labels: triaged, module: viewing and reshaping

### 🐛 Describe the bug

I guess this is just another facet of https://github.com/pytorch/pytorch/issues/28090

`flatten` could realloc only the needed amount needed to deal with the required dimensions (and keep some unaffected dimensions expanded).

In this example, `x1` consumes less memory than `x2`. If `flatten` was smarter, `x2` would consume as little memory as `x1`.

```python
import torch
# output from meshgrid
x0 = torch.arange(100).unsqueeze(-1).expand(-1, 200)
print(x0.numel(), x0.storage().size())
#bar.py:4: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage #will be the only storage class. This should only matter to you if you are using storages directly.  To #access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
#  print(x0.numel(), x0.storage().size())
# 20000 100

x1 = x0.flatten(start_dim = -2).unsqueeze(0).expand(30, -1)
print(x1.numel(), x1.storage().size())
# 600000 20000

x2 = x0.unsqueeze(0).expand(30, -1, -1).flatten(start_dim = -2)
print(x2.numel(), x2.storage().size())
# 600000 600000
```

### Versions

2.1.0 nightly
[inductor] Add ir.Scan and lower aten.cumsum on CUDA
Labels: open source, Merged, Reverted, ciflow/trunk, module: inductor, ciflow/inductor, ciflow/unstable, release notes: inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #110911
* #109132
* __->__ #106581
* #109601

This adds the `ir.Scan` node (currently only supported on CUDA) which re-uses the existing reduction kernel machinery to support different kinds of non-pointwise ops. Just like reductions it supports prologue and epilogue fusions and has both persistent and non-persistent kernel generation.

Currently this doesn't support the equivalent of `Reduction.create_multilayer` and will instead fall back to eager in those cases. This is because splitting into multiple kernel invocations ends up being far slower than cub's single kernel strategy which matches the performance of a copy kernel.


cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
[dynamo] Unsupported to trace through Boolean Tensor indexing
Labels: triaged, oncall: pt2, module: dynamic shapes

### 🐛 Describe the bug

even though indexing by boolean tensor is supported in normal torch, it is not supported in dynamo

### Reprod Script
```python
import torch

def reprod_function(t: torch.Tensor, a: torch.Tensor):
    t[a] *= 0.2
    return t

reprod_function_compile = torch.compile(reprod_function, backend="eager", fullgraph=True)

x  = torch.rand(1,2,3)
y = torch.rand(1,2,3).to(torch.bool)
z = reprod_function(x,y)
print(z.dtype)
z = reprod_function_compile(x,y)
print(z.dtype)
```

### Traceback
```python
File "PATH_TO_PYTHON_LIB/site-packages/torch/_dynamo/variables/builtin.py", line 450, in call_function
    unimplemented("dynamic Tensor.__getitem__(bool[])")
  File "PATH_TO_PYTHON_LIB/site-packages/torch/_dynamo/exc.py", line 71, in unimplemented
    raise Unsupported(msg)
torch._dynamo.exc.Unsupported: dynamic Tensor.__getitem__(bool[])

from user code:
   File "./reprod.py", line 4, in reprod_function
    t[a] *= 0.2

Set torch._dynamo.config.verbose=True for more information


You can suppress this exception and fall back to eager by setting:
    torch._dynamo.config.suppress_errors = True
```

cc: @cchan @ezhang887 @ezyang

### Versions

torch==2.0.1
python==3.8

### Workaround
```python
t = torch.where(a, t * 0.2, t) 
```

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305
Enable more flake8-bugbear lints
Labels: good first issue, module: lint, triaged, enhancement, better-engineering

### 🚀 The feature, motivation and pitch

* There are a lot of flake8-bugbear code reported by flake8-bugbear and ruff that need to be fixed. These would be a good starting issue for a new contributor.

This can be found be removing the codes from the ignore list in the .flake8 file and the .pyproject file for ruff.

- [ ] "B007", 
- [ ] "B008" 
- [ ] "B017",
- [ ] "B018", # Useless expression
- [ ] "B019", 
- [ ] "B020",
- [ ] "B023", 
- [ ] "B024", 
- [ ] "B026",
- [ ] "B028", # No explicit `stacklevel` keyword argument found
- [ ] "B904",

### Alternatives

_No response_

### Additional context

_No response_
DTensor Sharding prop cache stats
Labels: oncall: distributed, triaged, module: dtensor

### 🚀 The feature, motivation and pitch

We have built `_CachingPropagator` and we need some stats to monitor cache miss rate so that we can understand potential perf regression in DTensor CPU overhead.

### Alternatives

_No response_

### Additional context

_No response_

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu
NotImplementedError: Could not run 'aten::multinomial' with arguments from the 'Meta' backend.
Labels: triaged, module: random

### 🐛 Describe the bug

The error says:
NotImplementedError: Could not run 'aten::multinomial' with arguments from the 'Meta' backend. This could be because the operator doesn't exist for thi
s backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, pl
ease visit https://fburl.com/ptmfixes for possible resolutions. 'aten::multinomial' is only available for these backends: [CPU, CUDA, BackendSelect, Py
thon, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, 
AutogradHIP, AutogradXLA, AutogradMPS, AutogradIPU, AutogradXPU, AutogradHPU, AutogradVE, AutogradLazy, AutogradMeta, AutogradMTIA, AutogradPrivateUse1
, AutogradPrivateUse2, AutogradPrivateUse3, AutogradNestedTensor, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, FuncTorchVmapMode, Batched, Vmap
Mode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PythonDispatcher].


Code:
def sample_top_p(probs, p):
    probs_sort, probs_idx = torch.sort(probs, dim=-1, descending=True)
    probs_sum = torch.cumsum(probs_sort, dim=-1)
    mask = probs_sum - probs_sort > p
    probs_sort[mask] = 0.0
    probs_sort.div_(probs_sort.sum(dim=-1, keepdim=True))
    next_token = torch.multinomial(probs_sort, num_samples=1)
    next_token = torch.gather(probs_idx, -1, next_token)
    return next_token

### Versions

StatusCode        : 200
StatusDescription : OK
Content           :
                    # Unlike the rest of the PyTorch this file must be python2 compliant.
                    # This script outputs relevant system environment info
                    # Run it with `python collect_env.py`.
                    import datetime
                    import locale
                    impor...
RawContent        : HTTP/1.1 200 OK
                    Connection: keep-alive
                    Content-Security-Policy: default-src 'none'; style-src 'unsafe-inline'; sandbox
                    Strict-Transport-Security: max-age=31536000
                    X-Content-Type-Options: nosniff
                    ...
Forms             : {}
Headers           : {[Connection, keep-alive], [Content-Security-Policy, default-src 'none'; style-src 'unsafe-inline'; sandbox],
                    [Strict-Transport-Security, max-age=31536000], [X-Content-Type-Options, nosniff]...}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        : mshtml.HTMLDocumentClass
RawContentLength  : 21653


cc @pbelevich
DISABLED test_cpp_wrapper_cpu (__main__.FreezingCpuTests)
Labels: triaged, skipped, oncall: pt2

Platforms: macos

This test was disabled because it is failing on master ([recent examples](http://torch-ci.com/failure/inductor%2Ftest_inductor_freezing.py%3A%3AFreezingCpuTests%3A%3Atest_cpp_wrapper_cpu)).

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @albanD
[dynamo] teach dynamo about `pytree._broadcast_to_and_flatten`
Labels: triaged, oncall: pt2, module: dynamo

### 🐛 Describe the bug

Given that dynamo knows about a few pytree functions, it would be great to support `_broadcast_to_and_flatten` as it will allow us to improve `vmap` support for dynamo

Ref: https://github.com/pytorch/pytorch/pull/101707#discussion_r1280562461

Repro:
```python
import torch

torch._dynamo.config.suppress_errors=False

@torch.compile(backend="eager")
def test():
    args = (0, 1, 1, 0)
    _, arg_spec = torch.utils._pytree.tree_flatten(args)

    # Dynamo doesn't know how to handle this.
    torch.utils._pytree._broadcast_to_and_flatten(0, arg_spec)
    
test()

```

Output
```
  File "/home/kshiteej/Pytorch/pytorch_functorch/torch/fx/node.py", line 644, in <genexpr>
    t = tuple(map_aggregate(elem, fn) for elem in a)
  File "/home/kshiteej/Pytorch/pytorch_functorch/torch/fx/node.py", line 654, in map_aggregate
    return fn(a)
  File "/home/kshiteej/Pytorch/pytorch_functorch/torch/fx/node.py", line 636, in <lambda>
    return map_aggregate(a, lambda x: fn(x) if isinstance(x, Node) else x)
  File "/home/kshiteej/Pytorch/pytorch_functorch/torch/_dynamo/utils.py", line 1302, in visit
    return n.meta["example_value"]
torch._dynamo.exc.InternalTorchDynamoError: example_value
```


### Versions

master

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @aakhundov
`softmax` to handle dimensions comprised of `-inf`
Labels: module: nn, triaged

### 🚀 The feature, motivation and pitch

Currently `softmax` handles `-inf` in most of the cases, for example:

```python
>>> x = torch.tensor([-float('inf'), 1])
>>> x.softmax(-1)
tensor([0., 1.])
>>> x.cuda().softmax(-1)
tensor([0., 1.], device='cuda:0')
```

However, once the normalized dimension is comprised of solely `-inf`, there is an issue:
```python
>>> x = torch.tensor([-float('inf')] * 2)
>>> x.softmax(-1)
tensor([nan, nan])
>>> x.cuda().softmax(-1)
tensor([nan, nan], device='cuda:0')
```

I understand we get it because `inf - inf = nan`, but why not assume that `softmax` is computed in the limit?
In such a case `softmax` should return a uniform distribution. This case is quite easy to detect since `max == -inf` will imply a uniform distribution.

Issues for more context:
https://github.com/pytorch/pytorch/issues/25110
https://github.com/pytorch/pytorch/issues/103749
https://github.com/pytorch/pytorch/issues/103963


cc @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki
Branch name in double quotes ""
Labels: module: ci, triaged

### 🐛 Describe the bug

TBD

### Versions

It is not a bug per say, but a issue we are facing. We see that a new branch has been added with double quotes in it name: "serialize".
We have forked the pytorch repository and our internal automation of fetching the latest code is broken due to branch name in quotes. 
Just curious, if there any guidelines about branch name?

cc @seemethere @malfet @pytorch/pytorch-dev-infra
Enable Mypy Checking in torch/_inductor/fx_passes/split_cat.py
Labels: triaged, open source, module: inductor, ciflow/inductor

Fixes #105230 Enable Mypy Checking in torch/_inductor

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
CUDA device support does not register allocator to c10::GetAllocator(...)
Labels: module: cpp, module: cuda, triaged

### 🐛 Describe the bug

The CUDA device support does not register its device allocator to `c10::GetAllocator(c10::DeviceType)` but explicitly requires to use `c10::cuda::CUDACachingAllocator::get()`.

Here a small demonstrator, that prints the pointer to the CUDACachingAllocator (to proof that CUDA support is loaded), and then all known allocators that are registered to `c10::GetAllocator(...)`.
```c++

#include <c10/cuda/CUDACachingAllocator.h>
#include <iostream>

int main(int argc, char** argv) {
	std::cout << "CUDACachingAllocator: " << c10::cuda::CUDACachingAllocator::get() << std::endl;

	for(int D = 0; D < (int)c10::DeviceType::COMPILE_TIME_MAX_DEVICE_TYPES; D++) {
		auto d = (c10::DeviceType)D;
		std::cout << c10::DeviceTypeName(d) << ": " << c10::GetAllocator(d) << std::endl;
	}

	return 0;
}
```

And then run it using:
```bash
python3 -m venv error
. error/bin/activate
python3 -m pip install torch
LIB=$(python3 -c "import site;print(site.getsitepackages()[0])")

g++ -D_GLIBCXX_USE_CXX11_ABI=0 -L $LIB/torch/lib -I $LIB/torch/include/ -lc10 -lc10_cuda error.cpp
LD_LIBRARY_PATH=$LIB/torch/lib ./a.out
```

Output:
```
CUDACachingAllocator: 0x7f9ab62efda0
CPU: 0x7f9ab639ed68
CUDA: 0
MKLDNN: 0
OPENGL: 0
OPENCL: 0
IDEEP: 0
HIP: 0
FPGA: 0
ORT: 0
XLA: 0
VULKAN: 0
METAL: 0
XPU: 0
MPS: 0
META: 0
HPU: 0
VE: 0
LAZY: 0
IPU: 0
MTIA: 0
privateuseone: 0
```

To my understanding in `c10/cuda/CUDACachingAllocator.cpp` in `BackendStaticInitializer::BackendStaticInitializer()` it should call `c10::SetAllocator(c10::DeviceType::CUDA, r);`

### Versions

Collecting environment information...
PyTorch version: 2.0.1+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: CentOS Linux 7 (Core) (x86_64)
GCC version: (GCC) 10.5.0
Clang version: Could not collect
CMake version: version 3.27.0
Libc version: glibc-2.17

Python version: 3.8.17 (default, Jun 26 2023, 05:54:38)  [GCC 10.2.1 20210130 (Red Hat 10.2.1-11)] (64-bit runtime)
Python platform: Linux-3.10.0-1160.76.1.el7.x86_64-x86_64-with-glibc2.17
Is CUDA available: True
CUDA runtime version: 11.8.89
GPU models and configuration: GPU 0: Quadro P4000
Nvidia driver version: 535.54.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] torch==2.0.1
[conda] Could not collect

cc @jbschlosser @ptrblck
Pytorch + ROCm+ Windows
Labels: module: rocm, triaged

### 🚀 The feature, motivation and pitch

A week ago, amd published ROCm for windows and cards like the 6600xt. Yet, you can't install pytorch under this configuration:

![image](https://github.com/pytorch/pytorch/assets/92395216/d1737c9a-cbbc-4528-ad7a-98dc84958fc7)

I asked in the unofficial pytorch discord server and somebody told me to try:

pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm5.6

but that simply resulted in the error: 

ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
ERROR: No matching distribution found for torch

Is there any way to get a working version yet and if not, will that come in the future and when can it be expected?

Thanks for any help and I hope I posted this under the correct category.

### Alternatives

_No response_

### Additional context

_No response_

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang
Distributed torch.linalg.eigh (and other functions) on cuda using cuSOLVERMG
Labels: module: cuda, triaged, module: linear algebra

### 🚀 The feature, motivation and pitch

The current torch.linalg system seems to primarily call cuSOLVER for many of the operations. I was wondering if it would be possible to ``upgrade" this to use cuSOLVERMG, which is the distributed version of cuSOLVER. Doing this would allow us to, for example, take a eigenvalue decomposition of larger matrices without OOMing.

### Alternatives

_No response_

### Additional context

_No response_

cc @ptrblck @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano
Fix reset_parameters for nn.MHA
Labels: Stale

**This is not ready for review but hoping for general feedback on how we should resolve this:**

`_reset_parameters()` here does not follow the contract for `reset_parameters()` for two reasons 

1) The order in which we initialize things in MHA makes `torch.get_rng_state()` inconsistent when initializing the same param/buffer in `nn.MultiheadAttention.__init__()` and when resetting that param/buffer in `nn.MultiheadAttention()._reset_parameters`. 

   In `__init__` we initialize `out_proj` (a submodule) first before using `self._reset_parameters()` to initialize the reset of the parameters owned by the root. So if we consider the `rng_states` as some sequence $r_0, r_1, r_2, ...$ where $r_0$ is the first `rng_state` after `torch.manual_seed(0)`
   
    ```
     torch.manual_seed(seed)
     # torch.get_rng_state() when initializing (e.g. `self.in_proj_weight`) is r_2
     # 2 because NonDynamicallyQuantizableLinear has 2 parameters
     m = nn.MultiheadAttention()
 
     torch.manual_seed(seed)
     # torch.get_rng_state() when initializing (e.g. `self.in_proj_weight`) is r_0 
     m.reset_parameters()
     ```


2) It resets `out_proj.bias` which is a parameter owned by a submodule.

This PR tries to fix it by moving `out_proj` to be initialized after the call to `_reset_parameter()` in `__init__()` and also not initializing `out_proj.bias` in `_reset_parameters()` **but this would be BC-breaking**. 


Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #106508
* #106506
* #106715
Add ModuleInfo testing for reset_parameters
Labels: Stale, ciflow/trunk, topic: not user facing

The contract on `nn.Module.reset_parameters()` should be as follows

1) `reset_parameters` should re-initialize parameters and buffers to how they were initialized in `nn.Module.__init__()`
2) `reset_parameters` should **not** recurse, (i.e. it should only reset parameters and buffers owned by the root, and not those of submodules)
3) modules that do not have parameters do not need to have a `reset_parameters` (*perhaps we could add one that is a no-op to the `nn.Module` base class*)
4) The following should hold 
```
# set some seed
torch.manual_seed(some_seed)
m = SomeModule()
# modify params/buffers in some way 
... 
torch.manual_seed(some_seed)
m.apply(lambda mod: mod.reset_parameters()
# params/bufs given by m.{parameters/buffers}(recurse=False) should match those of the initial m
```

Going over the ModuleInfos that we have not migrated, only `nn.EmbeddingBag` has parameters/buffers. So otherwise the testing here should be quite complete.

Some modules have an underscore `_reset_parameters()`. However the behavior of these does not necessarily match the contract (in particular for `nn.MultiheadAttention`, see further discussion in the description of  #106508)

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #106508
* __->__ #106506
* #106715
Enabling Transformer fast path for not batch_first
Labels: fb-exported, Stale, release notes: nn

Summary:
The fast path for the `forward()` method in `MultiheadAttention` only accepted `batch_first = True`. This diff enables fast path for `batch_first=False` as well.

Previously https://github.com/pytorch/pytorch/pull/85576 (approved by Christian but went stale and auto-closed)

Test Plan: Added unit test for fast path for both values of `batch_first` producing identical outputs. Test location - `//caffe2/test/test_native_mha.py`

Differential Revision: D47982531
AOTAutograd should detect false aliasing.
Labels: triaged, module: viewing and reshaping, oncall: pt2, module: aotdispatch

Today, if you write code like this:

```
@torch.compile
def f(x, y):
    x.add_(1)
    y.add_(1)

base = torch.ones(2, 2)
x = base[:, 0]
y = base[:, 1]
f(x, y)
```

Then AOTAutograd will attempt to perform some complicated "synthetic base" logic detailed [here](https://github.com/pytorch/pytorch/blob/b37a50afda55c5b73298016d10fca1f8c6f65055/torch/_functorch/aot_autograd.py#L2283).

The synthetic base logic is important for general cases where you have two inputs that alias the same memory, and a mutation on one input needs to be reflected to the other input.

But there are two problems with the example above;

(1) The repro above exhibits **false aliasing**. Even though `x` and `y` share the same storage, their memory locations are actually completely disjoint. We can generate significantly more efficient code if we detect this false aliasing. Otherwise, if we run the "synthetic base" logic, we will have to materialize two updated versions of `base` throughout the compiled graph for `f`.

(2) The synthetic base logic has a few complicated corner cases that it cannot handle. In particular, if your inputs alias each other and have non-trivial `._base` attributes, this is a case that we don't currently handle today. Fixing it is pretty non-trivial ([code](https://github.com/pytorch/pytorch/blob/main/torch/_functorch/aot_autograd.py#L1817)).

What should we do? In theory, we should be able to use the sizes/strides/storage_offset of our two inputs to figure out that they do **not** have any overlapping memory, even though they alias the same storage.

We have a version of this check in C++ today, but it only handles very simple cases because it needs to run in eager mode, which means it needs to be fast. It cannot handle the check above, because it bails out if either of its inputs are non-contiguous. Since we are compiling, we have the luxury of being okay with being a bit slower in how we check for overlapping memory.

Context: this appears to show up in the Shampoo optimizer. This could also potentially help other cases, such as:
- FSDP (uses flat params)
- LSTM (I believe the LSTM module has an optimization where its parameters are part of a single buffer)

cc @ezyang @msaroufim @wconstab @anijain2305
vmap, jacrev, jacfwd, hessian, etc., in libTorch
Labels: module: cpp, triaged, module: vmap, module: functorch

### 🚀 The feature, motivation and pitch

Apparently, right now there is a gap between the PyTorch APIs and the libTorch APIs regarding vmap and its related (previously functorch) capabilities. Moreover, I am afraid that the two set of APIs are drifting further apart, especially in newer versions of PyTorch, see e.g. the comparison between PyTorch's grad and libTorch's grad:

https://pytorch.org/docs/stable/generated/torch.autograd.grad.html
https://pytorch.org/cppdocs/api/function_namespacetorch_1_1autograd_1a1e03c42b14b40c306f9eb947ef842d9c.html

A vmap implementation in libTorch would allow us to compute Jacobians of batched computations much more efficiently, which is often a necessary ingredient in the conventional scientific computing world (by which I mean communities like numerical methods for solving partial differential equations).

Some potentially related open issues:
https://github.com/pytorch/pytorch/issues/40208
https://github.com/pytorch/functorch/issues/767

We might be able to have some one take a stab at this next FY, and hopefully submit a PR. I would like to see what the PyTorch core development team thinks about this. Some pointers to get us started would be appreciated.

Just to prove I am not crazy about this and indeed this is possible, I have put together a repository to demonstrate the feasibility of this. Some examples are here:
https://github.com/hugary1995/jaxformtorch/blob/main/tests/test.cxx
I stopped there as I realized I would need to access some of the libTorch internals to get this to work for the most general case.

### Alternatives

_No response_

### Additional context

_No response_

cc @jbschlosser @zou3519 @Chillee @samdow @kshitij12345 @janeyx99
DISABLED test_cat_addmm (__main__.TestDoBench)
Labels: module: rocm, triaged, skipped

Platforms: rocm

This test was disabled because it is failing on master ([recent examples](http://torch-ci.com/failure/inductor%2Ftest_max_autotune.py%3A%3ATestDoBench%3A%3Atest_cat_addmm)).

Same issue as https://github.com/pytorch/pytorch/issues/106026, still investigating root cause. Currently seems like the triton template tests will fail on ROCm GPUs upstream due to lower number of SMs to return True in https://github.com/pytorch/pytorch/blob/main/torch/_inductor/utils.py#L655

cc: @jithunnair-amd 

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @hongxiayang
Fix Clang compilation error with Lib ATen for ppc64le
Labels: module: cpu, triaged, open source

This patch fixes error while compiling with Clang for ppc64le
I have used clang version 15.0.7 
Errors are as follow:
No matching function for call to 'vec_sel’
No matching function for call to 'vec_splats'
Excess elements in scalar initializer
Use of undeclared identifier 'vec_vsubudm'
Fix for multiple error within int64_t  DEFINE_MEMBER_OP_AND_ONE

References:
https://releases.llvm.org/9.0.0/tools/clang/docs/AttributeReference.html
https://reviews.llvm.org/D81083


cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10
[dynamo] Support dict.get with no default specified, and dict.copy
Labels: triaged, open source, Stale, module: dynamo, ciflow/inductor

Previously, `.get(key, default)` was supported, but `.get(key)` was not.

Also adds `.copy`

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng @anijain2305 @Xia-Weiwen @ipiszy
DISABLED test_aot_sequence_nr_dynamic_shapes (dynamo.test_aot_autograd.DynamicShapesAotAutogradFallbackTests)
Labels: triaged, module: flaky-tests, skipped, module: dynamo


This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_aot_sequence_nr_dynamic_shapes&suite=DynamicShapesAotAutogradFallbackTests) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/15538280405).

Over the past 3 hours, it has been determined flaky in 6 workflow(s) with 18 failures and 6 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_aot_sequence_nr_dynamic_shapes`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `dynamo/test_dynamic_shapes.py`

cc @voznesenskym @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @aakhundov
[PG NCCL][RFC] Pause before throwing exception
Labels: fb-exported, release notes: distributed (c10d)

Summary:
When this exception is thrown, torchx will kick in and take down the
remaining processes. Adding a bit of time before throwing it so that other
processes can also have their watchdogs fire, which can better contribute to
log understanding in terms of potential collective mismatches, shape
misalignment, etc.

Tested by running some jobs and ensuring all the watchdogs fire and report
errors in logs when collectives are misaligned, instead of just one.

Test Plan: CI

Differential Revision: D47975982
[JIT] .item() dict keys cause `RuntimeError: Cannot create dict for key type 'Scalar', only int, float, complex, Tensor, device and string keys are supported`
Labels: oncall: jit

### 🐛 Describe the bug

**Tasks**:
* [ ] (maybe) fix this - seems hard because torchscript doesn't know whether a tensor is a FloatTensor or an IntTensor
* [ ] Improve source attribution: maybe this error should be a FrontendError(? is that what it's called?) that traces back to the source that had this issue.

Repro:

```python
import torch
from typing import List, Tuple, Dict

def fn(x: List[torch.Tensor]):
    return {x[i].item(): i for i in range(len(x))}

torch.jit.script(fn)
```

Error:
```
Traceback (most recent call last):
  File "/data/users/dberard/scripts/dict_comprehension.py", line 7, in <module>
    torch.jit.script(fn)
  File "/data/users/dberard/pytorch/torch/jit/_script.py", line 1377, in script
    fn = torch._C._jit_script_compile(
RuntimeError: Cannot create dict for key type 'Scalar', only int, float, complex, Tensor, device and string keys are supported
```

### Versions

main branch as of aug 1, 2023

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel
inconsistent dtype of scale and zero_point in observers
Labels: high priority, triage review, oncall: quantization, triaged

### 🐛 Describe the bug

in https://github.com/pytorch/pytorch/blob/main/torch/ao/quantization/observer.py#L323-L324 scale is float (32bit) and zero_point is long (64 bit), we should make sure both of them are 32 bit

### Versions

master

cc @ezyang @gchanan @zou3519 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel
`torch.nn.utils.clip_grad_norm_()` causes H2D sync with foreach ops.
Labels: module: nn, triaged, module: mta

`clip_grad_norm_()` has an optional `foreach` flag ([here](https://github.com/pytorch/pytorch/blob/ef0576f203bff957d7be976840244f45ff510d2c/torch/nn/utils/clip_grad.py#L14)), that specifies that a foreach kernel should be used.

Unfortunately, that foreach kernel (called [here](https://github.com/pytorch/pytorch/blob/ef0576f203bff957d7be976840244f45ff510d2c/torch/nn/utils/clip_grad.py#L76)) performs a H2D sync:

(1) `clip_coef_clamped` is a scalar-tensor

(2) `aten._foreach_mul_` has a `Scalar` overload and a `TensorList` overload, but not a `Tensor` overload.

(3) `torch._foreach_mul_` coerces the scalar-tensor into a tensor to call the Scalar overload, resulting in an `item()` call (causing a H2D sync).

I had a tentative patch up here to try to force `clip_grad_norm_()` to use the existing `_foreach_mul_.List` overload. It avoids the H2D sync, but ends up dispatching several smaller kernels, which is still not ideal for performance. Instead, we should add a dedicated `_foreach_mul_.Tensor` overload.

cc @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki @crcrpar @mcarilli
PyTorchMPS not showing up in Instruments for `torch.mps.profiler`
Labels: triaged, module: mps

### 🐛 Describe the bug

Following these instructions: https://developer.apple.com/videos/play/wwdc2023/10050/?time=355, I'm not seeing the `PyTorchMPS` intervals showing up when trying to profile SDXL. I can see `events` in the `com.apple.Metal.AGXSignposts` but  I would expect to see PyTorchMPS here, with intervals breaking down each layer's latency.

<img width="618" alt="image" src="https://github.com/pytorch/pytorch/assets/1981179/6de491b4-5cbb-46eb-b614-96a7419cc85f">

Here's the code I'm using to profile:

```
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0")
pipe.to("mps")

prompt = "test prompt"
negative_prompt = "test negative prompt"

torch.mps.profiler.start(mode='interval', wait_until_completed=True)
print(os.getpid())
image = pipe(prompt, negative_prompt=negative_prompt, num_inference_steps=1).images[0]
torch.mps.profiler.stop()

```

This may be related to macOS 14 beta? I'm calling this from a notebook being run via VSCode, but I tested just running it from a standalone python file via iTerm2 and similar issue.


### Versions

PyTorch version: 2.1.0.dev20230801
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 14.0 (arm64)
GCC version: Could not collect
Clang version: 14.0.3 (clang-1403.0.22.14.1)
CMake version: version 3.26.3
Libc version: N/A

Python version: 3.10.12 (main, Jun 20 2023, 19:43:52) [Clang 14.0.3 (clang-1403.0.22.14.1)] (64-bit runtime)
Python platform: macOS-14.0-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M2 Pro

Versions of relevant libraries:
[pip3] clip-anytorch==2.5.2
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.23.5
[pip3] open-clip-torch==2.20.0
[pip3] pytorch-lightning==1.7.7
[pip3] torch==2.1.0.dev20230801
[pip3] torchaudio==2.1.0.dev20230801
[pip3] torchdiffeq==0.2.3
[pip3] torchmetrics==0.11.4
[pip3] torchsde==0.2.5
[pip3] torchvision==0.16.0.dev20230801
[conda] numpy                     1.24.3          py311hd0e4f0d_0
[conda] numpy-base                1.24.3          py311h947b413_0
[conda] numpydoc                  1.5.0           py311hecd8cb5_0

cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
[Pytorch][Vulkan] aten::gt
Labels: fb-exported, Stale, module: vulkan, release notes: vulkan, ciflow/periodic

Summary:
Add support for Vulkan [greater than](https://pytorch.org/docs/stable/generated/torch.gt.html) and its variants.

Note, gt is broadcastable one-way:
```
The second argument can be a number or a tensor whose shape is broadcastable with the first argument.
```

Also check scalar type matches. For non-inplace variants, output is a bool tensor. For in-place variants, an int tensor (same as input).

Test Plan:
New tests:
```
lfq@lfq-mbp fbsource % buck run --target-platforms ovr_config//platform/macos:arm64-fbsource //xplat/caffe2:pt_vulkan_api_test_binAppleMac\#macosx-arm64 -c pt.vulkan_full_precision=1 -- --gtest_filter="*gt*"
Building: finished in 0.2 sec (100%) 265/265 jobs, 0/265 updated
  Total time: 0.2 sec
BUILD SUCCEEDED
Running main() from xplat/third-party/gmock/googletest-1.12.1/googletest/src/gtest_main.cc
Note: Google Test filter = *gt*
[==========] Running 7 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 7 tests from VulkanAPITest
[ RUN      ] VulkanAPITest.gt
[       OK ] VulkanAPITest.gt (67 ms)
[ RUN      ] VulkanAPITest.gt_broadcast
[       OK ] VulkanAPITest.gt_broadcast (1 ms)
[ RUN      ] VulkanAPITest.gt_broadcast_invalid
[       OK ] VulkanAPITest.gt_broadcast_invalid (0 ms)
[ RUN      ] VulkanAPITest.gt_
[       OK ] VulkanAPITest.gt_ (3 ms)
[ RUN      ] VulkanAPITest.gt_broadcast_
[       OK ] VulkanAPITest.gt_broadcast_ (0 ms)
[ RUN      ] VulkanAPITest.gt_scalar
[       OK ] VulkanAPITest.gt_scalar (2 ms)
[ RUN      ] VulkanAPITest.gt_scalar_
[       OK ] VulkanAPITest.gt_scalar_ (1 ms)
[----------] 7 tests from VulkanAPITest (77 ms total)

[----------] Global test environment tear-down
[==========] 7 tests from 1 test suite ran. (77 ms total)
[  PASSED  ] 7 tests
```

All tests: https://www.internalfb.com/phabricator/paste/view/P797564233

```
xplat/caffe2/aten/src/ATen/test/vulkan_api_test.cpp:6751: Skipped
QueryPool is not available
[  SKIPPED ] VulkanAPITest.querypool_flushed_shader_log (0 ms)
[----------] 324 tests from VulkanAPITest (6024 ms total)

[----------] Global test environment tear-down
[==========] 324 tests from 1 test suite ran. (6024 ms total)
[  PASSED  ] 323 tests.
[  SKIPPED ] 1 test, listed below:
[  SKIPPED ] VulkanAPITest.querypool_flushed_shader_log
```

clang-format on `BinaryOp.cpp`

Reviewed By: SS-JIA

Differential Revision: D47767546
[torch.compile] autograd.Function with multiple return values
Labels: triaged, oncall: pt2, module: dynamo

### 🐛 Describe the bug

```
import torch
import torch._dynamo

class Foo(torch.autograd.Function):
    @staticmethod
    def forward(ctx, x):
        ctx.save_for_backward(x)
        return x.sin(), x.sin()
    @staticmethod
    def backward(ctx, grad0, grad1):
        x, = ctx.saved_tensors
        return grad * x.cos(), grad * x.cos()

@torch.compile(backend='aot_eager', fullgraph=True)
def f(x):
    return Foo.apply(x)

x = torch.randn([], requires_grad=True)
f(x)
```
produces:
```
le.call_function(self, tx, args, kwargs)
    771     return None
    773 p_args = (
    774     *(arg.as_proxy() for arg in args),
    775     *(arg for arg in body_lifted_freevars.keys()),
    776 )
--> 777 r = body_r.as_proxy().node.meta["example_value"]
    778 example_value = r
    780 _, p_kwargs = proxy_args_kwargs([], kwargs)

InternalTorchDynamoError: 'tuple' object has no attribute 'node'

```

### Error logs

_No response_

### Minified repro

_No response_

### Versions

main

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @aakhundov
Better export story for autograd.Function?
Labels: triaged, oncall: pt2

### 🐛 Describe the bug

A trampoline appears in the exported graph of autograd.Function:
```py
import torch
import torch._dynamo

class Foo(torch.autograd.Function):
    @staticmethod
    def forward(ctx, x):
        ctx.save_for_backward(x)
        return x.sin()
    @staticmethod
    def backward(ctx, grad):
        x, = ctx.saved_tensors
        return grad * x.cos()

def f(x):
    return Foo.apply(x)

x = torch.randn([], requires_grad=True)
gm, *_ = torch._dynamo.export(f, x)
print(gm.code)
```
returns:
```
def forward(self, x):
    arg0, = fx_pytree.tree_flatten_spec(([x], {}), self._in_spec)
    function_ctx = torch.autograd.function.FunctionCtx()
    trampoline_autograd_apply = torch__dynamo_variables_misc_trampoline_autograd_apply(arg0);  arg0 = None
    return pytree.tree_unflatten([trampoline_autograd_apply], self._out_spec)
```

We probably want to turn autograd.Function into a legit HigherOrderOp at some point.

### Error logs

_No response_

### Minified repro

_No response_

### Versions

main

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305
[torch.compile] autograd.Function where we assign a Tensor directly to ctx
Labels: triaged, oncall: pt2, module: dynamo

### 🐛 Describe the bug

```py
import torch

class Foo(torch.autograd.Function):
    @staticmethod
    def forward(ctx, x):
        ctx.x = x
        return x.sin()
    @staticmethod
    def backward(ctx, grad):
        return grad * ctx.x.cos()

@torch.compile(backend='aot_eager')
def f(x):
    return Foo.apply(x)

x = torch.randn([], requires_grad=True)
f(x)
```
produces:
```
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING] WON'T CONVERT f <ipython-input-1-1dbbf3a0bb5e> line 12
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING] due to:
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING] Traceback (most recent call last):
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING]   File "/raid/rzou/pt/debug-cpu/torch/_dynamo/output_gr
aph.py", line 1302, in lift_tracked_freevar_to_input
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING]     assert (
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING] AssertionError: lift_tracked_freevar_to_input should no
t be called on root SubgraphTracer
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING]
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING] from user code:
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING]    File "/raid/rzou/pt/debug-cpu/torch/_dynamo/variable
s/misc.py", line 241, in trampoline_autograd_bwd
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING]     return fn_cls.backward(*args, **kwargs)
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING]   File "<ipython-input-1-1dbbf3a0bb5e>", line 10, in ba
ckward
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING]     return grad * ctx.x.cos()
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING]
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING] Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for
more information
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING]
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING]
[2023-08-01 08:54:23,054] torch._dynamo.convert_frame: [WARNING] converting frame raised error, suppressing error
```

### Error logs

_No response_

### Minified repro

_No response_

### Versions

main

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @aakhundov
Installing torchvision for CPU leads to unwanted upgrade of torch + pip would not install nightly as considers that release is the latest (?)
Labels: oncall: binaries, triaged

### 🐛 Describe the bug

I have torch cpu install '2.0.0+cpu' (laptop without CUDA/GPU; no torchvision installed). Doing official `python3 -m pip install torchvision --index-url https://download.pytorch.org/whl/cpu` command downloads `torchvision-0.15.2%2Bcpu-cp38-cp38-linux_x86_64.whl` and then leads to undesired large upgrade of torch core to `torch-2.0.1%2Bcpu-cp38-cp38-linux_x86_64.whl`. Is it expected? Can it install automatically the good version of torchvision?  Or should one manually specify the right version from https://pytorch.org/get-started/previous-versions/?

Also, `torchvision` seems to have `requests` as a hard dependency, which is not ideal, as `requests` module can sometimes be broken (because of broken certificate intalls etc)? I can understand that this module can be required when calling some dataset/model download, but not at general `import torchvision.transforms.functional as TF`

### Versions

2.0.0+cpu

cc @seemethere @malfet
Command to reproduce error is incorrect
Labels: good first issue, module: tests, triaged, module: infra, module: testing

### 🐛 Describe the bug


```
To execute this test, run the following from the base repo dir: TEST_WITH_TORCHINDUCTOR=1 python test/test_torch.py -k test_nondeterministic_alert_NLLLoss_cuda
```

But I had to run it with 
```
PYTORCH_TEST_WITH_INDUCTOR=1 python test/test_torch.py -k test_nondeterministic_alert_NLLLoss_cuda
```

Ref CI failure: https://github.com/pytorch/pytorch/actions/runs/5678841216/job/15391208767?pr=106118
Ref PR: https://github.com/pytorch/pytorch/pull/106118

### Versions

master

cc @mruberry
nll_loss reference shouldn't be registered as a decomposition.
Labels: triaged, module: primTorch, module: decompositions

### 🐛 Describe the bug

`nll_loss` is a CompositeImplicit Op

https://github.com/pytorch/pytorch/blob/16df54239fecb97afbda30026931bbcac09542c4/aten/src/ATen/native/native_functions.yaml#L11043-L11046

But it is also registered as a decomposition:
https://github.com/pytorch/pytorch/blob/16df54239fecb97afbda30026931bbcac09542c4/torch/_refs/nn/functional/__init__.py#L734-L740

So, `nll_loss` shouldn't be registered as decomposition and the reference should be similar to actual implementation calling into other ops.

Also, note that this is a ref for `torch.nn.functional.nll_loss` which calls into `nll_loss_nd`.

cc: @lezcano 

### Versions

master

cc @ezyang @mruberry @Lezcano @peterbell10 @SherlockNoMad
[ONNX] scatter_reduce does not support `include_self=False`
Labels: module: onnx, triaged

### 🐛 Describe the bug

the latest verison pytorch[2023-08-01] has support aten::scatter_reduce operator, when export the model to ONNX. but occurs error:
raise errors.OnnxExporterError(
    "ONNX does not support include_self=False for scatter_reduce"
)
how to fix the error?

### Versions

Versions of relevant libraries:
[pip3] numpy==1.24.4
[pip3] pytorch-triton==2.1.0+e6216047b8
[pip3] torch==2.1.0.dev20230731+cu118
[pip3] torchaudio==2.1.0.dev20230731+cu118
[pip3] torchvision==0.16.0.dev20230731+cu118
[conda] numpy                     1.24.4                   pypi_0    pypi
[conda] pytorch-triton            2.1.0+e6216047b8          pypi_0    pypi
[conda] torch                     2.1.0.dev20230731+cu118          pypi_0    pypi
[conda] torchaudio                2.1.0.dev20230731+cu118          pypi_0    pypi
[conda] torchvision               0.16.0.dev20230731+cu118          pypi_0    pypi
Link compiled protobuf files to `protobuf::libprotobuf`
Labels: triaged, open source, Stale

This correctly propagates the required compile flags (such as `-DPROTOBUF_USE_DLLS` for shared builds of protobuf).

The manual extraction of the include path and adding that globally is also no longer required as that is part of that targets interface.

Fixes #106297   
Fixes #106206
Calling ops.aten.embedding_bag() function got silent crash
Labels: module: crash, module: nn, triaged

### 🐛 Describe the bug

```
def test2():
    import torch as t

    weight = t.tensor([
        [ 8.4520, -1.2693, -0.0696, -2.0721, -7.5149],
        [ 4.3191, -8.9345,  5.5872,  6.7340,  8.5114],
        [-2.1229, -7.3948,  2.0235,  4.9718, -8.9578],
        [-2.0429, -5.3951, -0.7872, -4.4299, -3.6789],
        [-2.8571, -8.5527,  7.3846,  7.5450, -1.4118],
        [-1.0249, -3.6731, -8.1276, -8.7583,  3.3449],
        [-4.9414, -5.7859, -0.7022, -2.9971, -2.9117],
        [ 0.2892, -1.9090, -3.0988, -4.3093, -7.3244],
        [ 7.5466, -3.6017,  2.3848, -3.1227,  0.7314],
        [ 8.3907,  4.1465, -7.7994,  3.5721,  8.5432]
    ])

    indices = t.tensor([
        [1, 6, 3, 6, 2],
        [5, 8, 4, 2, 8],
        [3, 0, 5, 2, 4],
        [8, 4, 3, 1, 2],
        [0, 3, 4, 2, 2]      
    ])
    offsets = t.tensor([0, 2])
    mode = 2
    r = t.ops.aten.embedding_bag(
            weight, indices, offsets, mode=mode)
    print(r)
test2()
```

### Versions

2.0.1+CPU

cc @ezyang @gchanan @zou3519 @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki
DISABLED test_ddp_apply_optim_in_backward_ignored_params (__main__.TestDistBackendWithSpawn)
Labels: oncall: distributed, module: flaky-tests, skipped

Platforms: linux

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_ddp_apply_optim_in_backward_ignored_params&suite=TestDistBackendWithSpawn) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/15502591692).

Over the past 3 hours, it has been determined flaky in 2 workflow(s) with 2 failures and 2 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_ddp_apply_optim_in_backward_ignored_params`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `distributed/test_distributed_spawn.py`

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu