gopikrsmscs/torch-issues huggingface dataset:
Add ``onnx`` backend to ``torch.export`` API
Labels: module: onnx, open source, release notes: onnx, module: dynamo, ciflow/inductor, module: export

Related to https://github.com/pytorch/pytorch/issues/109131

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #109650
* #109649

This PR adds the ONNX exporter backend for ``torch.export``.
The ONNX exporter will leverage Torch Dynamo under the hood, but produce
a ONNX model as a result of the export. The ONNX backend will also
enable ONNX Runtime to transparently execute the ONNX model.

An example usage is shown below:

```python
import torch

def func(x):
    y = x + 1
    z = y.relu()
    return (y, z)
inputs = (torch.randn(1, 1, 2),)
export_options = torch.onnx.ExportOptions()
exported_onnx = torch.export.export(func, inputs, backend="onnx", options={"export_options":  export_options})

onnxruntime_output = exported_onnx(*inputs)
```

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Add `backend` concept to `torch.export` API
Labels: module: bc-breaking, open source, topic: bc breaking, module: dynamo, module: export, suppress-bc-linter

Related to https://github.com/pytorch/pytorch/issues/109131

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #109650
* __->__ #109649

This PR proposes a way to allow ``torch.export.export`` API to support
multiple backends, similar in principle to the ``torch.compile`` API.

With the new API, one backend can produce, for example, ``GraphModule`` (default backend?)
while the ``onnx`` backend can produce a ``onnx.ModelProto``, unifying all export APIs
from different vendors into one cohesive API.

Backend implementation are separate as it is today, and users could register
custom backends through ``register_backend`` or
list the available ones through ``list_backends``

Below is an example on how a model would be exported with the new API

```python
import torch

def f(x, y):
    return x[0] + y

inp = ([torch.ones(1, 3)], torch.ones(1, 3))

exported_program = torch.export.export(f, inp, backend="dynamo", options={})  # ``backend`` and ``options`` are optionals
output = exported_program(*inp)
```

Besides the main `torch.export.export` API, there are others that need to be discussed:

* `torch.export.load` and `torch.export.save` are also backend specific and ONNX backend, for example, could also implement the same functionalities
* `constrain_as_size`, `constrain_as_value`, `dynamic_dim` that may or may not be reused by other backends

cc @ezyang @gchanan @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Enable masked_scatter_backward for inductor
Labels: module: inductor, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #109642

masked_scatter_backward was previously implemented as a
CompositeExplicitAutograd, which involved a decomp that calls
masked_select, and masked_select in general produces data-dependent
shapes that inductor doesn't support. But masked_scatter_backward
reshapes the return value of masked_select such that the end result has
a static shape again.

I have converted masked_scatter_backward into an aten op to avoid this
issue.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
dynamo: break graph when "out" has complex dtype
Labels: open source, module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #109633



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
[Inductor] Move fake_tensors to the same device as example_inputs
Labels: topic: not user facing, module: inductor, ciflow/inductor

When I torch.export a model in GPU, and then AOTInductor compile it with CPU sample_inputs, I expect hitting the CPU compilation path, however, I got triton compiled path. 

This PR fixes this. 

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
AsyncCompile loses useful exception backtrace in __get_result
Labels: good first issue, module: logging, triaged, oncall: pt2, module: inductor

### 🐛 Describe the bug

Steps to reproduce:

1. Inject a synthetic failure in some code that is invoked solely from the generated wrapper py file. E.g.,

```
diff --git a/torch/_inductor/triton_heuristics.py b/torch/_inductor/triton_heuristics.py
index badacf0d2a4..7364a5b8909 100644
--- a/torch/_inductor/triton_heuristics.py
+++ b/torch/_inductor/triton_heuristics.py
@@ -961,6 +961,7 @@ def pointwise(size_hints, meta, tile_hint=None, filename=None):
 def reduction(size_hints, reduction_hint=False, meta=None, filename=None):
     """args to @triton.heuristics()"""
     assert meta is not None
+    assert False
     rnumel = size_hints[-1]
     if len(size_hints) == 2:
         contiguous_config = triton_config_reduction(
```

2. Trigger this codepath in a test program

```
import torch

@torch.compile(dynamic=True)
def f(x):
    return (x + 1).sum()

f(torch.randn(20000, device='cuda'))
```

Here's the backtrace you get:

```
  File "/tmp/torchinductor_ezyang/ww/cwwnfs543wa53bsz7p6na72nsllphuso5hdekryzodahas6hcwlu.py", line 112, in <module>
    async_compile.wait(globals())
  File "/data/users/ezyang/a/pytorch/torch/_inductor/codecache.py", line 1858, in wait
    scope[key] = result.result()
  File "/data/users/ezyang/a/pytorch/torch/_inductor/codecache.py", line 1705, in result
    self.future.result()
  File "/home/ezyang/local/a/pytorch-env/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/home/ezyang/local/a/pytorch-env/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
AssertionError: 
```

This is useless, we lost the stack trace associated with the exception in the other process.

In small scale cases, you can recover useful information with `TORCHINDUCTOR_COMPILE_THREADS=1 python n.py`

```
  File "/data/users/ezyang/a/pytorch/torch/_inductor/codecache.py", line 1415, in load
    mod = PyCodeCache.load(source_code)
  File "/data/users/ezyang/a/pytorch/torch/_inductor/codecache.py", line 1281, in load
    return cls.load_by_key_path(key, path, linemap)
  File "/data/users/ezyang/a/pytorch/torch/_inductor/codecache.py", line 1303, in load_by_key_path
    exec(code, mod.__dict__, mod.__dict__)
  File "/tmp/torchinductor_ezyang/i3/ci3j5qcext3jp3rjk5l45e5smehavbnh4cs36yj7zxe5th7tg2gx.py", line 10, in <module>
    @reduction(
  File "/data/users/ezyang/a/pytorch/torch/_inductor/triton_heuristics.py", line 964, in reduction
    assert False
torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
AssertionError: 
```

which is a nice workaround, but we should just fix this properly.

### Versions

main

cc @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
[inductor] Update triton pin
Labels: open source, with-ssh, ciflow/trunk, topic: not user facing, module: inductor, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #110911
* #109132
* #106581
* __->__ #109601



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler
ensure uint8 is honoured for cpu operations in dynamo
Labels: module: cpu, triaged, open source, module: amp (automated mixed precision), NNC, release notes: quantization, topic: not user facing, ciflow/mps, module: inductor, module: dynamo, ciflow/inductor, module: export

Fixes #105264 

follow on from pull request #108513 


cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @mcarilli @ptrblck @leslie-fang-intel @EikanWang @voznesenskym @penguinwu @Guobing-Chen @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
test_memory_timeline fails on PPC due to extra temopraries
Labels: triaged, module: POWER, oncall: profiler

### 🐛 Describe the bug

Running the test on PPC64le with `python profiler/test_memory_profiler.py  -k test_memory_timeline` results in a failure. Printing the full diff yields an extra temporary between the first 2:
```
+             create                     TEMPORARY                   11(v0)          0.7 kB
+             destroy                    TEMPORARY                   11(v0)          0.7 kB
```

Raising the minimum size from 256 to 1024 to filter this allocation makes the test pass. So I assume it is to sensitive.

### Versions

PyTorch version: 2.0.1
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Red Hat Enterprise Linux Server release 7.6 (Maipo) (ppc64le)
GCC version: (GCC) 12.2.0
Clang version: Could not collect
CMake version: version 3.24.3
Libc version: glibc-2.17

Python version: 3.10.8 (main, Jul 25 2023, 10:52:38) [GCC 12.2.0] (64-bit runtime)
Python platform: Linux-4.14.0-115.19.1.el7a.ppc64le-ppc64le-with-glibc2.17


cc @robieta @chaekit @aaronenyeshi @nbcsm @guotuofeng @guyang3532 @gaoteng-git @tiffzhaofb @dzhulgakov @davidberard98
[WIP] Trace model attribute mutation
Labels: module: dynamo, ciflow/inductor, module: export

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #109590



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
[dynamo][jagged tensor] Slow compilation time for a helper function of jagged tensor
Labels: triaged, oncall: pt2, module: dynamic shapes

### 🐛 Describe the bug

~~~
import copy
import torch


mylist = []

@torch.compile(backend="eager")
def foo(stride_per_key_per_rank):
    return [sum(stride) for stride in stride_per_key_per_rank]


x = list(range(100))
x = [copy.deepcopy(x) for _ in range(100)]
foo(x)
~~~

This type of pattern is common and is very expensive for Dynamo and is really slow. 

### Error logs

_No response_

### Minified repro

_No response_

### Versions

NA

cc @ezyang @msaroufim @wconstab @bdhirsh
Make Dropout take a dim=... argument
Labels: module: nn, triaged, needs research, has workaround

### 🚀 The feature, motivation and pitch

Currently PyTorch has Dropout, Dropout1d, Dropout2d and Dropout3d, for handling data with shape either shape (*, D), (N, C, D), (N, C, W, H) or (N, C, W, H, Z).
However none of them support the case where I want to do channel dropout but have shape, say, (B, N, C, D).
It would be much easier if the standard Dropout class just took a `dim` argument for specifying the level at which to do dropout.
For example, `dim=-1` could be the default, and would correspond to the current Dropout behavior.
`dim=-2` would be like Dropout1d, but support more than one batch dimensions, e.g. (B, N, C, D).

### Alternatives

_No response_

### Additional context

_No response_

cc @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki
torch.optim.Adafactor
Labels: feature, module: optimizer, triaged, needs research

### 🚀 The feature, motivation and pitch

Models are getting harder to fit on a limited number of GPUs and ADAM doesn't help since its memory overhead is 2N where N is the number of parameters in a model

![image](https://github.com/pytorch/pytorch/assets/3282513/00c82fb4-a811-4b4f-8135-8b752b46a9f4)

We don't like to merge optimizers in core because they rarely stand the test of time but ADAM has and a memory efficient alternative that's been in use at many startups I've talked to, Twitter and larger companies https://github.com/google-research/t5x/blob/main/t5x/adafactor.py, has been Adafactor, see the discussion here  https://twitter.com/HamelHusain/status/1702004478369743125 - it's also come up in a github issue before here https://github.com/pytorch/pytorch/issues/98434

Assigning to myself since I want a starter task in optimizers


### Alternatives

There is a working implementation in fairseq which Huggingface has borrowed https://github.com/facebookresearch/fairseq/blob/main/fairseq/optim/adafactor.py#L66 which is a good starting point

### Additional context

You might be thinking why merge yet another optimizer? Optimizers are plagued by lack of reproducibility and sensitivity to hyperparameters 

However, ADAM has stood the test of time but ADAM also has a high memory overhead, for each parameter you need to store the first and second moment so if your model has N parameters then your optimizer state is 2N. Also few people change its hyperparameters in fact the hyperparams have remained the same since torch days
* https://github.com/pytorch/pytorch/blob/main/torch/optim/adam.py
* https://github.com/torch/optim/blob/master/adam.lua

So as Blueberries projects move towards finetuning after PyTorch conference and as more members of the community try to fit larger models on their GPUs it's critical we find memory efficient alternative to ADAM which is why Adafactor might just be a strong candidate for a default optimizer we recommend to people finetuning with a limited amount of GPUs and this includes ourselves in the torchbench CI, Adafactor comes with its own LR scheduler and will help us not to have to dramatically reduce batch sizes in our benchmarks


cc @vincentqb @jbschlosser @albanD @janeyx99 @crcrpar
[Android: React Native] couldn't find DSO to load: libtorch-code-gen.so when loading model 
Labels: oncall: mobile

### 🐛 Describe the bug

![image](https://github.com/pytorch/pytorch/assets/67507961/06e94ac3-8ebe-45ab-ae81-1f05441758aa)

my build.gradle:
```
   .....
    // exclude to prevent duplication class
    implementation ('org.pytorch:pytorch_android_lite:1.9.0'){
        exclude module: 'fbjni-java-only'
    }
    implementation ('org.pytorch:pytorch_android_torchvision:1.9.0'){
        exclude module: 'fbjni-java-only'
    }
```

My code to load model
```
this.module = LiteModuleLoader.load(assetFilePath(context, "optimized_model.ptl"));
```

### Versions

Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 Pro
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.11.3 (tags/v3.11.3:f3909b8, Apr  4 2023, 23:49:59) [MSC v.1934 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19045-SP0
Is CUDA available: N/A
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

CPU:
Architecture=9
CurrentClockSpeed=1382
DeviceID=CPU0
Family=205
L2CacheSize=5120
L2CacheSpeed=
Manufacturer=GenuineIntel
MaxClockSpeed=1382
Name=11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
ProcessorType=3
Revision=

Versions of relevant libraries:
[pip3] numpy==1.25.2
[conda] Could not collect

```[tasklist]
### Tasks
```
Add Half support for AvgPool2d on CPU
Labels: module: cpu, open source, module: half, ciflow/trunk, topic: not user facing, ciflow/mps, ciflow/inductor

Add Half support for AvgPool2d (both channels last and channels first) on CPU

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10
[Not for merge][Repro] Unbacked symint in Inductor size_hint output
Labels: module: inductor, ciflow/inductor

Repro script:
```python
# Test 1: tolist -> zeros -> cat

import torch
import torch.export
import torch._dynamo.config

def f(x):
    a, b = x.tolist()
    torch.export.constrain_as_size(a)
    torch.export.constrain_as_size(b)
    return torch.cat([torch.zeros(a, device="cuda"), torch.zeros(b, device="cuda")])

with (
    torch._dynamo.config.patch(
        dynamic_shapes=True,
        capture_dynamic_output_shape_ops=True,
        capture_scalar_outputs=True,
    ),
):
    compiled = torch.compile(f, fullgraph=True, dynamic=True)
    x = torch.tensor([1, 4], device="cuda")
    ref = f(x)
    actual = compiled(x)
    print(f"ref: {ref}, actual: {actual}")
    assert torch.allclose(ref, actual)
```

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @chenyang78 @kadeng @muchulee8 @aakhundov
[fake/meta] Bad meta kernel for conv1d
Labels: triaged, oncall: pt2, module: fakeTensor

### 🐛 Describe the bug

```
import torch

class M(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = torch.nn.Conv1d(6, 6, 6, groups=2)

    def forward(self, x):
        return self.conv(x)

mod = M()
opt_mod = torch.compile(mod, backend="aot_eager")
opt_mod(torch.rand(1, mod.conv.in_channels, mod.conv.weight.shape[1]))
```

### Error logs

```
  File "/data/users/anijain/pytorch/torch/_dynamo/utils.py", line 1416, in run_node
    return nnmodule(*args, **kwargs)
  File "/data/users/anijain/pytorch/torch/nn/modules/module.py", line 1519, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/users/anijain/pytorch/torch/nn/modules/module.py", line 1528, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/users/anijain/pytorch/torch/nn/modules/conv.py", line 310, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/data/users/anijain/pytorch/torch/nn/modules/conv.py", line 306, in _conv_forward
    return F.conv1d(input, weight, bias, self.stride,
  File "/data/users/anijain/pytorch/torch/utils/_stats.py", line 20, in wrapper
    return fn(*args, **kwargs)
  File "/data/users/anijain/pytorch/torch/_subclasses/fake_tensor.py", line 1304, in __torch_dispatch__
    return self.dispatch(func, types, args, kwargs)
  File "/data/users/anijain/pytorch/torch/_subclasses/fake_tensor.py", line 1547, in dispatch
    op_impl_out = op_impl(self, func, *args, **kwargs)
  File "/data/users/anijain/pytorch/torch/_subclasses/fake_tensor.py", line 737, in conv
    out = func(**kwargs)
  File "/data/users/anijain/pytorch/torch/_ops.py", line 458, in __call__
    return self._op(*args, **kwargs or {})
  File "/data/users/anijain/pytorch/torch/_meta_registrations.py", line 2021, in meta_conv
    out = input_tensor.new_empty(shape_out)
  File "/data/users/anijain/pytorch/torch/_refs/__init__.py", line 4569, in new_empty
    return torch.empty(
torch._dynamo.exc.TorchRuntimeError: Failed running call_module L__self___conv(*(FakeTensor(..., size=(1, 6, 3)),), **{}):
Trying to create tensor with negative dimension -2: [1, 6, -2]

```

### Minified repro

_No response_

### Versions

NA

cc @ezyang @msaroufim @wconstab @bdhirsh @eellison
[wip]: fsspec remote code cache
Labels: topic: not user facing, module: inductor, ciflow/inductor

The applications for this PR are around reducing cold start times
1. Going from training to inference in case you're using the same machine
2. Distributed
3. CI where the majority of some code does not change and you'd like to reduce test time

fsspec is amazing and can support the widest possible amount of file systems in my tests i mocked s3 and local paths but more setups could work, it's already a pytorch dependency https://download.pytorch.org/whl/nightly/fsspec/ 

In case test is skipped in CI

```
(sourcetorch) ubuntu@ip-172-31-1-136:~/pytorch/test/inductor$ pytest -s test_cache.py
============================================================================ test session starts ============================================================================
platform linux -- Python 3.10.10, pytest-7.3.1, pluggy-1.0.0
rootdir: /home/ubuntu/pytorch
configfile: pytest.ini
plugins: hypothesis-6.75.1
collected 4 items                                                                                                                                                           

test_cache.py ....

============================================================================= 4 passed in 2.74s =============================================================================
(sourcetorch) ubuntu@ip-172-31-1-136:~/pytorch/test/inductor$ 
```


cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
Torch FX SubgraphMatcher Any / Oneof Patterns
Labels: triaged, module: fx, module: fx.passes

### 🚀 The feature, motivation and pitch

I'm using the [SubgraphMatcher](https://github.com/pytorch/pytorch/blob/aed9bee0413dac190452fbfa9ab2a44b6e6843f5/torch/fx/passes/utils/matcher_utils.py#L51) to match torch.fx graphs.  I'd like to have some sort of set of nodes functionality when authoring patterns (ie match a convolution -> batch norm -> {relu, gelu, ... any activation}), without having to author one pattern per node type.

This was [previously implemented](https://github.com/pytorch/pytorch/pull/82853), but [then backed out](https://github.com/pytorch/pytorch/pull/83922) (for reasons I did not see).  Is this on the roadmap, or is there an alternative way to achieve this functionality that I'm not aware of?

### Alternatives

_No response_

### Additional context

_No response_

cc @ezyang @SherlockNoMad @EikanWang @jgong5 @wenzhe-nrv
_assert_bound_is_rational can fail
Labels: triaged, module: dynamic shapes

### 🐛 Describe the bug

This test

```
    def test_tensor_symfloat(self):
        def f(a):
            r = torch.tensor(a.size(0) * 1.0)
            assert r.dtype is torch.float
            return r

        r = str(make_fx(f, tracing_mode="symbolic")(torch.randn(2)).code).strip()
        # NB: this specializes, which is fine, the point is to make sure the
        # dtype inference is correct
        self.assertExpectedInline(r, """""")
```

fails with

```
  File "/data/users/ezyang/a/pytorch/test/test_proxy_tensor.py", line 1052, in f
    r = torch.tensor(a.size(0) * 1.0)
  File "/data/users/ezyang/a/pytorch/torch/fx/experimental/symbolic_shapes.py", line 991, in guard_float
    r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node)
  File "/data/users/ezyang/a/pytorch/torch/fx/experimental/recording.py", line 227, in wrapper
    return fn(*args, **kwargs)
  File "/data/users/ezyang/a/pytorch/torch/fx/experimental/symbolic_shapes.py", line 3798, in evaluate_expr
    static_expr = self._maybe_evaluate_static(expr)
  File "/data/users/ezyang/a/pytorch/torch/fx/experimental/symbolic_shapes.py", line 1557, in wrapper
    return fn_cache(self, *args, **kwargs)
  File "/data/users/ezyang/a/pytorch/torch/fx/experimental/symbolic_shapes.py", line 3478, in _maybe_evaluate_static
    _assert_bound_is_rational(new_expr, out)
  File "/data/users/ezyang/a/pytorch/torch/fx/experimental/symbolic_shapes.py", line 603, in _assert_bound_is_rational
    assert bound.lower.is_rational or bound.lower.is_Boolean or not bound.lower.is_finite or expr.has(sympy.Pow), (bound, expr)
AssertionError: (ValueRanges(lower=2.00000000000000, upper=9.22337203685478e+18, is_bool=False), 1.0*shape_0 + 1.0)
```

cc @ysiraichi 

### Versions

main
[dynamo] torch._dynamo.exc.Unsupported: call_function BuiltinVariable(setattr) [TensorVariable(), ConstantVariable(str), ConstantVariable(bool)] {}
Labels: triaged, oncall: pt2, module: aotdispatch, module: dynamo

### 🐛 Describe the bug

Occurs when you do

```
foo.requires_grad = True
```

in torch.compile'd region.

Occurred while implementing torch.tensor polyfill.

### Versions

main

cc @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
[dynamo] torch._dynamo.exc.Unsupported: comparison SymNodeVariable() <built-in function is_> ListVariable()
Labels: good first issue, triaged, oncall: pt2, module: dynamo

### 🐛 Describe the bug

Self explanatory.

Occurred when implementing self-referentiality check in torch.tensor ref

```
            if cur_item is obj:
                raise TypeError("new(): self-referential lists are incompatible")
```

### Versions

main

cc @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Report NameError when name is not defined, rather than unimplemented
Labels: module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #109503

Signed-off-by: Edward Z. Yang <ezyang@meta.com>

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Avoid cuda stubs libraries being RPATHed
Labels: triaged, module: mkldnn, open source, release notes: build

Rebase of #87593 which is a continuation of #68912

The new function `link_cuda_libraries` is a drop-in replacement for `target_link_libraries` which calls the latter and does the check and manual-rpath-setting on the libraries.
I also made it to work on CMake prior to 3.20 by using `get_filename_component` instead of `cmake_path`

Fixes #35418

@kumpera in the other PR you mentioned that some changes should/need to be split off. How would this need to look like? The changes depend on each other: The 3 CMakeLists.txt require the new module/file in the `cmake` folder and the `test_torch.py` will fail without those changes.

cc @gujinghui @PenghuiCheng @XiaobingSuper @jianyuh @jgong5 @mingfeima @sanchitintel @ashokei @jingxu10 @min-jean-cho @yanbing-j @Guobing-Chen @Xia-Weiwen
[inductor] Remove `is_big_gpu` check
Labels: triaged, open source, module: inductor, ciflow/inductor

Fixes https://github.com/pytorch/pytorch/issues/109489

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
Fix MultiProcess failure on nodes with 1 GPU
Labels: triaged, open source

The decorator(s) is written to `sys.exit` when the test function is called which is AFTER the `setup` call which forks the processes and uses (potentially) a GPU/NCCL based barrier which requires "n GPUs" to be present befor checking if "n GPUs" are available.

Rewrite those decorators to use `unittest.skipIf` which will not even enter the `setup` function.
This also exposed that `require_n_gpus_for_nccl_backend` is the same as `nccl_skip_if_lt_x_gpu` but the former has a better name so I removed the latter.

Fixes #89686

Note that the torch.cuda.is_available() check is redundant as torch.cuda.device_count() will return a value of at most zero and the former might already use the latter, i.e. is_available == device_count() > 0

Reopened from #89750 after rebase
Fix access to unitialized memory in VSX vector functions for quantized values
Labels: module: cpu, triaged, open source, ciflow/trunk, release notes: quantization, topic: bug fixes

Similar to #89833 those function may access uninitialized memory leading to undefined behavior/results.
Initialize with zeros as done before.


cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10
[dynamo][symbolic shapes] Long compilation time for KJT helper function
Labels: triaged, oncall: pt2, module: dynamic shapes

### 🐛 Describe the bug

Code - https://gist.github.com/anijain2305/94f59df4e5267fefa046b8cb718be2cd

Input has a list of integers, which changes from run to run, and this list goes to torch.split, triggering a large number of symbolic shape guards. I think Dynamo should skip compiling this type of function. Looking for ideas on how to proceed on this.

@voznesenskym @ezyang 

### Error logs

_No response_

### Minified repro

_No response_

### Versions

NA

cc @ezyang @msaroufim @wconstab @bdhirsh
Revert "[inductor] let codegen not rely on node order (#107320)"
Labels: module: inductor, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #109466

This reverts commit 556bfe7cb547f8073c86982e186c780ba5a53a8a.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
Revert "[inductor] Fix inputs with existing offsets (#108168)"
Labels: module: inductor, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #109465

This reverts commit 2c87ef3dbfe1e5bd9ee4b6a9e4aa5030dc714284.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
[Decomposition] hann_window.periodic
Labels: fb-exported, ciflow/inductor

Summary: Add decomposition for hann_window.periodic and include it in core_aten_decompositions

Test Plan:
Phabricator + OSS Tests

```
EXPECTTEST_ACCEPT=1 python3 test/test_decomp.py -k test_quick_hann_window
```

Reviewed By: SS-JIA

Differential Revision: D48939996
[dynamo]Scuba log some debug info about list of integer inputs
Labels: module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #109452
* #109410
* #109411


Some helper functions for jagged tensor have list of integers, which upset dynamic shapes as their content change from run to run. Logging some information to write heuristics.


cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
RNN Documentation is Confusing / Wrong
Labels: module: docs, module: nn, module: rnn, triaged, actionable

### 📚 The doc issue

The docs on https://pytorch.org/docs/stable/generated/torch.nn.RNN.html say:

> For each element in the input sequence, each layer computes the following function:
> 
> h(t) = tanh(x(t) * W_ih^T + b_ih + h(t-1) * W_hh^T + b_hh)
> 
> Where:
> 
> h(t) is the hidden state at time t.
> x(t) is the input at time t.
> h(t-1) is the hidden state of _the previous layer_ at time t-1 or the initial hidden state at time 0.

I'd like to point out the ambiguity surrounding the term "the previous layer" in the context above. The term "layer" here appears to be inconsistent with the definition used in the "num_layers" argument.

For an RNN with `num_layers=1`, the phrase "hidden state of the previous layer at time t-1" could be misinterpreted. Does this mean that we always refer to the initial hidden state since there's no preceding layer? This would imply the RNN isn't recurrent.

Upon testing a `num_layers=1` RNN, I observed that changing the input at t=1 did indeed influence the output at t=10, confirming the network's recurrent nature. Thus, I suspect the correct phrasing should be that x(t) is taken "from the previous layer".

One last thing: It would be useful to have a bit of information on how the RNN is implemented.
Is there a trick to evaluate it over a whole sequence as a batch operation?
Or is there a python/C loop under the hood, making `L` matrix multiplications sequentially for each layer?

### Suggest a potential alternative/fix


> h(t) is the hidden state at time t.
> x(t) is the input at time t _or output from the previous layer_.
> h(t-1) is the hidden state of _the same layer_ at time t-1 or the initial hidden state at time 0.

cc @svekars @carljparker @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki @zou3519
[FSDP] supports QLora finetuning
Labels: feature, triaged, module: fsdp

### 🚀 The feature, motivation and pitch

Currently FSDP is rejecting tensor parameters with dtype unit8. is_floating_point() only allows one of the (torch.float64, torch.float32, torch.float16, and torch.bfloat16)  
```
torch/distributed/fsdp/flat_param.py:720
>>> if dtype is None and not tensor.is_floating_point():
>>>    raise ValueError("Cannot flatten integer dtype tensors")
```
Since in Qlora setup, the base model is loaded as unit8 dtype. This makes qlora + fsdp imcompatible. 

### Alternatives

I have considered simply removed this dtype check but concerned about unknown impact. 

### Additional context

_No response_

cc @zhaojuanmao @mrshenli @rohan-varma @awgu @fegin @penguinwu
Can dtensor flexibly modify the layout via devicemesh?
Labels: oncall: distributed, module: dtensor

### 🚀 The feature, motivation and pitch

Hi,torch team:
Large tensor can not be stored in a single device, usually need to global tensor chunking, if  dtensor can be flexible to modify the layout according to devicemesh or torch.chunk  is best!
<img width="1016" alt="image" src="https://github.com/pytorch/pytorch/assets/48181529/17401024-cab9-4e63-aae4-e1abcfb28707">


### Alternatives

```python
@spawn_threads_and_init_comms
def shard_big_tensor(world_size):
    mesh = DeviceMesh("cpu", [[0,1],[2,3]])
    mesh1 = DeviceMesh("cpu", [0,1,2,3])
    big_tensor = torch.range(0,15).reshape(4,4)
    dtensor = distribute_tensor(big_tensor, mesh, [Shard(0),Shard(1)])
    print(f"on rank: {dist.get_rank()}",dtensor.to_local())
    dtensor1 = distribute_tensor(dtensor, mesh1, [Shard(0)])
    print(f"on rank: {dist.get_rank()}",dtensor1.to_local())

### Additional context

_No response_

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu
[Dynamo] Match closures by code ID
Labels: triaged, open source, Merged, Reverted, ciflow/trunk, topic: not user facing, module: dynamo, ciflow/inductor

Closes https://github.com/pytorch/pytorch/issues/107866

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Use weakref in fast tracebacks
Labels: triaged, open source, topic: not user facing, module: dynamo

I think this fixes https://github.com/pytorch/pytorch/issues/107469#issuecomment-1713031756

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Supporting Block_Ptrs in inductor code gen
Labels: triaged, oncall: pt2, module: inductor

### Summary
More curiosity and a tracking issue for me or someone else to pick up.
Triton has introduced Block_pointer: https://triton-lang.org/main/getting-started/tutorials/08-experimental-block-pointer.html

Personally I think these are easier to work with/reason about than raw strides. I could be missing something fundamental to inductors loop level IR but would be interesting to see what would be needed to support usage in this for example:
https://github.com/pytorch/pytorch/blob/main/torch/_inductor/kernel/mm.py#L31-L91 

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
Move eval_frame global variables into module state
Labels: triaged, open source, module: dynamo

Partially addresses https://github.com/pytorch/pytorch/issues/108942

Updated PR from pytorch repo instead of my fork. This was previously https://github.com/pytorch/pytorch/pull/108943

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
[export] Support tracing constant attribute mutations
Labels: module: dynamo, ciflow/inductor

This adds support for tracing (specifically when exporting) code that contains mutations on constant (int, str, bool) attributes. Basically, I just set the object's attribute to be the new constant. Since they're constants, they shouldn't result in any operations appearing in the graph. I'm unsure if this is the proper way to add support -- any suggestion is greatly appreciated!

Some datapoints on why we need this fix: 

* 2 of the HF models (GPT2ForSequenceClassification, LayoutLMForSequenceClassification) run into a tracing failure when they're setting some config to be a string value ([example code](https://github.com/huggingface/transformers/blob/0a55d9f7376f72ad3ff296d4249840021b03bcc4/src/transformers/models/longformer/modeling_longformer.py#L1953-L1972)).
* An internal model contains code where user wants to print to stdout based on a boolean attribute self._print_once, and they tend to do self._print_once = False a lot in forward functions. This also fails when tracing.


cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Revert D49284640: Multisect successfully blamed "D49284640: [inductor][Optimus]Improve logging for group batch fusion" for test or build failures
Labels: fb-exported, topic: not user facing, module: inductor, ciflow/inductor

Summary:
This diff is reverting D49284640
D49284640: [inductor][Optimus]Improve logging for group batch fusion by jackiexu1992 has been identified to be causing the following test or build failures:

Tests affected:
- [aps_models/examples/dlrm/tests:deterministic_ne_test - test_determinitic_ne (aps_models.examples.dlrm.tests.deterministic_ne_test.TestDLRMDeterministicNE)](https://www.internalfb.com/intern/test/281475072657704/)

Here's the Multisect link:
https://www.internalfb.com/multisect/3056925
Here are the tasks that are relevant to this breakage:

We're generating a revert to back out the changes in this diff, please note the backout may land if someone accepts it.

If you believe this diff has been generated in error you may Commandeer and Abandon it.

Test Plan: NA

Reviewed By: mengluy

Differential Revision: D49338482




cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
Interleaved isend and irecv causes hang
Labels: oncall: distributed, triaged

There seems to be a bug with `isend()` and `irecv()` which prevents asynchronicity when the calls are interleaved. In this case we are calling

rank 0: sync send() [completed] -> Async recv() -> Async send() [completed] -> Async recv()
rank 1: sync recv() [completed] -> Async recv() [completed] -> Async send() -> Async send()

Minimal example

```python
import torch
import torch.distributed as dist
import torch.multiprocessing as mp
import os
import datetime

def setup_dist(rank, world_size):
    os.environ['MASTER_ADDR'] = 'localhost'
    os.environ['MASTER_PORT'] = '12355'
    torch.cuda.set_device(rank)
    dist.init_process_group("nccl", rank=rank, world_size=world_size, timeout=datetime.timedelta(seconds=10))

def main(rank, world_size):
    setup_dist(rank, world_size)
    fwd_handles = []
    bwd_handles = []
    if rank == 0:
        dist.isend(torch.randn([1]).to("cuda"), dst=1).wait()
        bwd_handles.append(dist.irecv(torch.randn([1]).to("cuda"), src=1))
    elif rank == 1:
        dist.irecv(torch.randn([1]).to("cuda"), src=0).wait()

    if rank == 0:
        fwd_handles.append(dist.isend(torch.randn([1]).to("cuda"), dst=1))
        bwd_handles.append(dist.irecv(torch.randn([1]).to("cuda"), src=1))
    elif rank == 1:
        fwd_handles.append(dist.irecv(torch.randn([1]).to("cuda"), src=0))
        bwd_handles.append(dist.isend(torch.randn([1]).to("cuda"), dst=0))
        bwd_handles.append(dist.isend(torch.randn([1]).to("cuda"), dst=0))

    print(f"Rank {rank} waiting for {fwd_handles=}, {bwd_handles=}")
    for handle in fwd_handles:
        handle.wait()
    print(f"Rank {rank} done waiting for fwd_handles, now running {bwd_handles=}")
    for handle in bwd_handles:
        handle.wait()

if __name__ == "__main__":
    n_gpus = 2
    world_size = n_gpus
    mp.spawn(main, args=(world_size,), nprocs=world_size, join=True)
```

output

```
Rank 1 waiting for fwd_handles=[<torch.distributed.distributed_c10d.Work object at 0x7f672134b9b0>], bwd_handles=[<torch.distributed.distributed_c10d.Work object at 0x7f6720f0d330>, <torch.distributed.distributed_c10d.Work object at 0x7f67210ea8b0>]
Rank 1 done waiting for fwd_handles, now running bwd_handles=[<torch.distributed.distributed_c10d.Work object at 0x7f6720f0d330>, <torch.distributed.distributed_c10d.Work object at 0x7f67210ea8b0>]
Rank 0 waiting for fwd_handles=[<torch.distributed.distributed_c10d.Work object at 0x7faec809e5b0>], bwd_handles=[<torch.distributed.distributed_c10d.Work object at 0x7faec809d2b0>, <torch.distributed.distributed_c10d.Work object at 0x7faec8036e70>]
Rank 0 done waiting for fwd_handles, now running bwd_handles=[<torch.distributed.distributed_c10d.Work object at 0x7faec809d2b0>, <torch.distributed.distributed_c10d.Work object at 0x7faec8036e70>]
```


cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @kwen2501 @awgu @penguinwu
[FSDP] Implement additional check for turn on 2D TP + FSDP extension
Labels: triaged, module: fsdp

### 🚀 The feature, motivation and pitch

Currently, since we only support TP + FSDP, we automatically turn on the extension, when we detect FSDP's DeviceMesh has a parent mesh. This check is currently sufficient. 

However, additional check is required once we support other types of composition, for example, FSDP + PiPPy. Creating an issue to keep track of this.

### Alternatives

_No response_

### Additional context

_No response_

cc @zhaojuanmao @mrshenli @rohan-varma @awgu @fegin @penguinwu
cuda rng state for 2.0.1 cannot be used for 2.1.0
Labels: module: cuda, triaged, module: random

### 🐛 Describe the bug

Running `torch.cuda.random.get_rng_state().size()` in 2.0.1 and 2.1.0 (nightly) gives different results (816 vs 16). This breaks backwards compatibility of checkpoint loading/saving between the two versions because [rng state can't be set](https://github.com/pytorch/pytorch/blob/v2.0.1/torch/cuda/random.py#L43) when the expected tensor size is different.

### Versions

pytorch 2.0.1 and 2.1.0 (8/27 nightly and rc4)

cc @ptrblck @pbelevich
DISABLED test_ddp_activation_checkpointing (__main__.TestMultiProc)
Labels: module: rocm, triaged, skipped

Platforms: rocm

First known bad: https://hud.pytorch.org/pytorch/pytorch/commit/34ddf08f2737025fb1447070fae3087889b2e8bb 

This test was disabled because it is failing on main branch ([recent examples](http://torch-ci.com/failure/distributed%2Ftest_dynamo_distributed.py%3A%3ATestMultiProc%3A%3Atest_ddp_activation_checkpointing)).

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang
DISABLED test_compute_local_shape_and_global_offset_1D (__main__.UtilTest)
Labels: oncall: distributed, module: flaky-tests, skipped

Platforms: linux

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_compute_local_shape_and_global_offset_1D&suite=UtilTest) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/16828471140).

Over the past 3 hours, it has been determined flaky in 2 workflow(s) with 2 failures and 2 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_compute_local_shape_and_global_offset_1D`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `distributed/_tensor/test_utils.py`

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu
DISABLED test_nondeterministic_alert_median_cuda_float64 (__main__.TestTorchDeviceTypeCUDA)
Labels: module: tests, triaged, module: flaky-tests, skipped, oncall: pt2

Platforms: inductor

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_nondeterministic_alert_median_cuda_float64&suite=TestTorchDeviceTypeCUDA) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/16813760825).

Over the past 3 hours, it has been determined flaky in 5 workflow(s) with 5 failures and 5 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_nondeterministic_alert_median_cuda_float64`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_torch.py`

cc @mruberry @ezyang @msaroufim @wconstab @bdhirsh @anijain2305
DISABLED test_nondeterministic_alert_kthvalue_cuda_float64 (__main__.TestTorchDeviceTypeCUDA)
Labels: module: tests, triaged, module: flaky-tests, skipped, oncall: pt2

Platforms: inductor

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_nondeterministic_alert_kthvalue_cuda_float64&suite=TestTorchDeviceTypeCUDA) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/16805573167).

Over the past 3 hours, it has been determined flaky in 12 workflow(s) with 12 failures and 12 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_nondeterministic_alert_kthvalue_cuda_float64`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_torch.py`

cc @mruberry @ezyang @msaroufim @wconstab @bdhirsh @anijain2305