gopikrsmscs/torch-issues huggingface dataset:
[dynamo] `no_grad`, `enable_grad` - `_NoParamDecoratorContextManager` are not handled correctly
Labels: oncall: pt2

### 🐛 Describe the bug

Root cause of https://github.com/pytorch/pytorch/issues/109138

Reproducer:
```python
import torch

def cool_name(x):
    return x.sin()

def fn(x):
    return torch.no_grad(cool_name)(x)

x = torch.zeros(10)
result = fn(x)
print(result)
result = torch.compile(fn, backend="eager", fullgraph=True)(x)
print(result)
```

Also fails:
```python
def fn(x):
    @torch.no_grad
    def cool_name(x):
        return x.sin()

    return cool_name(x)

x = torch.zeros(10)
result = fn(x)
print(result)
result = torch.compile(fn, backend="eager", fullgraph=True)(x)
print(result)
```

Does not fail: no_grad is instantiated outside of compile region
```python
@torch.no_grad
def cool_name(x):
    return x.sin()

def fn(x):
    return cool_name(x)

x = torch.zeros(10)
result = fn(x)
print(result)
result = torch.compile(fn, backend="eager", fullgraph=True)(x)
print(result)
```

### Solution
Handle it properly when it is called as a function. It might need to instantiate the gradmodevariable whenever the function is called. 

To do so, one can put a `context_var_hook` which sets up the context var when the function is called.

### Versions

main

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519
Functorch FCD breaks with tensor subclasses
Labels: triaged, module: functorch, module: first class dims

### 🐛 Describe the bug

Classes that inherits from torch.Tensor do not work with firct class dimensions when `__getitem__` is overwritten:

```python
import torch
from torch import Tensor
from functorch import dim

class MyTensor(Tensor):
    def __getitem__(self, item):
        return super().__getitem__(item)

t = Tensor([[1, 2],[3, 4]])
d0 = dim.dims(1)
t[d0] # works

t = MyTensor([[1, 2],[3, 4]])
d0 = dim.dims(1)
t[d0] # breaks

```

which gives
```
ValueError                                Traceback (most recent call last)
Cell In[3], line 1
----> 1 t[d0]

Cell In[2], line 7, in MyTensor.__getitem__(self, item)
      6 def __getitem__(self, item):
----> 7     return super().__getitem__(item)

ValueError: dimension d0 is unbound
```

I guess that since `t[d0]` in the regular `Tensor` case is not a `Tensor` anymore but a `functorch.dim.Tensor`, the expected behaviour is ill-defined. Maybe a better error message in this case would help to let users know what is going on?

cc @ezyang @msaroufim @albanD @zou3519 @Chillee @samdow @kshitij12345 @janeyx99 @zdevito



### Versions

Recent nightly built ('2.2.0.dev20231011')
Insufficient hasattr guards on user defined objects
Labels: oncall: pt2

### 🐛 Describe the bug

```
import torch
import torch._dynamo

@torch.compile(backend="eager", fullgraph=True)
def f(x, y):
    return x + y.a

class A:
    a = 2

print(f(torch.zeros(2), A()))

del A.a

print(f(torch.zeros(2), A()))
```

produces

```
ERROR RUNNING GUARDS f /data/users/ezyang/a/pytorch/a.py:4
lambda L, **___kwargs_ignored:
  ___guarded_code.valid and
  ___check_global_state() and
  hasattr(L['x'], '_dynamo_dynamic_indices') == False and
  ___check_type_id(L['y'], 102721216) and
  ___check_type_id(L['y'].a, 7640416) and
  L['y'].a == 2 and
  utils_device.CURRENT_DEVICE == None and
  ___skip_backend_check() or ___current_backend() == ___lookup_backend(140059152699776) and
  ___check_tensors(L['x'], tensor_check_names=tensor_check_names)
Traceback (most recent call last):
  File "/data/users/ezyang/a/pytorch/a.py", line 15, in <module>
    print(f(torch.zeros(2), A()))
  File "/data/users/ezyang/a/pytorch/torch/_dynamo/eval_frame.py", line 401, in _fn
    return fn(*args, **kwargs)
  File "<string>", line 13, in guard
AttributeError: 'A' object has no attribute 'a'
```

### Versions

main

cc @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519
[pt2+profiler] attach aot_id to CompiledFunction
Labels: oncall: pt2

### 🚀 The feature, motivation and pitch

torch-compiled models will have profiles that contain `CompiledFunction` and `CompiledFunctionBackward` regions.

Meanwhile PT2 logs (e.g. TORCH_COMPILE_DEBUG=1) will also dump the graphs. But in graphs that have multiple CompiledFunctions, it's kind of hard to figure out which graph (in the logs) maps to a given CompiledFunction (in the profile).

We should figure out how to attach the aot_id to the CompiledFunction profiler event.

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519 who had this great idea :) 

### Alternatives

_No response_

### Additional context

Idea: autograd.Functions have handling in C++ to add the record_function event. That's where the CompiledFunction event comes from.

It might be possible to attach a static parameter or method on the CompiledFunction (like `_compiled_autograd_key()`) that is used for additional information attached to the C++ RecordFunction.

We should make sure this doesn't slow down CompiledFunction, especially in the no-profiling case. It would be great if we can collect the aot_id during CompiledFunction construction so that we don't have to do the python/C++ conversion on each iteration.
[ci] Save various json files from test infra into folder
Labels: ciflow/trunk, release notes: releng, suppress-bc-linter

We pull a lot of files from https://github.com/pytorch/test-infra/blob/generated-stats/stats and name them separately when we add them to the artifacts in the build, so stick them in a folder and just add that instead.

Slow test and disabled test jsons remain as they were since they are pulled during the test step and do not need to be included in the artifacts during build since they are not used for sharding.

Sanity checked that test times could be found for linux, mac, windows, and rocm.
DISABLED test_detach_cpu_float32 (__main__.TestNestedTensorDeviceTypeCPU)
Labels: triaged, module: flaky-tests, module: nestedtensor, skipped, oncall: pt2

Platforms: dynamo

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_detach_cpu_float32&suite=TestNestedTensorDeviceTypeCPU) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17831419866).

Over the past 3 hours, it has been determined flaky in 4 workflow(s) with 12 failures and 4 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_detach_cpu_float32`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_nestedtensor.py`

cc @cpuhrsch @jbschlosser @bhosmer @drisspg @soulitzer @ezyang @msaroufim @wconstab @bdhirsh @anijain2305
DISABLED test_meta_outplace_addmm_cpu_complex128 (__main__.TestMetaCPU)
Labels: triaged, module: flaky-tests, skipped, module: primTorch, oncall: pt2

Platforms: dynamo

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_meta_outplace_addmm_cpu_complex128&suite=TestMetaCPU) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17830649623).

Over the past 3 hours, it has been determined flaky in 8 workflow(s) with 24 failures and 8 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_meta_outplace_addmm_cpu_complex128`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_meta.py`

cc @ezyang @mruberry @Lezcano @peterbell10 @msaroufim @wconstab @bdhirsh @anijain2305
[re-land][inductor] Refactor and optimize allocation calls (#111117)
Labels: fb-exported, topic: not user facing, module: inductor, ciflow/inductor

Summary:
This is a re-land of https://github.com/pytorch/pytorch/pull/111117 with
updates to our internal tests included.

This splits out changes from
https://github.com/pytorch/pytorch/pull/102625 to make things easier to
review.

This diff creates a `make_allocation()` method that extracts the logic
from `make_buffer_allocation()` while allowing us to allocate non-buffer
objects. In particular, we will use this to allocate memory pools during
memory planning.

This diff also includes a small optimization -- if the desired
allocation is contiguous, then we emit a call to `empty()` instead of
`empty_strided()` with its superfluous stride argument.

Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/9ce0ae836d6801a39776897b9e891cd978b28aea

Differential Revision: D50429424



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler
[BE] Enable Ruff's Flake8 PYI036
Labels: module: cpu, triaged, open source, module: amp (automated mixed precision), release notes: distributed (fsdp)

Enable [bad-exit-annotation (PYI036)](https://docs.astral.sh/ruff/rules/bad-exit-annotation/#bad-exit-annotation-pyi036)

Link: #110950

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @mcarilli @ptrblck @leslie-fang-intel
XLA Tensor creation fails on functionalization inside dynamo.
Labels: triaged, module: xla, module: functionalization

### 🐛 Describe the bug

Minimal reproducible program:

```python
import torch
import torch_xla.core.xla_model as xm

device = xm.xla_device()

def foo():
    return torch.tensor([0.0], device=device)

compiled = torch.compile(backend="openxla")(foo)
tensor = compiled()
```

```python
  File "torch/_subclasses/functional_tensor.py", line 304, in __torch_dispatch__
    return return_and_correct_aliasing(func, args, kwargs, outs_wrapped)
  File "torch/utils/_python_dispatch.py", line 364, in return_and_correct_aliasing
    _correct_storage_aliasing(func, schema_info, args, (out,) if not isinstance(out, tuple) else out)
  File "torch/utils/_python_dispatch.py", line 251, in _correct_storage_aliasing
    alias_non_inplace_storage(args[arg_idx], outs[return_idx])
  File "torch/utils/_python_dispatch.py", line 234, in alias_non_inplace_storage
    torch.ops.aten.set_.source_Storage_storage_offset(ret, arg.untyped_storage(), ret.storage_offset(), ret.shape)
  File "torch/_ops.py", line 499, in __call__
    return self._op(*args, **kwargs or {})
torch._dynamo.exc.BackendCompilerFailed: backend='openxla' raised:
RuntimeError: Attempted to set the storage of a tensor on device "xla:0" to a storage on different device "lazy:0".  This is no longer allowed; the devices must match.

While executing %tensor : [num_users=1] = call_function[target=torch.tensor](args = ([0.0],), kwargs = {device: xla:0})
Original traceback:
  File "examples/bug-device.py", line 7, in foo
    return torch.tensor([0.0], device=device)
```

### Versions

PyTorch version: 2bf3ca1be759460cf9fbf011d96d3246001361e9 (Oct 4)
PyTorch/XLA version: c9a132484fb89bfdc9c602ada7bd8a3cec0db1aa (Oct 3)


cc @bdhirsh @ezyang
Dynamo runner: add FSDP handcrafted module wrapping policy
Labels: topic: not user facing, module: dynamo, ciflow/inductor

The default size based auto wrap policy may not be representative of actual usage of the models. We add support for a few handpicked models, and fallback to the size based policy.

sample command: 
`PYTHONPATH=~/benchmark/ python benchmarks/dynamo/torchbench.py -dcuda --training --backend=inductor --multiprocess --performance --only nanogpt --fsdp`

1.257x                                                           
1.256x                                                           
1.257x                                                           
1.252x                                                           
1.257x                                                                                                                            
1.262x                                
1.258x             
1.272x

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Add support to ExportedProgram as input to torch.onnx.dynamo_export
Labels: module: onnx, open source, onnx-triaged, release notes: onnx

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #111498
* __->__ #111497

Fixes #109889

This PR adds `torch.export.export` as another `FXGraphExtractor` implementation. `torch.onnx.dynamo_export` automatically uses this new FX tracer when a `torch.export.ExportedProgram` is specified as `model`

Implementation is back compatible, thus non `ExportedProgram` models are handled the exact same way as before
[ONNX][dynamo] Parameter to export flat graphs
Labels: module: onnx, triaged

### 🚀 The feature, motivation and pitch

I'm exporting a model with a single linear layer to ONNX. Each layer of the generated graph is an ONNX function with an underlying function body composed of other functions and operators. The feature I'm requesting is a configurable option to generate a flat graph with no ONNX functions. 

The motivation for this request is to enable optimizations like constant folding. With function nodes, important information is not passed down to the function body resulting in fewer optimizations than possible with a flat graph. Additionally, the TorchScript-based ONNX export does provide an argument to [export modules as functions](https://github.com/pytorch/pytorch/blob/main/torch/onnx/utils.py#L481). It would be beneficial for users if the TorchDynamo-based ONNX exporter had a similar feature. 


Code to reproduce:
```
import torch

class LinearModel(torch.nn.Module):
  def __init__(self):
      super().__init__()
      self.fc0 = torch.nn.Linear(5, 7)

  def forward(self, tensor_x: torch.Tensor):
      output = self.fc0(tensor_x)
      return output

def linearDataloader():
    yield torch.randn(3, 5).cuda()

# Get model and input
model = LinearModel()
data = next(linearDataloader())

# ONNX Export
export_output = torch.onnx.dynamo_export(
    model.eval().to('cuda'),
    data
)
export_output.save('linear_dynamo.onnx')
```


### Alternatives

_No response_

### Additional context

_No response_
[dynamo] support comparing LHS constant with tensor 
Labels: open source, ciflow/trunk, topic: not user facing, module: dynamo, ciflow/inductor

Fixes https://github.com/pytorch/pytorch/issues/108582

Depends on https://github.com/pytorch/pytorch/pull/111557 for fixing broken integration tests. (due to this PR unblocking an in-graph set membership)

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
BFloat16 datatype support in Quantization
Labels: oncall: quantization, triaged

### 🚀 The feature, motivation and pitch

I am working on quantization using Pytorch2 infrastructure and would like for Bfloat16 to be supported in quantization annotation. If you look in torch/ao/quantization/quantizer/quantizer.py; BFloat16 is currently not a supported dtype. 

### Alternatives

_No response_

### Additional context

_No response_

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel
When keep_inference_input_mutations=True is set, one dynamic shape test fails
Labels: triaged, oncall: pt2, module: aotdispatch

### 🐛 Describe the bug

In order to get aot_eager and inductor backends to be consistent, I submitted a PR to set `keep_inference_input_mutations=True` but `dynamo/test_dynamic_shapes.py::DynamicShapesAotAutogradFallbackTests::test_call_fn_with_non_const_inputs_aot_unsafe_dynamic_shapes` started failing with
```
"RuntimeError: a leaf Variable that requires grad is being used in an in-place operation."
```

Example failure: https://hud.pytorch.org/pr/111453

@bdhirsh noted that this is probably a bug with autograd

### Versions

github master

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305
BUG: fix np.typecodes under Dynamo
Labels: triaged, module: numpy, open source, module: dynamo, ciflow/inductor, release notes: dynamo

`numpy.typecodes` is a module-level Dict[Str, Str] which lists allowed dtypes by category. Since we only support a subset of numpy dtypes, `torch._numpy.typecodes` differs from `numpy.typecodes`.

Since this is a dict of primitive types, need to trick dynamo to stop it from simply inlining the numpy dict.

This is only a partial fix, it only works if there's something to compile. Otherwise, e.g. bare

```
def fn():
   return np.typecodes['AllInteger']
```

still generates a numpy dict.

Partially fixes item 2 of  https://github.com/pytorch/pytorch/issues/111370.


cc @mruberry @rgommers @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
[caffe2] avoid variable shadowing
Labels: fb-exported

Summary:
Some builds use -Wshadow and currently there is a compiler warning when building that file.

Code inspection shows that `torch::autograd::impl::get_view_autograd_meta` simply extracts information from the passed object, which is `const`. Therefore the returned views should be the same all the time, and we can fetch the view only once.

Test Plan:
CI

NOTE: please advise for a more comprehensive test plan.

Differential Revision: D50407625
[qnnpack] suppress empty translation unit warning
Labels: module: cpu, fb-exported, release notes: quantization

Summary: Spotted this while compiling on a Mac M1. The code in these files is gated behind #ifdef and requires SSE, so when building for ARM these files become empty.

Test Plan: CI

Differential Revision: D50407334




cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10
Rephrase sentence in "Why and when to use sparsity" for better understanding.
Labels: module: docs, triaged

### 📚 The doc issue

Current Sentence:

By default PyTorch stores [torch.Tensor](https://pytorch.org/docs/stable/tensors.html#torch.Tensor) stores elements contiguously physical memory.

### Suggest a potential alternative/fix

Should be (I feel it reads better this way):

By default PyTorch stores [torch.Tensor](https://pytorch.org/docs/stable/tensors.html#torch.Tensor) elements in physically contiguous memory.

cc @svekars @carljparker
yolov5_train
Labels: feature, module: cuda, triaged, module: determinism

### 🐛 Describe the bug

 File "D:\anaconda\envs\pytorch\lib\site-packages\torch\_tensor.py", line 487, in backward
    torch.autograd.backward(
  File "D:\anaconda\envs\pytorch\lib\site-packages\torch\autograd\__init__.py", line 197, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: adaptive_avg_pool2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True)'. You can turn off determinism just for this operation, or you can use the 'warn_only=True' option, if that's acceptable for your application. You can also file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation.


### Versions

1

cc @ptrblck @mruberry @kurtamohler
DISABLED test_meta_inplace_addmm_decomposed_cpu_complex128 (__main__.TestMetaCPU)
Labels: triaged, module: flaky-tests, skipped, module: primTorch, oncall: pt2

Platforms: dynamo

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_meta_inplace_addmm_decomposed_cpu_complex128&suite=TestMetaCPU) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17802491680).

Over the past 3 hours, it has been determined flaky in 2 workflow(s) with 6 failures and 2 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_meta_inplace_addmm_decomposed_cpu_complex128`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_meta.py`

cc @ezyang @mruberry @Lezcano @peterbell10 @msaroufim @wconstab @bdhirsh @anijain2305
[dynamo] allow DeviceMesh variable desugar ProcessGroup
Labels: ciflow/trunk, module: dynamo, ciflow/inductor, release notes: distributed (dtensor)

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111461

As titled, when calling get_dim_groups, DeviceMeshVariable should be
de-sugared to ProcessGroup Variable in dynamo

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng

Differential Revision: [D50400451](https://our.internmc.facebook.com/intern/diff/D50400451)
[not for review] testing memory planning
Labels: module: inductor, module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111459
* #111402
* #111587



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler
Turn keep_inference_input_mutations on in aot_eager
Labels: ciflow/trunk, module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111453



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
[FX Quant] operator.matmul (@ operator ) is not converted to torch.ops.quantized.matmul
Labels: oncall: quantization, triaged

### 🐛 Describe the bug

Maybe, this line? only torch.matmul is included here.
 
https://github.com/pytorch/pytorch/blob/main/torch/ao/quantization/fx/_lower_to_native_backend.py#L359




### Versions

main branch

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel
DISABLED test_compile_dtensor_redistribute_backward (__main__.TestDTensorCompileE2E)
Labels: oncall: distributed, triaged, skipped

Platforms: linux

This test was disabled because it is failing on main branch ([recent examples](http://torch-ci.com/failure/distributed%2F_tensor%2Ftest_dtensor_compile.py%3A%3ATestDTensorCompileE2E%3A%3Atest_compile_dtensor_redistribute_backward)).

This starts to fail on periodic multigpu job https://hud.pytorch.org/pytorch/pytorch/commit/6f06832219e1059ea794c9e1a57a81ed9448a7a1.  The root cause is unclear

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu
Add support for sym_ite
Labels: ciflow/trunk, release notes: fx, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111440

This PR supports sym_ite. This is useful for converting SymBool to SymInt in e.g. #109916. Internally, it uses sympy.Piecewise. We cannot use sympy.ITE because it expects the arguments and output all to be boolean type but we want return SymInt type when converting a SymBool to SymInt. So we use sympy.Piecewise to denote the symbolic relationship. 

Note that this pr uses the range analysis for sympy.Piecewise implemented in https://github.com/pytorch/pytorch/blob/main/torch/utils/_sympy/value_ranges.py. 

Test Plan:
See added test.
Avoid c++ exception and stack trace
Labels: fb-exported, release notes: jit

Summary:
When raising an exception here this causes pybind11's dispatcher to kick in, which causes aiplatform's logic to kick in (aiplatform::error_reporting::util::printAddressesWithBestEffortLocationInfo), which ultimately uses `folly::symbolizer::Symbolizer::symbolize` for building up the stack trace.  In 3.8 this uses about 3.62% of the CPU time per pyperf (https://fburl.com/scuba/pyperf_experimental/on_demand/oi554uvy).  In Cinder 3.8 for some reason this is worse - using 5.94% of the CPU.

This exception is happening when doing a hasattr() on `prims` for things like `bitwise_left_shift` which don't exist: https://www.internalfb.com/code/fbsource/[2d695f650d00]/fbcode/caffe2/torch/_inductor/lowering.py?lines=590

That exception is ultimately going to be swallowed anyway, and the stack trace has no meaningful value.  Furthermore because this is kind of an expected outcome in the code versus some random C++ exception the stack trace is less valuable as well.

This changes this to return a (None, None) on the failure case instead of returning a valid op/overload list, avoiding the exception, and reclaiming the 3.62%-5.94% of time.

Test Plan: Existing CI and perf run: https://fburl.com/scuba/pyperf_experimental/on_demand/oi554uvy

Differential Revision: D50018789
[dynamo] Inlining Translator will compile partial subgraphs
Labels: open source, module: dynamo, ciflow/inductor, release notes: dynamo

Reduce number of backend compiler invocations and number of compiled artifacts

Fixes: https://github.com/pytorch/pytorch/issues/111003

TODO:
1. Handle backedges in nested graph breaks properly (now we just killswitch to eager as long as one of the outer functions has a backedge. Might be a little too strict - let's try to accomodate backedges in outer functions if possible).
2. May not handle certain edge-cases properly, need to scrutinize.
- [ ] Test for bad interaction with generator - e.g. generator calling into function with graph breaks.

cc @voznesenskym @jansel @yanboliang
[Inductor] Support user defined triton kernels in inductor
Labels: ciflow/trunk, module: inductor, module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #111627
* __->__ #111434



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler
torch.compile x autograd.Function: Make the backward strict mode less srict
Labels: module: autograd, oncall: pt2, module: dynamo

## Issue description

Today, when Dynamo sees an autograd.Function, it traces both the forward() and backward(). Once it has proved safety for both of these APIs, we put a call to the autograd.Function.apply into the generated FX graph.

We trace the backward() underneath a "strict mode". That is, we ban accesses to Tensor.stride() (and a number of other APIs). This is because we don't actually know many properties of the gradients at the time of the forward pass.

We should lift some of these restrictions. For example:
- we should only ban accesses to .stride() on the gradients (I think we ban this on all tensors)
- we should only ban control flow on the .stride(). Things like assertions seem fine.


cc @ezyang @albanD @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7 @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
[RFC][inductor] FX graph cache: Add support for symbolic shapes
Labels: release notes: fx, topic: not user facing, module: inductor, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111421

Summary: Add support for caching graphs that have tensor args with symbolic shapes. The high-level appraoch is to serialize guards with the on-disk cached object and validating those guards pass before serving a cached object.

Test Plan: New unit tests

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler
nonnull error
Labels: triage review, needs reproduction, module: build

### 🐛 Describe the bug

It is impossible to identify the error, it is only known that it is a very serious error that prevents compilation.

```
[ 95%] Building CXX object test_api/CMakeFiles/test_api.dir/dataloader.cpp.o
En el fichero incluido desde /usr/include/c++/12.1.0/memory:63,
                 desde $DIRTORCH/third_party/googletest/googletest/include/gtest/gtest.h:57,
                 desde $DIRTORCH/test/cpp/api/dataloader.cpp:1:
En la función miembro static ‘static _Tp* std::__copy_move<_IsMove, true, std::random_access_iterator_tag>::__copy_m(const _Tp*, const _Tp*, _Tp*) [with _Tp = unsigned int; bool _IsMove = false]’,
    incluido en línea de ‘_OI std::__copy_move_a2(_II, _II, _OI) [with bool _IsMove = false; _II = const unsigned int*; _OI = unsigned int*]’ en /usr/include/c++/12.1.0/bits/stl_algobase.h:495:30,
    incluido en línea de ‘_OI std::__copy_move_a1(_II, _II, _OI) [with bool _IsMove = false; _II = const unsigned int*; _OI = unsigned int*]’ en /usr/include/c++/12.1.0/bits/stl_algobase.h:522:42,
    incluido en línea de ‘_OI std::__copy_move_a(_II, _II, _OI) [with bool _IsMove = false; _II = __gnu_cxx::__normal_iterator<const unsigned int*, vector<unsigned int> >; _OI = __gnu_cxx::__normal_iterator<unsigned int*, vector<unsigned int> >]’ en /usr/include/c++/12.1.0/bits/stl_algobase.h:529:31,
    incluido en línea de ‘_OI std::copy(_II, _II, _OI) [with _II = __gnu_cxx::__normal_iterator<const unsigned int*, vector<unsigned int> >; _OI = __gnu_cxx::__normal_iterator<unsigned int*, vector<unsigned int> >]’ en /usr/include/c++/12.1.0/bits/stl_algobase.h:620:7,
    incluido en línea de ‘std::vector<_Tp, _Alloc>& std::vector<_Tp, _Alloc>::operator=(const std::vector<_Tp, _Alloc>&) [with _Tp = unsigned int; _Alloc = std::allocator<unsigned int>]’ en /usr/include/c++/12.1.0/bits/vector.tcc:244:21:
/usr/include/c++/12.1.0/bits/stl_algobase.h:431:30: error: argument 1 null where non-null expected [-Werror=nonnull]
  431 |             __builtin_memmove(__result, __first, sizeof(_Tp) * _Num);
      |             ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/12.1.0/bits/stl_algobase.h:431:30: nota: en una llamada a la función interna ‘void* __builtin_memmove(void*, const void*, unsigned int)’
cc1plus: algunos avisos se tratan como errores
make[2]: *** [test_api/CMakeFiles/test_api.dir/build.make:118: test_api/CMakeFiles/test_api.dir/dataloader.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:8822: test_api/CMakeFiles/test_api.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
```


### Versions

linux

cc @malfet @seemethere
AOTAutograd generates wrong strides for view+inplace op
Labels: ezyang's list, oncall: pt2, module: aotdispatch

### 🐛 Describe the bug

There is strides mismatch between eager and custom compiler backend like Habana HPU which produce contiguous strides. Here is an example to demonstrate the issue.

```python
import torch

def transpose_inplace_mul(x):
    y = x.t()
    y.mul_(2)
    return y

x = torch.arange(6, dtype=torch.float32).reshape([2, 3])
ref_res = transpose_inplace_mul(x)
print('ref_cpu_x.stride: ', x.stride())
print('ref_cpu_x: ', x.cpu())
print('ref_cpu_res.stride: ', ref_res.stride())
print('ref_cpu_res: ', ref_res.cpu())

x = torch.arange(6, dtype=torch.float32).reshape([2, 3]).to(device='hpu')
compiled_fn = torch.compile(transpose_inplace_mul, backend='aot_hpu_training_backend')
res = compiled_fn(x)
print('hpu_x.strdides:', x.stride())
print('hpu_x: ', x.cpu())
print('hpu_res.strdides:', res.stride())
print('hpu_res: ', res.cpu())
```

Outputs is like below:
```
ref_cpu_x.stride:  (3, 1)
ref_cpu_x:  tensor([[ 0.,  2.,  4.],
        [ 6.,  8., 10.]])
ref_cpu_res.stride:  (1, 3)
ref_cpu_res:  tensor([[ 0.,  6.],
        [ 2.,  8.],
        [ 4., 10.]])

hpu_x.strdides: (3, 1)
hpu_x:  tensor([[ 0.,  2.,  4.],
        [ 6.,  8., 10.]])
hpu_res.strdides: (2, 1)
hpu_res:  tensor([[ 0.,  2.],
        [ 4.,  6.],
        [ 8., 10.]])
```

We can see that **hpu_res.strides** is different from the referenced **cpu_res.strides**. And hence output results is different with CPU reference result. For Habana HPU, currently it produces contiguous strides, hence there is a handling (https://github.com/pytorch/pytorch/blob/main/torch/_refs/__init__.py#L466) for non-contiguous output.
We expect AOTAutograd can also handle this. Previously in another issue thread, stride-agnostic mode was mentioned.

The generated functional FX graph as below:
```
class <lambda>(torch.nn.Module):
    def forward(self, arg0_1: f32[2, 3]):
        
        t: f32[3, 2] = torch.ops.aten.t.default(arg0_1);  arg0_1 = None
        
        mul: f32[3, 2] = torch.ops.aten.mul.Tensor(t, 2);  t = None
        t_1: f32[2, 3] = torch.ops.aten.t.default(mul);  mul = None
        t_2: f32[3, 2] = torch.ops.aten.t.default(t_1)
        return (t_1, t_2)
```

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519

### Error logs

_No response_

### Minified repro

_No response_

### Versions

Collecting environment information...
PyTorch version: 2.1.0a0+gitf8b6084
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.6 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
CMake version: version 3.27.4
Libc version: glibc-2.31

Python version: 3.8.10 (default, May 26 2023, 14:05:08)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.4.0-163-generic-x86_64-with-glibc2.29
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
The results of masked.log_softmax on MPS are inconsistent with those on CPU
Labels: high priority, triage review, module: correctness (silent), module: mps

### 🐛 Describe the bug

The output of `masked.log_softmax` on CPU with inputs of cpu_args:  [tensor(-8.4844, dtype=torch.float16, requires_grad=True), 0] and cpu_kwargs:  {'mask': tensor(False)},  i.e., the masked element, is nan. While the result on MPS is 0. https://github.com/pytorch/pytorch/actions/runs/6542102776/job/17764726988.
`masked.softmax` and `masked.softmin` should have the same issue.

### Versions

PyTorch: main branch.

cc @ezyang @gchanan @zou3519 @kadeng @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
[dynamo] Eagerly install guards
Labels: topic: not user facing, module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #111726
* #111725
* __->__ #111415
* #111614
* #111717
* #111306



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
[inductor] Adding a way to force fusion of int_mm with mul
Labels: module: inductor, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #111631
* __->__ #111413

Summary: When doing quantization int_mm -> mul or int_mm -> mul ->
to(dtype) is an extremely common op pattern which is currently not
handled well by inductor. Ideally, since the output of
int_mm has dtype int32 we'd prefer to only realize a smaller dtype like
bf16 or float16. Currently inductor doesn't have a way to force this, in
many cases the mul gets fused with a bunch of subsequent pointwise and reduction
ops from the dequant  and following ops creating an increase in memory overhead and a general
slowdown compared to the fused version.

as an external benchmark, for SAM this seems to improve our e2e image encoder times by 3-5% depending on
batchsize and reduces memory usage by 20%

Test Plan: python test/inductor/test_pattern_matcher.py -k
"int_mm_mul"

Reviewers:

Subscribers:

Tasks:

Tags:
AOTAutograd: avoid intermediate_base logic when all aliased outputs came from a multi_output_view
Labels: ciflow/inductor, release notes: AO frontend

Partially addresses https://github.com/pytorch/pytorch/issues/111081

This fixes the majority of the slowness from https://fb.workplace.com/groups/1405155842844877/permalink/7491314274228973/. In particular, the type of example that suffers the most perf-wise in AOTAutograd looks like this:
```
@torch.compile
def f(x):
    intermediate = x.mul(2)
    outs = intermediate.unbind(0)
    return *outs

x = torch.randn(50, 50, requires_grad=True)
outs = f(x)
sum(outs).sum().backward()
```

There are 50 output tensors in the above function, that all alias each other. AOTAutograd will dutifully exercise its intermediate base [logic](https://github.com/pytorch/pytorch/blob/main/torch/_functorch/aot_autograd.py#L294), and try to regenerate the aliases outside of the compiled `autograd.Function` at runtime, to ensure that the autograd engine is aware of the aliasing.

In this case, this will result in **50 AsStridedBackward nodes in the backward**, because we will fall back to using as_strided to generate each of those 50 outputs. The current PR as is (somewhat unsafely) ensures that the backward graph consists of a single `UnbindBackward`, or a call to `aten.cat()`.

I left a long comment in the code describing the situation, but the core idea is that **autograd does not let you mutate grad_fn of tensor aliases that come from multi-output views**. So if we have `k` outputs that alias each other, but `k-1` of them are aliases that came from multi-output views, then in eager mode, it would not be possible to mutate one of the aliases in a way that would change the grad_fn of any of the other aliases, without causing an error in the backward. So the claim I'm making is that if we hide this aliasing from the autograd engine, then it is impossible for the user to perform any mutations that would cause autograd metadata to diverge between torch.compile and eager in a way that isn't an error in eager mode.

To be fair, I think that taking the approach outlined in https://docs.google.com/document/d/1DlfFq8TKbuAn2zyJxLfoW-X1qkkm5PLdHFtySo03QAk/edit would also help us avoid the as_strided calls in this particularly egregious case, **and** keep the autograd error messages. This relies on both pre-dispatch functionalization being fully hardened **and** adding some pretty invasive changes to AOTAutograd though, and is probably at least several months out.

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111411
Use of -Wl,--as-needed in cmake config files can leak into third-party users' code and modify their own private libraries
Labels: module: build, triaged, topic: build

https://github.com/pytorch/pytorch/blob/a0632389b78bc52d66cfc31c72ee06df8d8dc986/cmake/public/utils.cmake#L264-L268

Use of `-Wl,--no-as-needed` paired with `-Wl,--as-needed` will corrupt the initial linker state upon returning. This appears to be the cause of the cmake config files providing the same linker args to other projects that depend on pytorch.

It will prevent other projects which also use Wl,--no-as-needed for their own code, from working with pytorch.

The linker has a dedicated flag for this which is much safer:
```
-Wl,--push-state,--as-needed <your args here> -Wl,--pop-state
```

It should be used instead. Reference: https://github.com/mesonbuild/meson/pull/12299

cc @malfet @seemethere
adding way to force int_mm fusion
Labels: module: inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111405
* #111125

Summary: current issue with minified example, i think i fixed the Rocm
issue though.

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler
[HigherOrderOp] Move map_impl to torch.ops.higher_order
Labels: fb-exported, ciflow/trunk, module: export, release notes: export

The purpose of this pr is as titled. Because of some misusage of ghstack, ghimport, and export to github from internal, the stack of https://github.com/pytorch/pytorch/pull/111092 is a mess. I'll try to land them one by one. This is a replacement for #111092 and #111400.


cc @avikchaudhuri @gmagogsfm @zhxchen17 @tugsbayasgalan
[inductor] Defer memory operation lowering to wrapper
Labels: topic: not user facing, module: inductor, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #111459
* __->__ #111402
* #111587

Right now, memory ops are being lowered to strings partly in
scheduler.codegen() and partly in wrapper.codegen(). But that makes
static memory planning (which is done entirely in `wrapper.codegen()`)
difficult to implement as information is "lost" by that point.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler
DISABLED test_fused_int_mm_mul_gating (__main__.TestPaternMatcher)
Labels: module: rocm, triaged, module: flaky-tests, skipped, module: inductor

Platforms: rocm

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/failure/test_fused_int_mm_mul_gating) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17728965207).

Over the past 72 hours, it has flakily failed in 6 workflow(s).

**Debugging instructions (after clicking on the recent samples link):**
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Grep for `test_fused_int_mm_mul_gating`


Test file path: `inductor/test_pattern_matcher.py`

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler
[inductor][BE] split triton_meta and inductor_meta
Labels: release notes: foreach_frontend, topic: not user facing, module: inductor, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #111397

triton_meta is intended to be passed directly to triton. Previous we were also putting other metadata into triton_meta; but we should split out the other metadata into a separate dict to avoid possible conficts in the future.

This PR splits out triton_meta and inductor_meta so we have a place to put additional metadata that isn't intended to be passed to triton.

Tests - wait for CI

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler

Differential Revision: [D50442547](https://our.internmc.facebook.com/intern/diff/D50442547)
Add an explicit _shutdown method to ProcessGroupNCCL
Labels: triaged, open source, release notes: distributed (c10d)

Currently, the only way ProcessGroupNCCL shuts down its background threads and aborts all communicators is via the destructor.

However, given how python GC works and code holding references to the PG in multiple places, in practice calling `destroy_process_group` doesn't actually end up invoking the destructor.

As a result, in this PR I'm adding a explicit shutdown method to that users can call to cleanup all resources.
[quant][pt2] Default batchnorm aten op has poor numerics during QAT
Labels: oncall: quantization, triaged

If a model contains batch norm, exporting `model.cpu()` gives `torch.ops.aten._native_batch_norm_legit.default` while exporting `model.cuda()` gives `torch.ops.aten.cudnn_batch_norm.default`. Quick experimentation revealed that these two ops have significantly different numerics. From running PT2 QAT on MobileNetV2, we found that these numerical differences are magnified through many back-to-back `[conv - bn]` or `[conv - bn - relu]` patterns, and training with `cudnn_batch_norm` gave significantly better accuracies.

For the short term, we recommend users of PT2 QAT to either export with `model.cuda()` from the start, or manually swap the batch norm op from `_native_batch_norm_legit` to `cudnn_batch_norm` before training. See this [tutorial](https://pytorch.org/tutorials/prototype/pt2e_quant_qat.html#prepare-the-model-for-quantization-aware-training) for more detail.

For the longer term, however, we wish to fix this so that the user does not have to do this manual swapping. We can achieve this by consolidating the batch norm ops such that exporting the model always gives the same op, regardless whether it is on CPU or cuda, and that op will automatically pick the right kernel to run on. See this relevant [comment](https://github.com/pytorch/pytorch/pull/108809#issuecomment-1724566234).



cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel
Rename name->qualname in torch.library.impl_abstract

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #111661
* #111660
* #111310
* #111659
* __->__ #111380

See title. Makes this consistent with torch.library.{define, impl, impl_device}, where we have named the same argument `qualname`. This is not BC-breaking because we have not released a version of PyTorch with impl_abstract in it yet.
Tensor.lerp inconsistent when using -Infinity between MPS and CPU
Labels: triaged, module: mps

### 🐛 Describe the bug

lerping between -Inf and another number is NaN on a MPS tensor and -Inf on the CPU. 
This breaks several Diffusion schedulers.

```
import torch

for device in ['cpu', 'mps']:

   minusInfinity = torch.tensor([0]).log().to(device)
   actualNumber = torch.tensor([1.0]).to(device);

   print(minusInfinity);
   print(actualNumber);

   print(minusInfinity.lerp(actualNumber, 0.5))
```

```
tensor([-inf])
tensor([1.])
tensor([-inf])
tensor([-inf], device='mps:0')
tensor([1.], device='mps:0')
tensor([nan], device='mps:0')
```



### Versions

Collecting environment information...
PyTorch version: 2.1.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 14.0 (arm64)
GCC version: Could not collect
Clang version: 15.0.0 (clang-1500.0.40.1)
CMake version: version 3.24.4
Libc version: N/A

Python version: 3.10.13 (main, Sep 28 2023, 14:27:43) [Clang 12.0.5 (clang-1205.0.22.9)] (64-bit runtime)
Python platform: macOS-14.0-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1

Versions of relevant libraries:
[pip3] numpy==1.26.0
[pip3] torch==2.1.0
[pip3] torchvision==0.16.0
[conda] Could not collect


cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev