gopikrsmscs/torch-issues huggingface dataset:
Remove parameter `self` in `typeConvertIndices`
Labels: triaged, module: advanced indexing, better-engineering

### 🚀 The feature, motivation and pitch

In the file `aten/src/ATen/TensorIndexing.h`, there's a function `typeConvertIndices`:

```c++
static inline c10::List<c10::optional<Tensor>> typeConvertIndices(
    const Tensor& /*self*/,
    std::vector<Tensor>&& indices) {
  c10::List<c10::optional<Tensor>> converted_inds;
  converted_inds.reserve(indices.size());
  for (const auto& i : indices) {
    converted_inds.push_back(std::move(i));
  }
  return converted_inds;
}
```

This function is later invoked by `dispatch_index_put_`:

```c++
static inline Tensor dispatch_index_put_(
    Tensor& self,
    std::vector<Tensor>&& indices,
    const Tensor& value) {
  return self.index_put_(
      impl::typeConvertIndices(self, std::move(indices)), value);
}
```

Upon inspection, I'm unsure about the necessity of passing the `self` parameter to `typeConvertIndices` as it seems to be unused within the function. Is it possible to simplify the function to:

```c++
static inline c10::List<c10::optional<Tensor>> typeConvertIndices(std::vector<Tensor>&& indices) {
  c10::List<c10::optional<Tensor>> converted_inds;
  converted_inds.reserve(indices.size());
  for (const auto& i : indices) {
    converted_inds.push_back(std::move(i));
  }
  return converted_inds;
}
```

by removing the `self` parameter?

### Alternatives

_No response_

### Additional context

_No response_
DISABLED test_conv_weight_layout_convert_cuda (__main__.FreezingCudaTests)
Labels: module: rocm, triaged, module: flaky-tests, skipped, module: inductor

Platforms: rocm

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/failure/test_conv_weight_layout_convert_cuda) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/16195333580).

Over the past 72 hours, it has flakily failed in 6 workflow(s).

**Debugging instructions (after clicking on the recent samples link):**
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Grep for `test_conv_weight_layout_convert_cuda`


Test file path: `inductor/test_inductor_freezing.py`

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
[BUG] "weight_norm_fwd_first_dim_kernel" not implemented for 'BFloat16'
Labels: module: nn, triaged, module: bfloat16, actionable

### 🐛 Describe the bug

Attempting to `weight_norm` a bfloat16 layer in a CUDA environment results in an error like 
```RuntimeError: "weight_norm_fwd_first_dim_kernel" not implemented for 'BFloat16'```

No error on `weight_norm` in CPU environment, but only in CUDA environment.

# errror
```
"weight_norm_fwd_first_dim_kernel" not implemented for 'BFloat16'
  File "[user-dir]/lib/python3.10/site-packages/torch/nn/utils/weight_norm.py", line 25, in compute_weight
    return _weight_norm(v, g, self.dim)
  File "[user-dir]/lib/python3.10/site-packages/torch/nn/utils/weight_norm.py", line 50, in apply
    setattr(module, name, fn.compute_weight(module))
  File "[user-dir]/lib/python3.10/site-packages/torch/nn/utils/weight_norm.py", line 109, in weight_norm
    WeightNorm.apply(module, name, dim)
  File "/home/jsb193/workspace/github/llm/LLM42/train/train_dpo.py", line 48, in <module>
    nn.utils.weight_norm(module, "weight")
RuntimeError: "weight_norm_fwd_first_dim_kernel" not implemented for 'BFloat16'
```

# example
```python
import torch.nn as nn

module = torch.nn.Linear(20, 40)
module = module.to(torch.bfloat16)
module = module.to("cuda")
nn.utils.weight_norm(module, "weight")
```

### Versions

pytorch: 2.0.1+cu117
python: 3.10.11
os: ubuntu-18.04
gpu: GeForce3090


cc @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki
Provide a `reset_parameters()` method for MultiheadAttention to support FSDP meta device initializtion
Labels: module: nn, triaged, module: fsdp

### 🚀 The feature, motivation and pitch

The [MultiheadAttention](https://github.com/pytorch/pytorch/blob/2fbe6ef2f866fe6ce42a950f2053f2f6b4bdab90/torch/nn/modules/activation.py) layer has a protected [`_reset_parameters()`](https://github.com/pytorch/pytorch/blob/2fbe6ef2f866fe6ce42a950f2053f2f6b4bdab90/torch/nn/modules/activation.py#L1020) method that is responsible for initializing the params. In light of the newly introduced approach to init modules on the meta device and materializing them with FSDP (https://github.com/pytorch/pytorch/issues/104187), it would be great if the `MultiheadAttention` module could expose the `_reset_parameters` as public so FSDP can call it internally. 


### Alternatives

Currently, the user has to modify the source code or patch it like so:
```py
MultiheadAttention.reset_parameters = MultiheadAttention._reset_parameters
```

### Additional context

cc @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki @zhaojuanmao @mrshenli @rohan-varma @awgu @fegin @penguinwu

I'd be happy to send a PR if there are no objections.
Rzou/out dtype
Labels: module: dynamo, ciflow/inductor

Fixes #ISSUE_NUMBER


cc @voznesenskym @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov
Support values backward on sparse CSR, CSC, BSR, and BSC tensors
Labels: module: autograd, open source, release notes: sparse, topic: new features

Fixes https://github.com/pytorch/pytorch/issues/107286

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #107903
* #107150
* #107777
* #107638



cc @ezyang @albanD @zou3519 @gqchen @nikitaved @soulitzer @Lezcano @Varal7
wip add a test
Labels: module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #107897
* #107834



cc @voznesenskym @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov
[feature request] [ux proposal] Min-max linear normalization to be supported in F.normalize (or in a new function)
Labels: module: nn, triaged, topic: new features

### 🚀 The feature, motivation and pitch

It's useful as one of way of various image / arbitrary normalizations

E.g. supported in OpenCV: https://docs.opencv.org/4.x/d2/de8/group__core__array.html#ggad12cefbcb5291cf958a85b4b67b6149fa9f0c1c342a18114d47b516a88e29822e in the cv2.normalize function: https://docs.opencv.org/4.x/d2/de8/group__core__array.html#ga87eef7ee3970f86906d69a92cbf064bd

Also, related, NumPy supports https://numpy.org/doc/stable/reference/generated/numpy.ptp.html function which computes difference `amax() - amin()`

My impl is simple (althgough it would be best to set by default `eps = torch.finfo(x.dtype).min` to accomodate float16 inputs), but it would be nice to have it supported by core convenience F.normalize function:
```python
def normalize_min_max_(x, dim, eps = 1e-12):
    # workaround for https://github.com/pytorch/pytorch/issues/61582
    amin, amax = x.amin(dim = dim, keepdim = True), x.amax(dim = dim, keepdim = True)
    return x.sub_(amin).div_(amax.sub_(amin).add_(eps))
```

### Alternatives

_No response_

### Additional context

_No response_

cc @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki
DISABLED test_conv_with_as_strided_cpu (__main__.FreezingCpuTests)
Labels: module: rocm, module: cpu, triaged, skipped

Platforms: rocm

This test was disabled because it is failing on main branch ([recent examples](http://torch-ci.com/failure/inductor%2Ftest_inductor_freezing.py%3A%3AFreezingCpuTests%3A%3Atest_conv_with_as_strided_cpu)).

Skipping for now as this has shown up on a few PRs e.g https://github.com/pytorch/pytorch/pull/107812. Long term I do not think we want these CPU tests running on ROCm either way. Disabling with issue for now and will assess further.

cc: @pragupta @jithunnair-amd 

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @hongxiayang @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10
[POC][HSDP] Add option to disable all-reduce only
Labels: release notes: distributed (fsdp), topic: not user facing

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #107880
* #107784
* #106080
* #106068

With >= 4 GPUs:
```
python -m pytest test/distributed/fsdp/test_fsdp_hybrid_shard.py -k test_fsdp_hybrid_shard_accumulation
```
FakeMode should not fakify non persistent buffer
Labels: triaged, oncall: pt2, module: fakeTensor

I initially came across this information while working with ONNX fake mode export, where the FakeMode fakifies all tensors except for constant tensors.

Referring to https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module.register_buffer, non-persistent buffers can not be accessed through state_dict(). This implies that when attempting to open and load the state_dict() of a model in FakeMode, the non-persistent buffers are absent forever. Is there a way to retain non-persistent buffers in FakeMode, similar to how constant tensors are retained?

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @BowenBao
[BE] Consolidation of SymNode methods constant_int, maybe_as_int, etc
Labels: triaged, oncall: pt2

This is something of a preexisting problem, but I wonder why I didn't just have `maybe_as_int` take care of everything...

_Originally posted by @ezyang in https://github.com/pytorch/pytorch/pull/107089#discussion_r1294920333_
            


cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305
Graph break: call_function partial in skip_files
Labels: triaged, oncall: pt2, module: dynamo

### 🐛 Describe the bug

Log looks like:

```
[rank0]:[2023-08-23 20:16:32,832] [12/21] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [UserFunctionVariable(), NNModuleVariable(), UserDefinedObjectVariable(PyTorchEnv), TupleVariable(), ConstDictVariable(), SkipFilesVariable(), UserFunctionVariable(), NNModuleVariable()]
[rank0]:[2023-08-23 20:16:32,833] [12/21] torch._dynamo.symbolic_convert: [DEBUG] empty checkpoint
[rank0]:[2023-08-23 20:16:32,833] [12/21] torch._dynamo.symbolic_convert: [DEBUG] FAILED INLINING <code object call_module_impl at 0x7fb4e45a8570, file "<torch_package_0>.dper3/core/environment.py", line 1267>
[rank0]:[2023-08-23 20:16:32,833] [12/21] torch._dynamo.output_graph: [DEBUG] restore_graphstate: removed 0 nodes
[rank0]:[2023-08-23 20:16:32,833] [12/21] torch._dynamo.symbolic_convert: [DEBUG] break_graph_if_unsupported triggered compile
[rank0]:[2023-08-23 20:16:32,833] [12/21] torch._dynamo.symbolic_convert.__graph_breaks: [DEBUG] Graph break: call_function partial in skip_files /data/users/ftruzzi/fbsource/buck-out/v2/gen/fbcode/4f72fc365136a62e/third-party-buck/platform010/build/python/cinder.3.8/__python_runtime__/python_runtime/lib/python3.8/functools.py from user code at:
[rank0]:[2023-08-23 20:16:32,833] [12/21] torch._dynamo.symbolic_convert.__graph_breaks: [DEBUG]   File "<torch_package_0>.dper3/core/environment.py", line 1279, in <resume in call_module>
[rank0]:[2023-08-23 20:16:32,833] [12/21] torch._dynamo.symbolic_convert.__graph_breaks: [DEBUG]     return call_module_impl(self, m, *args, **kwargs)
[rank0]:[2023-08-23 20:16:32,833] [12/21] torch._dynamo.symbolic_convert.__graph_breaks: [DEBUG]   File "<torch_package_0>.dper3/core/environment.py", line 1270, in call_module_impl
[rank0]:[2023-08-23 20:16:32,833] [12/21] torch._dynamo.symbolic_convert.__graph_breaks: [DEBUG]     m, env, args, kwargs, partial(torch.nn.Module.__call__, m)
[rank0]:[2023-08-23 20:16:32,833] [12/21] torch._dynamo.symbolic_convert.__graph_breaks: [DEBUG] 
```

@voznesenskym didn't you have a fix for this? Can we land it?

### Versions

main

cc @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov
About the multi-node example not working properly
Labels: oncall: distributed, triaged

### 🐛 Describe the bug

My machine: 2 machines with different ips and 2 available Gpus on each machine

When I use the multigpu_torchrun.py example, when I pass these two directives:
`torchrun --nproc_per_node=2 --nnodes=2 --node_rank=0  --rdzv_id=456 --rdzv_backend=c10d --rdzv_endpoint=172.xx.1.150:29603 multi_node_torchrun.py 50 10`
and
`torchrun --nproc_per_node=2 --nnodes=2 --node_rank=1  --rdzv_id=456 --rdzv_backend=c10d --rdzv_endpoint=172.xx.1.150:29603 multi_node_torchrun.py 50 10`
When I started, the program got stuck in `self.model = DDP(self.model, device_ids=[self.local_rank])` and stopped running, But with `nvidia-smi` we can see that processes on both machines have been created and are already occupying memory. I wonder why

Looking through the history I was able to find similar issues, saying they involved synchronization deadlocks, but I don't think that was the root cause since I was using the official example.

### Versions

os: ubuntu20.04
cuda: 11.8
python: 3.8.17
torch: 1.12.1

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu
Throw error if setting static grads to `None` in `zero_grad()`
Labels: ciflow/trunk, module: dynamo, release notes: dynamo, release notes: optimizer

If a tensor is marked a static address for dynamo, this implies that the pointer will be the same across calls to the optimizer step. For this constraint to hold, we should not set grads marked as static to None. 

After some discussion, I opted to throw an error.

cc @voznesenskym @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov
[Inductor] Extend Pattern Matcher to Match Equivalent Function Invocation
Labels: fb-exported, Merged, Reverted, ciflow/trunk, topic: not user facing, module: inductor, ciflow/inductor

Fixes #104391


cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
FSDP custom args per module
Labels: triaged, module: fsdp

### 🚀 The feature, motivation and pitch

FSDP gives a set of arguments which are global across all wrapped modules. Instead, I would like to wrap some modules with custom args. For example, I have a module that I want to sync only across some GPUs, so I would like to specify custom process_groups which only hit a subset of ranks. 

Currently, PyTorch FSDP works with this but does not support an API to expose it. For example, in this [PR](https://github.com/mosaicml/composer/pull/2460), I monkeypatch `auto_wrap` to allow returning kwargs instead of just a bool which are then used to override the FSDP args for that wrap. This lets me change the process group for specific modules. So far, we remonkeypatch PyTorch every single release since 1.13 since this is critical for us but not supported. It would be amazing if this was supported. 

Our implementation extends auto_wrap to return either a bool or a set of override kwargs. I am happy to upstream this if it is acceptable. 

I imagine this would be useful to squeeze out more performance in general, e.g. some things I might wrap in FULL_SHARD and some in SHARD_GRAD_OP.

### Alternatives

Monkeypatching pytorch https://github.com/mosaicml/composer/pull/2460

### Additional context

_No response_

cc @zhaojuanmao @mrshenli @rohan-varma @awgu @fegin @penguinwu
[WIP/CI Test] Try to tighten up VT stack invariant
Labels: module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #107804
* #107803



cc @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov
[feature request] [discussion] Include basic `ctypes` bindings for `cudart`/`cublasLt`/`cublas`/`nvrtc`/`cudnn` with stock PyTorch
Labels: feature, module: cuda, triaged, module: cublas

### 🚀 The feature, motivation and pitch

This would make it more approachable experimenting with some new function variants (like quantized int8 gemms) earlier.

Example of such ctypes bindings: https://github.com/OpenBMB/cpm_kernels/tree/master/cpm_kernels/library
Including in core some bindings like this would be great! (maybe under some `torch.cuda.ctypes.cublasLt` or something similar)

Examples of such C bindings: https://github.com/TimDettmers/bitsandbytes/blob/18e827d666fa2b70a12d539ccedc17aa51b2c97c/csrc/ops.cu#L434

Another set of bindings is now in `bitsandbytes`, but having it directly available in Python would make it more approachable for experimentation and benchmarking

There might be problems with versions, but maybe then some bindings could be versioned as well: `torch.cuda.ctypes.cublatltV8` or sth like that, so that the user is responsible for using the correct bindings for their experiments

### Alternatives

_No response_

### Additional context

_No response_

cc @ptrblck @csarofeen @xwang233
Back out "[inductor] make thread order consistent with loop order (#106827)"
Labels: fb-exported, module: inductor, ciflow/inductor

Summary: D48295371 cause batch fusion failure

Test Plan: Without revert, f469732293. With revert diff f472266199.

Differential Revision: D48593029



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
Add caffe2 ideep/onednn tests to OSS CI
Labels: oncall: releng, module: ci, triaged, module: mkldnn

### 🐛 Describe the bug

Following PR https://github.com/pytorch/pytorch/pull/97957 that updates ideep have failed in these tests internally:
https://github.com/pytorch/pytorch/tree/main/caffe2/python/ideep

caffe2 module is still used intrnally a lot. Hence we want to enable OSS CI execute ideep tests from time to time.
Perhaps we should be adding these tests as periodic.


### Versions

2.1.0

cc @seemethere @malfet @pytorch/pytorch-dev-infra @gujinghui @PenghuiCheng @XiaobingSuper @jianyuh @jgong5 @mingfeima @sanchitintel @ashokei @jingxu10 @min-jean-cho @yanbing-j @Guobing-Chen @Xia-Weiwen
DISABLED test_conv_stride_constraints (__main__.CPUReproTests)
Labels: module: rocm, triaged, skipped

Platforms: rocm

This test was disabled because it is failing on main branch ([recent examples](http://torch-ci.com/failure/inductor%2Ftest_cpu_repro.py%3A%3ACPUReproTests%3A%3Atest_conv_stride_constraints)).

I will take a look at this further locally, currently unsure why this shows up on ROCm for a CPU test.

cc: @jithunnair-amd 

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @hongxiayang
libtorch infer error : CUDNN_STATUS_INTERNAL_ERROR
Labels: oncall: jit

### 🐛 Describe the bug

# code for model export
```
traced_script_module = torch.jit.trace(model, \
  (data_dict['voxels'], data_dict['voxel_num_points'], data_dict['voxel_coords']))
traced_script_module.save(args.output_path)
```

# code for model infer
```
std::vector<torch::Tensor> output =
    net_.forward(torch_inputs).toTensorVector();
```

# question
model init is ok:
```
torch::Device device(device_type_, device_id_);
net_ = torch::jit::load(model_file_, device);
net_.eval();
```
The error occurs when running the forward function:
![image](https://github.com/pytorch/pytorch/assets/39874362/7610a502-774f-42b3-98ba-fc7b030e47a0)


### Versions

pytorch 1.7.0

libtorch 1.7.0
cudnn 8.0.4.30
cuda 11.1

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel
libtorch vs (onnx+tensorRT) show different object detection results
Labels: module: onnx, triaged

### 🐛 Describe the bug

# libtorch
```
traced_script_module = torch.jit.trace(model, image)
traced_script_module.save(args.output_path)
```

# onnx
```
torch.onnx.export(model, image,
onnx_model_save_path, opset_version=11, verbose=False, export_params=True, 
operator_export_type=OperatorExportTypes.ONNX,
input_names=['image'], output_names=['orient','conf','dim'])
```

# question
We used the two approaches to export the trained model (YOLO), and use it for inference in the object detection task.
However, different results were obtained, and the model exported by onnx shows a better and stable performance.
Could you please give some possible reasons?

### Versions

torch 2.0.1
onnx 1.14.0
onnxconverter-common 1.14.0
onnxmltools 1.11.2
onnxruntime-gpu 1.15.1
Enable Mypy Checking in torch/_inductor/bounds.py
Labels: triaged, open source, module: inductor, ciflow/inductor

Fixes #105230 

Summary:

As suggested in https://github.com/pytorch/pytorch/issues/105230 mypy checking is enabled in torch/_inductor/bounds.py

After Fix:

mypy --follow-imports=skip torch/_inductor/bounds.py Success: no issues found in 1 source file

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
DISABLED test_make_fx_symbolic_exhaustive_special_bessel_y0_cpu_float32 (__main__.TestProxyTensorOpInfoCPU)
Labels: triaged, module: flaky-tests, skipped, module: ProxyTensor

Platforms: linux

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_make_fx_symbolic_exhaustive_special_bessel_y0_cpu_float32&suite=TestProxyTensorOpInfoCPU) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/16128051710).

Over the past 3 hours, it has been determined flaky in 3 workflow(s) with 3 failures and 3 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_make_fx_symbolic_exhaustive_special_bessel_y0_cpu_float32`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_proxy_tensor.py`
Add Half support for range, logspace, logit, median, nanmedian, kthvalue, poisson, cummax, cummin, prod, cumprod, histc, logcumsumexp, vander, cross, aten2, logaddexp, logaddexp2, hypot, and nextafter on CPU
Labels: module: cpu, open source, module: half, ciflow/trunk, topic: not user facing, ciflow/periodic, ciflow/mps, module: inductor, ciflow/inductor

Add Half support for range, logspace, logit, median, nanmedian, kthvalue, poisson, cummax, cummin, prod, cumprod, histc, logcumsumexp, vander, cross, aten2, logaddexp, logaddexp2, hypot, and nextafter on CPU.


cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @voznesenskym @penguinwu @EikanWang @Guobing-Chen @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @Xia-Weiwen @ngimel
DISABLED test_make_fx_symbolic_exhaustive_special_bessel_j1_cpu_float32 (__main__.TestProxyTensorOpInfoCPU)
Labels: triaged, module: flaky-tests, skipped, module: ProxyTensor

Platforms: linux

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_make_fx_symbolic_exhaustive_special_bessel_j1_cpu_float32&suite=TestProxyTensorOpInfoCPU) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/16122457339).

Over the past 3 hours, it has been determined flaky in 3 workflow(s) with 3 failures and 3 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_make_fx_symbolic_exhaustive_special_bessel_j1_cpu_float32`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_proxy_tensor.py`
conv cudnn support integers
Labels: module: cudnn, triaged

### 🚀 The feature, motivation and pitch

conv cudnn support integers

i tried implementing by adding
 
```cpp
else if (scalar_type == at::kChar) {
    return CUDNN_DATA_INT8;
  }
```
to https://github.com/pytorch/pytorch/blob/c093fdf9245875213cae65eba9e246a70748f9c0/aten/src/ATen/cudnn/Descriptors.cpp#L25C1-L26

but ran into 

```python
    output = F.conv2d(input, kernel)
RuntimeError: cuDNN error: CUDNN_STATUS_BAD_PARAM
```

@ezyang @dzdang do you some pointers on what specifically i am missing?

related PR https://github.com/pytorch/pytorch/pull/3666 
https://github.com/pytorch/pytorch/pull/74673

cc: @cchan @ezhang887

### Alternatives

_No response_

### Additional context

_No response_

cc @csarofeen @ptrblck @xwang233
DISABLED test_make_fx_symbolic_exhaustive_special_airy_ai_cpu_float32 (__main__.TestProxyTensorOpInfoCPU)
Labels: triaged, module: flaky-tests, skipped, module: ProxyTensor

Platforms: linux

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_make_fx_symbolic_exhaustive_special_airy_ai_cpu_float32&suite=TestProxyTensorOpInfoCPU) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/16113161016).

Over the past 3 hours, it has been determined flaky in 3 workflow(s) with 3 failures and 3 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_make_fx_symbolic_exhaustive_special_airy_ai_cpu_float32`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_proxy_tensor.py`
[ONNX] Retire FXSymbolicTracer in FX exporter
Labels: module: onnx, triaged, onnx-triaged

FXSymbolicTracer is a pioneer of exploring fake tensor export, which does successfully export large scale transformers, like BLOOM, and GPT2. However, it's not using dynamo export, but `torch.fx._symbolic_trace.Tracer` which is not actively maintained. Besides, there are patches used to only cover FXSymbolicTracer, which is not exposed to public API, and will never be.. Now, with the fake mode api in fx exporter becomes mature. We should retire FXSymbolicTracer.
DISABLED test_multilayer_var_dynamic_shapes_cpu (__main__.DynamicShapesCpuTests)
Labels: triaged, skipped, module: dynamic shapes

Platforms: macos

This test was disabled because it is failing on main branch ([recent examples](http://torch-ci.com/failure/inductor%2Ftest_torchinductor_dynamic_shapes.py%3A%3ADynamicShapesCpuTests%3A%3Atest_multilayer_var_dynamic_shapes_cpu)).

cc @ezyang
Hardtanh docs are inaccurate/incomplete, since hardtanh behaves like clamp
Labels: module: docs, triaged

### 📚 The doc issue

The issue is more evident when `input` < `max_val` < `min_val`, for example:

```python
import torch

torch.ops.aten.hardtanh(torch.tensor(2), min_val=4, max_val=3)
```
outputs:
```
tensor(3)
```

however, according to the [hardtanh docs](https://pytorch.org/docs/stable/generated/torch.nn.Hardtanh.html) it should return the `min_val`, i.e. 
```
tensor(4)
``` 

### Suggest a potential alternative/fix

The documentation should use `min` and `max`, like clamp's documentation. Something like:

```
HardTanh(x) = min(max(x, min_val), max_value)
```

cc @svekars @carljparker
Inconsistencies when handling scalars that are out of the range relative to the input tensor's dtype
Labels: triaged, module: int overflow

### 🐛 Describe the bug


```python
import torch

# Both should fail, or both should output 3.
# However ATen converts -129 to 127 uint8, applying that as the min. But it fails if the dtype is int8

torch.ops.aten.clamp.out(torch.tensor([3], dtype=torch.uint8), -129, None, out=torch.tensor([], dtype=torch.uint8))
# tensor([127], dtype=torch.uint8)

torch.ops.aten.clamp.out(torch.tensor([3], dtype=torch.int8), -129, None, out=torch.tensor([], dtype=torch.int8))
# RuntimeError: value cannot be converted to type int8_t without overflow
```



### Versions

Collecting environment information...
PyTorch version: 2.0.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.5 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.202)
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.6 (default, Oct 18 2022, 12:41:40) [Clang 14.0.0 (clang-1400.0.29.202)] (64-bit runtime)
Python platform: macOS-13.5-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Pro

Versions of relevant libraries:
[pip3] numpy==1.24.3
[pip3] torch==2.0.1
[conda] Could not collect
arange.out produces incorrect output when out tensor has dtype long
Labels: triaged, module: python frontend

### 🐛 Describe the bug

```python
import torch

print(torch.ops.aten.arange.out(2.3, out=torch.tensor([], dtype=torch.uint8)))
print(torch.ops.aten.arange.out(2.3, out=torch.tensor([], dtype=torch.int8)))
print(torch.ops.aten.arange.out(2.3, out=torch.tensor([], dtype=torch.int16)))
print(torch.ops.aten.arange.out(2.3, out=torch.tensor([], dtype=torch.int32)))
print(torch.ops.aten.arange.out(2.3, out=torch.tensor([], dtype=torch.int64)))
```

Outputs:
```
tensor([0, 1, 2], dtype=torch.uint8)
tensor([0, 1, 2], dtype=torch.int8)
tensor([0, 1, 2], dtype=torch.int16)
tensor([0, 1, 2], dtype=torch.int32)
tensor([0, 1])
```

### Versions

Collecting environment information...
PyTorch version: 2.0.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.5 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.202)
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.6 (default, Oct 18 2022, 12:41:40) [Clang 14.0.0 (clang-1400.0.29.202)] (64-bit runtime)
Python platform: macOS-13.5-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Pro

Versions of relevant libraries:
[pip3] numpy==1.24.3
[pip3] torch==2.0.1
[conda] Could not collect

cc @albanD
where.self_out doesn't fail gracefully when inputs have different dtypes
Labels: triaged, module: type promotion, module: advanced indexing, module: edge cases

### 🐛 Describe the bug

This error seems to happen every time that `input`, `other` and `out` don't have the same dtype

```python
import torch

cond = torch.zeros(2, dtype=torch.bool)
input = torch.zeros(2, dtype=torch.int)
other = torch.zeros(2, dtype=torch.long)
out = torch.zeros(2, dtype=torch.int)

torch.ops.aten.where.self_out(cond, input, other, out=out)
```

Throws this error:
```
RuntimeError: !needs_dynamic_casting<func_t>::check(iter) INTERNAL ASSERT FAILED at "/Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/native/cpu/Loops.h":310, please report a bug to PyTorch.
```

### Versions

Collecting environment information...
PyTorch version: 2.0.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.5 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.202)
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.6 (default, Oct 18 2022, 12:41:40) [Clang 14.0.0 (clang-1400.0.29.202)] (64-bit runtime)
Python platform: macOS-13.5-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Pro

Versions of relevant libraries:
[pip3] numpy==1.24.3
[pip3] torch==2.0.1
[conda] Could not collect

cc @ezyang @gchanan @zou3519 @nairbv @mruberry
index.Tensor_out & index_put.out errors or segfaults with indices list containing only null tensors 
Labels: high priority, triaged, module: advanced indexing

### 🐛 Describe the bug

Indicies with **only** null tensors causes an error or segfault.
This doesn't happen if `indices` contains other tensors that aren't null

```python
import torch

input = torch.tensor([[1, 2], [3, 4]], dtype=torch.float)
indices = [None]
values = torch.tensor([10, 20], dtype=torch.float)
out = torch.tensor([[0, 0], [0, 0]], dtype=torch.float)

# Run any of the following two lines
torch.ops.aten.index.Tensor_out(input, indices, out=out)
torch.ops.aten.index_put.out(input, indices, values, False, out=out)
```

sometimes it segfaults:
```
zsh: segmentation fault
```
sometimes it throws this error:
```
RuntimeError: ntensor >= 3 INTERNAL ASSERT FAILED at "/Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/native/cpu/IndexKernelUtils.h":10, please report a bug to PyTorch.
```

### Versions

Collecting environment information...
PyTorch version: 2.0.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.5 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.202)
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.6 (default, Oct 18 2022, 12:41:40)  [Clang 14.0.0 (clang-1400.0.29.202)] (64-bit runtime)
Python platform: macOS-13.5-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Pro

Versions of relevant libraries:
[pip3] numpy==1.24.3
[pip3] torch==2.0.1
[conda] Could not collect

cc @ezyang @gchanan @zou3519
Enable thp(transparent huge pages) for buffer sizes >=2MB
Labels: triaged, open source

The 2MB thp pages provide better allocation latencies compared to the standard 4KB pages. This change has shown substantial improvement for batch mode usecases where the tensor sizes are larger than 100MB.

Only enabled if THP_MEM_ALLOC_ENABLE environment variable is set.

re-landing https://github.com/pytorch/pytorch/pull/93888
cc: @izaitsevfb
New variables in torch._ops.py pollute the torch.ops namespace
Labels: triaged, module: library

### 🐛 Describe the bug

e.g. torch.ops.dl_open_guard is a thing, and prevents people from creating operators under a "dl_open_guard" namespace (e.g. torch.ops.dl_open_guard.my_conv2d.default)

### Versions

main

cc @anjali411
masked_fill_ outputs incorrect results for 'mps' tensor after transpose
Labels: triaged, module: mps

### 🐛 Describe the bug

I found a bug with 'mps' backend tensor, which results in incorrect output for `masked_fill_` operation after `transpose`
The following cod reproduces the behavior on my mac
```py
import torch

x = torch.tensor([[
         False, False,  True,  True,],
        [True,  True,  True,  True]], device='mps:0')

def transform(x):
    return torch.zeros_like(x, dtype=torch.float32).masked_fill_(x, 1.0)

print(x)
print(transform(x))

print(x.t())
print(transform(x.t()))
```

The output I get from the script is
```
tensor([[False, False,  True,  True],
        [ True,  True,  True,  True]], device='mps:0')
tensor([[0., 0., 1., 1.],
        [1., 1., 1., 1.]], device='mps:0')
tensor([[False,  True],
        [False,  True],
        [ True,  True],
        [ True,  True]], device='mps:0')
tensor([[0., 1.],
        [1., 1.],
        [0., 1.],
        [1., 1.]], device='mps:0')
```

`transform(x)` works as expected, but `transform(x.t())` results in incorrect output.

### Versions

Collecting environment information...
PyTorch version: 2.0.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.5 (arm64)
GCC version: Could not collect
Clang version: 14.0.3 (clang-1403.0.22.14.1)
CMake version: version 3.25.1
Libc version: N/A

Python version: 3.11.4 (main, Jul 25 2023, 17:36:13) [Clang 14.0.3 (clang-1403.0.22.14.1)] (64-bit runtime)
Python platform: macOS-13.5-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Pro

Versions of relevant libraries:
[pip3] numpy==1.25.2
[pip3] torch==2.0.1
[conda] No relevant packages

cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
Actually raise an error on all graph breaks with fullgraph=True
Labels: Stale, module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #107683

Previously, if you had a graph break inside a fullgraph=True compile,
but Dynamo was able to compile a partial graph, we would let the graph
through anyway (instead of erroring).  Now you error.

Fixes https://github.com/pytorch/pytorch/issues/107639

Signed-off-by: Edward Z. Yang <ezyang@meta.com>

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng @anijain2305 @Xia-Weiwen
torch.nn.functional.cross_entropy different loss when providing one_hot_target and class weights
Labels: module: nn, module: loss, triaged

### 🐛 Describe the bug

```python
inputs = torch.tensor([[2.3, 2.1, 4.5],
                         [4.2, 3.2, 5.1]])

target = torch.tensor([1, 2])
one_hot_target = torch.tensor([[0,1,0], [0,0,1]]).float()

weights = torch.tensor([25., 25., 100.])

torch.nn.functional.cross_entropy(inputs, target, reduction="mean", weight=weights) #output: 0.8705
torch.nn.functional.cross_entropy(inputs, one_hot_target, reduction="mean", weight=weights) #output: 54.4052
```

This seems to only occur with `mean` reduction when `weight` is provided, `sum` and `none` output the same thing.

### Versions

PyTorch version: 2.0.1+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux 10 (buster) (x86_64)
GCC version: (Debian 8.3.0-6) 8.3.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.28

Python version: 3.10.0 (default, Mar  3 2022, 09:58:08) [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.15.0-1041-azure-x86_64-with-glibc2.28
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       48 bits physical, 48 bits virtual
CPU(s):              8
On-line CPU(s) list: 0-7
Thread(s) per core:  2
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          25
Model:               1
Model name:          AMD EPYC 7763 64-Core Processor
Stepping:            1
CPU MHz:             2445.425
BogoMIPS:            4890.85
Virtualization:      AMD-V
Hypervisor vendor:   Microsoft
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            32768K
NUMA node0 CPU(s):   0-7
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm

Versions of relevant libraries:
[pip3] numpy==1.25.2
[pip3] torch==2.0.1+cpu
[pip3] torch-scatter==2.1.1
[conda] numpy                     1.25.2                   pypi_0    pypi

cc @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki
Enable Mypy Checking in torch/_inductor/debug.py
Labels: open source, topic: not user facing, module: inductor, ciflow/inductor

Fixes #105230

Summary:

As suggested in https://github.com/pytorch/pytorch/issues/105230 mypy checking is enabled in torch/_inductor/debug.py

After Fix:

mypy --follow-imports=skip torch/_inductor/debug.py Success: no issues found in 1 source file

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
Add mha to Autocast CPU
Labels: triaged, open source, module: amp (automated mixed precision), ciflow/trunk, topic: not user facing, intel

Fixes #106751.

This PR adds `_native_multi_head_attention` to Autocast CPU policy.

Behavior: Within the scope of torch.cpu.amp.autocast(dtype=torch.bfloat16) , `_native_multi_head_attention` will be forced to run with bf16 data type.

cc @mcarilli @ptrblck @leslie-fang-intel @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10
adding _int_mm to out_dtype mm WIP
Labels: Stale, module: inductor, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #107664

Summary: it looks like out_dtype doesn't work with max-autotune in
torch.compile

Test Plan: python pytorch/torch/_inductor/ir.py -k "int_mm"

Reviewers:

Subscribers:

Tasks:

Tags:

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler
fullgraph=True doesn't actually raise error when you don't manage full graph inside DDP
Labels: triaged, oncall: pt2, module: dynamo

### 🐛 Describe the bug

This is same repro as https://github.com/pytorch/pytorch/issues/107637

In the logs https://gist.github.com/ezyang/e4bf7345326092138969387ba364f3ea#file-dist-log-L752-L761 you can see that we don't raise an error when the first graph fails compiling, we just keep going.

### Versions

main

cc @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov
Support integer implementations for max_pool1d/2d/3d (cpu and cuda)
Labels: module: cpu, triaged, open source

Fixes #107412


cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10

cc: @cchan @ezhang887


also supports `torch.compile`
```python
import torch
f = lambda y:torch.nn.functional.max_pool1d(y, kernel_size=3)

f_compiled = torch.compile(f)

x = torch.randn(3,3,3, device="cpu").to(torch.int8)
print(x)
print(f_compiled(x))
```

output
```
tensor([[[ 0,  0,  1],
         [ 0,  0,  0],
         [ 0, -1,  0]],

        [[ 0,  1,  0],
         [-1, -1,  0],
         [ 0,  1,  0]],

        [[ 1,  0,  0],
         [ 1,  1,  0],
         [-1,  0, -1]]], dtype=torch.int8)
tensor([[[1],
         [0],
         [0]],

        [[1],
         [0],
         [1]],

        [[1],
         [1],
         [0]]], dtype=torch.int8)
```
nvfuser does not respect CMAKE_INSTALL_PREFIX when build (cmake) libtorch
Labels: triaged, module: nvfuser

### 🐛 Describe the bug

I followed the libtorch build instructions from https://github.com/pytorch/pytorch/blob/main/docs/libtorch.rst#building-libtorch-using-cmake:

```
git clone -b main --recurse-submodule https://github.com/pytorch/pytorch.git
mkdir pytorch-build
cd pytorch-build
cmake -DBUILD_SHARED_LIBS:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=Release -DPYTHON_EXECUTABLE:PATH=`which python3` -DCMAKE_INSTALL_PREFIX:PATH=../pytorch-install ../pytorch
cmake --build . --target install
```

However, "libnvfuser_codegen.so" is missing from the installation dir "pytorch-install". Looks like it's bc nvfuser_codegen's install destination is hard-coded to <ProjectRoot>/torch/lib. In nvfuser/CMakeLists.txt:

```
set(TORCH_INSTALL_LIB_DIR ${TORCH_ROOT}/torch/lib)
...
install(TARGETS ${NVFUSER_CODEGEN} EXPORT NvfuserTargets DESTINATION "${TORCH_INSTALL_LIB_DIR}")
```

see https://github.com/pytorch/pytorch/blob/2b32a74ab084a9379c9e46a792c95348a6fc0971/third_party/nvfuser/CMakeLists.txt#L21

Please make nvfuser lib install destination settings consistent with other libs.


### Versions

since 2.0.0

cc @kevinstephano @jjsjann123
torch.fx.Interpreter modules don't get compiled
Labels: triaged, module: dynamo

As a small example:
```
import torch
import torch.fx

class InterpreterModule(torch.fx.GraphModule):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.interpreter = torch.fx.Interpreter(self)

    def __call__(self, *args, **kwargs):
        return self.interpreter.run(*args, enable_io_processing=False)

class Mod(torch.nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, x):
        x = x + x
        x = x * x
        return x

gm = torch.fx.symbolic_trace(Mod())
imodule = InterpreterModule({}, gm.graph)
opt_mod = torch.compile(imodule)

for i in range(1000):
    opt_mod(torch.ones(2, 3))
```
Turning on the logging, I see that we avoid compiling over `torch.fx.Interpreter` because it's getting skipfile'd. When I try adding `torch.fx.Interpreter` to the skipfile, it skips again with `GraphModule.graph`, so I gave up here and decided to record an issue instead.

Not sure how easy/hard this is to support with dynamo, but it would be nice. Generally we like using `torch.fx.Interpreter` instead of the torch.fx Python codegen for two reasons:
1. You get better stack traces in the case of exceptions
2. It's easier to debug/interact with intermediates

cc @voznesenskym @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov
[ao] updating embedding_bag support for fx and eager
Labels: release notes: quantization

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #107623

Summary: our docs were saying dynamic embedding bag wasn't supported but
it actually is (at least at the same level as embeddings were) it just wasn't previously tested/listed.

Test Plan: python test/test_quantization.py -k "test_embedding"

Reviewers:

Subscribers:

Tasks:

Tags: