gopikrsmscs/torch-issues huggingface dataset:
p.block != nullptr && p.block->ptr != nullptr INTERNAL ASSERT FAILED
Labels: needs reproduction, triaged

### 🐛 Describe the bug

Traceback (most recent call last):
  File "main.py", line 141, in <module>
    output = model(train_img)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/timm/models/efficientnet.py", line 557, in forward
    x = self.forward_features(x)
  File "/opt/conda/lib/python3.7/site-packages/timm/models/efficientnet.py", line 545, in forward_features
    x = self.blocks(x)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/container.py", line 119, in forward
    input = module(input)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/container.py", line 119, in forward
    input = module(input)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/timm/models/efficientnet_blocks.py", line 185, in forward
    x = self.se(x)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/timm/models/efficientnet_blocks.py", line 56, in forward
    return x * self.gate(x_se)

RuntimeError: p.block != nullptr && p.block->ptr != nullptr INTERNAL ASSERT FAILED at "/pytorch/c10/cuda/CUDACachingAllocator.cpp":689, please report a bug to PyTorch.


### Versions

pytorch1.8+cu11
Couldn't install pytorch 2.0
Labels: needs reproduction, oncall: binaries, triaged

### 🐛 Describe the bug

 Using the command in V100 cuda 11.7 with py3.8: 
```bash
pip3 install numpy --pre torch[dynamo] torchvision torchaudio --force-reinstall --extra-index-url https://download.pytorch.org/whl/nightly/cu117
```

but got following information while installing torchaudio and torchvision:
```bash
Collecting torchaudio
  Downloading https://download.pytorch.org/whl/nightly/cu117/torchaudio-0.14.0.dev20221202%2Bcu117-cp38-cp38-linux_x86_64.whl (4.2 MB)
     |████████████████████████████████| 4.2 MB 270 kB/s
  Downloading https://download.pytorch.org/whl/nightly/cu117/torchaudio-0.14.0.dev20221201%2Bcu117-cp38-cp38-linux_x86_64.whl (4.2 MB)
     |████████████████████████████████| 4.2 MB 668 kB/s
  Downloading https://download.pytorch.org/whl/nightly/cu117/torchaudio-0.14.0.dev20221130%2Bcu117-cp38-cp38-linux_x86_64.whl (4.2 MB)
     |████████████████████████████████| 4.2 MB 665 kB/s
  Downloading https://download.pytorch.org/whl/nightly/cu117/torchaudio-0.14.0.dev20221129%2Bcu117-cp38-cp38-linux_x86_64.whl (4.2 MB)
     |████████████████████████████████| 4.2 MB 906 kB/s
  Downloading https://download.pytorch.org/whl/nightly/cu117/torchaudio-0.14.0.dev20221128%2Bcu117-cp38-cp38-linux_x86_64.whl (4.2 MB)
     |████████████████████████████████| 4.2 MB 694 kB/s
  Downloading https://download.pytorch.org/whl/nightly/cu117/torchaudio-0.14.0.dev20221127%2Bcu117-cp38-cp38-linux_x86_64.whl (4.2 MB)
     |████████████████████████████████| 4.2 MB 694 kB/s
  Downloading https://download.pytorch.org/whl/nightly/cu117/torchaudio-0.14.0.dev20221126%2Bcu117-cp38-cp38-linux_x86_64.whl (4.2 MB)
     |████████████████████████████████| 4.2 MB 674 kB/s
  Downloading https://download.pytorch.org/whl/nightly/cu117/torchaudio-0.14.0.dev20221125%2Bcu117-cp38-cp38-linux_x86_64.whl (4.2 MB)
     |████████████████████████████████| 4.2 MB 685 kB/s
  Downloading https://download.pytorch.org/whl/nightly/cu117/torchaudio-0.14.0.dev20221124%2Bcu117-cp38-cp38-linux_x86_64.whl (4.2 MB)
     |████████████████████████████████| 4.2 MB 693 kB/s
  Downloading https://download.pytorch.org/whl/nightly/cu117/torchaudio-0.14.0.dev20221123%2Bcu117-cp38-cp38-linux_x86_64.whl (4.2 MB)
     |████████████████████████████████| 4.2 MB 689 kB/s
  
..........

NFO: pip is looking at multiple versions of torchvision to determine which version is compatible with other requirements. This could take a while.
Collecting torchvision

.........

INFO: pip is looking at multiple versions of torchtriton to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of torch[dynamo] to determine which version is compatible with other requirements. This could take a while.
```


### Versions

torch.1.14

cc @ezyang @seemethere @malfet
documentation need to be as per python version
Labels: module: docs, triaged

### 📚 The doc issue

something like this,

python 3.9
```
def init_weights(m):
    if isinstance(m, nn.Linear):
        nn.init.normal_(m.weight)
        nn.init.normal_(m.bias)

model = nn.Sequential(nn.Linear(2, 2))
model.apply(init_weights)
```

python 3.10
```
def init_weights(m):
    match m:
        case nn.Linear():
            nn.init.normal_(m.weight)
            nn.init.normal_(m.bias)

model = nn.Sequential(nn.Linear(2, 2))
model.apply(init_weights)
```
motivation -
docs of FastAPI https://fastapi.tiangolo.com/tutorial/body/, 
pydantic https://pydantic-docs.helpmanual.io/usage/dataclasses/ 
follow a similar strategy

### Suggest a potential alternative/fix

no

cc @svekars @carljparker
no matches found: torch[dynamo]
Labels: module: docs, triaged, actionable, oncall: pt2

### 🐛 Describe the bug

Following [PyTorch 2.0 Official Guide](https://pytorch.org/blog/Accelerating-Hugging-Face-and-TIMM-models/), I am trying to install it using

```bash
pip3 install numpy --pre torch[dynamo] --force-reinstall --extra-index-url https://download.pytorch.org/whl/nightly/cu117
```

Resulting in the following error 

```
zsh: no matches found: torch[dynamo]
```

Thanks

### Versions

```
OS: Ubuntu 22.10 (x86_64)
GCC version: (Ubuntu 12.2.0-3ubuntu1) 12.2.0
Clang version: Could not collect
CMake version: version 3.25.0
Libc version: glibc-2.36

Python version: 3.9.13 (main, Aug 25 2022, 23:26:10)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.19.0-26-generic-x86_64-with-glibc2.36
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090
Nvidia driver version: 515.76
cuDNN version: Probably one of the following:
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn.so.8
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_adv_train.so.8
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_ops_train.so.8
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.24.0rc1
[pip3] torch==1.14.0.dev20221202+cu117
[pip3] torchaudio==0.12.1+cu116
[pip3] torchtriton==2.0.0+0d7e753227
[pip3] torchvision==0.13.1+cu116
[conda] numpy                     1.24.0rc1                pypi_0    pypi
[conda] torch                     1.14.0.dev20221202+cu117          pypi_0    pypi
[conda] torchaudio                0.12.1+cu116             pypi_0    pypi
[conda] torchtriton               2.0.0+0d7e753227          pypi_0    pypi
[conda] torchvision               0.13.1+cu116             pypi_0    pypi
````

cc @svekars @carljparker @ezyang @soumith
Tensor.uniform_ fails to compile when using torch._dynamo
Labels: triaged, module: initialization, module: dynamo

### 🐛 Describe the bug

Compiler expects `torch.uniform_` method to need a dtype & device arguments.

```python
import torch

def gen_uniform(shape):
    r = torch.empty(shape, dtype=torch.float32)
    return r.uniform_(0, 1)

class Model1(torch.nn.Module):
    def forward(self):
        return gen_uniform([10, 20])

model1 = torch.compile(Model1())
print(model1())
```

```
Exception has occurred: BackendCompilerFailed       (note: full exception trace is shown but execution is paused at: _run_module_as_main)
_compile_fn raised TypeError: uniform() missing 2 required keyword-only arguments: 'dtype' and 'device'

While executing %uniform_ : [#users=1] = call_method[target=uniform_](args = (%empty, 0, 1), kwargs = {})
Original traceback:
Module stack: {}
  File "test.py", line 5, in gen_uniform
    return r.uniform_(0, 1)
 |   File "test.py", line 9, in forward
    return gen_uniform([10, 20])
```

### Versions

```
Collecting environment information...
PyTorch version: 1.14.0 - commit acd68f90970e325caf155cbcfbd30b8a3f3f0b38
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.5 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.25.0
Libc version: glibc-2.31

Python version: 3.8.10 (default, Jun 22 2022, 20:18:18)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.13.0-1023-aws-x86_64-with-glibc2.29
Is CUDA available: True
CUDA runtime version: 11.8
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: 
GPU 0: NVIDIA A100-SXM4-40GB
GPU 1: NVIDIA A100-SXM4-40GB
GPU 2: NVIDIA A100-SXM4-40GB
GPU 3: NVIDIA A100-SXM4-40GB
GPU 4: NVIDIA A100-SXM4-40GB
GPU 5: NVIDIA A100-SXM4-40GB
GPU 6: NVIDIA A100-SXM4-40GB
GPU 7: NVIDIA A100-SXM4-40GB

Nvidia driver version: 520.61.05
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.6.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.17.4
[pip3] pytorch-lightning==1.8.3.post1
[pip3] rotary-embedding-torch==0.1.5
[pip3] torch==1.14.0
[pip3] torchdata==0.5.0
[pip3] torchmetrics==0.11.0
[pip3] torchvision==0.15.0a0+01c11a0
[conda] Could not collect
```

cc @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire
[GradScaler] Inconsistent scale values across different GPUs caused by uneven inputs for AMP DDP training
Labels: triaged, module: amp (automated mixed precision)

### 🐛 Describe the bug

Hello, I want to add [Automatic Mixed Precision (AMP)](https://pytorch.org/docs/master/amp.html?highlight=gradscaler#) training into my DDP training codes.

In my case, different GPUs could have uneven inputs during the training process. Thus I use model.join() to handle this problem and it works.

However, to realize the AMP training, I use the [torch.cuda.amp.GradScaler](https://pytorch.org/docs/master/amp.html#torch.cuda.amp.GradScaler) class. Due to the uneven inputs problem, we get inconsistent scale values after several epochs, which gradually causes a large loss difference and training error. 

For example, 

1. Due to data imbalance, rank 0 could run more batches/iterations using model.join().
![image](https://user-images.githubusercontent.com/23723621/205296877-7ae6b562-b65d-4dee-8e81-bb11aba2ed2f.png)

2. Since growth_interval is same (default: 2000) for rank 0 and 1, after several training epochs, rank 0 and 1 could run different numbers of batchs/iterations, which leads to different growth timestamp of scale. As shown below, at Epoch 17 Iter 2071, they got different scale values.
![image](https://user-images.githubusercontent.com/23723621/205297947-800daa15-5af4-42d6-9d5e-ff956f35c760.png)
![image](https://user-images.githubusercontent.com/23723621/205296993-10019b59-9c7a-4f1e-8870-bdc98ea6cec5.png)

I think this could be a bug of torch.cuda.amp.GradScaler while using uneven inputs for DDP training. 
**There should be some synchronization mechanism to keep the consistency of the scale value (or the growth_tracker) across different GPUs in the case of uneven inputs.**

Thanks a lot if you can help answer my question!


### Versions

pytorch 1.12.1
2 * A100

cc @mcarilli @ptrblck @leslie-fang-intel @jgong5
forward-mode AD formula for torch.add (and possibly others) accidentally upcasts float32 to float64
Labels: module: performance, module: autograd, triaged, actionable, ZeroTensor, module: forward ad, module: functorch

```python
import torch
import functorch

print(functorch.__version__)
# '1.13.0+cu117'

dtype = torch.float32
device = torch.device('cpu')

def foo(x):
    # unpack
    q, p = x
    # add float, ok with adding int
    q, p = q + 1.0, p + 1.0
    return torch.stack([q, p])

x = torch.tensor([0.0, 0.0], dtype=dtype, device=device)

print(functorch.jacfwd(foo)(x))
print()

# tensor([[1., 0.],
#         [0., 1.]], dtype=torch.float64)

print(functorch.jacrev(foo)(x))
print()

# tensor([[1., 0.],
#         [0., 1.]])
```

cc @VitalyFedyunin @ngimel @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7 @Chillee @samdow @soumith
DDP overlapped optimizer: set grads to None enhancements
Labels: oncall: distributed, module: data parallel, module: ddp

### 🚀 The feature, motivation and pitch

https://github.com/pytorch/pytorch/pull/89194 is adding DDP overlapped optimizer with very basic support for setting grads to None, i.e. it will set all DDP managed parameters grads to None. As enhancements, we should:

1. Only set grads to None for the parameters with in backwards optimizers which might not necessarily be all DDP parameters
2. set grads to None as we run the optimizer step, to save memory. Currently peak memory is not reduced because we only set the grads to None at the end of backwards.

There is currently no use case for (1) and DDP reducer current implementation blocks doing (2) effectively, so both of these require follow up discussion.

### Alternatives

_No response_

### Additional context

_No response_

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu
[feature request] Need dtype torch.complex64 support on MPS Device
Labels: feature, triaged, module: mps

If you have a question or would like help and support, please ask at our
[forums](https://discuss.pytorch.org/).

If you are submitting a feature request, please preface the title with [feature request].
If you are submitting a bug report, please fill in the following details.

## Issue description

I need to simulate a script that is using math functions that are complex ( complex64 or cfloat type matrix).

## Code example

myMatrix = torch.tensor(4000, 4000, dtype=torch.cfloat, device='mps')

## System Info

Python version: 3.9.6 (default, Sep 26 2022, 11:37:49)  [Clang 14.0.0 (clang-1400.0.29.202)] (64-bit runtime)
Python platform: macOS-13.0.1-arm64-arm-64bit
Is CUDA available: N/A
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

- PyTorch or Caffe2: PyTorch
- How you installed PyTorch (conda, pip, source): conda
- Build command you used (if compiling from source): pip3 install torch torchvision torchaudio
- OS: Ventura 13.0.1
- PyTorch version: 1.13.0
Is MPS (Metal Performance Shader) built? True
Is MPS available? True

- Python version: Python 3.9.6
- CUDA/cuDNN version: N/A
- GPU models and configuration: Apple GPU 10 Core (MacBook Air M2)
- GCC version (if compiling from source): Apple clang version 14.0.0 (clang-1400.0.29.202)
- CMake version: GNU Make 3.81
- Versions of any other relevant libraries:


cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
TensorWithTFOverrideVariable unwraps too early
Labels: triaged, bug

### 🐛 Describe the bug

TensorWithTFOverrideVariable.call_method unwraps immediately before inlining `__torch_function__`. But the logical unwrapping doesn't truly happen until you make a super() call inside torch function (or you DisableTorchFunction). This means recursive calls to torch function are not handled correctly right now.

### Error logs

_No response_

### Minified repro

_No response_
Can not use x=torch.tensor(b), to create a Tensor out of a List[List[Tensor]] (A List of Lists of Tensors)
Labels: oncall: jit, triaged

### 🐛 Describe the bug

Can not use x=torch.tensor(b), to create a Tensor out of a List[List[Tensor]] (A List of Lists of Tensors) in Torchscript

Minimal Sample:
``` python
@torch.jit.script_if_tracing
def get_boxes(batch_bboxes):
      s = torch.jit.annotate(List[List[Tensor]], [])
      s.append([torch.tensor(3.14)]
      s.append([torch.tensor(42.23)]
      bb = torch.tensor(s)
```
``` bash
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
bb = torch.tensor(b)
                 ~~~~~~~~~~~~ <--- HERE
RuntimeError: Input must be of ints, floats, or bools, got Tensor
```

### Versions

OS: macOS 12.2.1 (x86_64)
GCC version: Could not collect
Clang version: 13.1.6 (clang-1316.0.21.2)
CMake version: version 3.21.3
Libc version: N/A

Python version: 3.8.10 (v3.8.10:3d8993a744, May  3 2021, 08:55:58)  [Clang 6.0 (clang-600.0.57)] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.5
[pip3] torch==1.13.0
[pip3] torchvision==0.14.0
[conda] Could not collect


cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel
Error in Adam.step(): If capturable=True, params and state_steps must be CUDA tensors.
Labels: module: optimizer, triaged, actionable

When the Adam optimizer is initialized by explicitly specifying the `capturable` flag for each parameter group, if the `capturable` flag of some group is different from default/global setting (e.g. the argument `capturable` of Adam's initializer), the error occurs in `Adam.step()`.

Here is a simple test script:

```python
import torch

x = torch.nn.Parameter(torch.empty(1, device="cuda"))
params_groups = [{
    "params": [x],
    "capturable": True
}]
optimizer = torch.optim.Adam(params_groups)
y = x ** 2
optimizer.zero_grad()
y.backward()
optimizer.step()
```

which outputs:

```shell
Traceback (most recent call last):
  File ".../test_adam.py", line 12, in <module>
    optimizer.step()
  File ".../lib/python3.10/site-packages/torch/optim/optimizer.py", line 140, in wrapper
    out = func(*args, **kwargs)
  File ".../lib/python3.10/site-packages/torch/optim/optimizer.py", line 23, in _use_grad
    ret = func(self, *args, **kwargs)
  File ".../lib/python3.10/site-packages/torch/optim/adam.py", line 234, in step
    adam(params_with_grad,
  File ".../lib/python3.10/site-packages/torch/optim/adam.py", line 300, in adam
    func(params,
  File ".../lib/python3.10/site-packages/torch/optim/adam.py", line 348, in _single_tensor_adam
    assert param.is_cuda and step_t.is_cuda, "If capturable=True, params and state_steps must be CUDA tensors."
AssertionError: If capturable=True, params and state_steps must be CUDA tensors.
```

PyTorch version: 1.13.0

-----------------------------------------

I traced the script of Adam and found that the device of state_steps for each parameter is determined by default `capturable` and `fused` settings:

https://github.com/pytorch/pytorch/blob/0e7918b9317d0a481c3747264b3584c37dc4a28f/torch/optim/adam.py#L212-L216

In `_single_tensor_adam()`, it will check the device of state_steps against the `capturable` setting of the corresponding parameter group:

https://github.com/pytorch/pytorch/blob/0e7918b9317d0a481c3747264b3584c37dc4a28f/torch/optim/adam.py#L347-L348

which is called by `step()` as:

https://github.com/pytorch/pytorch/blob/0e7918b9317d0a481c3747264b3584c37dc4a28f/torch/optim/adam.py#L234-L252

It seems that the inconsistency causes the error.

A simple solution is changing the L212-L216 of adam.py to determine the device by group's `capturable` and `fused` settings, but I'm not sure it's logically correct and will not cause other side effects:

```python
 state['step'] = ( 
     torch.zeros((1,), dtype=torch.float, device=p.device) 
     if group['capturable'] or group['fused'] 
     else torch.tensor(0.) 
 ) 
```


cc @vincentqb @jbschlosser @albanD @janeyx99
minified code can not produce fp64_ref result
Labels: triaged

Minified code:
```
import torch._inductor.overrides
import torch
from torch import tensor, device
import torch.fx as fx
from torch._dynamo.testing import rand_strided
from math import inf
from torch.fx.experimental.proxy_tensor import make_fx
# torch version: 1.14.0a0+fb
# torch cuda version: 11.4.0
# CUDA Info: 
# nvcc not found
# GPU Hardware Info: 
# NVIDIA A100-PG509-200 : 8 
from torch.nn import *
class Repro(torch.nn.Module):
    def __init__(self):
        super().__init__()


    def forward(self, arg147_1, arg149_1, arg151_1, arg153_1, add_37, permute_62, permute_63, permute_64, permute_65):
        convert_element_type_11 = torch.ops.prims.convert_element_type.default(add_37, torch.float32);  add_37 = None
        addmm_60 = torch.ops.aten.addmm.default(arg147_1, convert_element_type_11, permute_62);  arg147_1 = permute_62 = None
        addmm_61 = torch.ops.aten.addmm.default(arg149_1, addmm_60, permute_63);  arg149_1 = addmm_60 = permute_63 = None
        mul_54 = torch.ops.aten.mul.Tensor(convert_element_type_11, addmm_61);  addmm_61 = None
        add_38 = torch.ops.aten.add.Tensor(mul_54, convert_element_type_11);  mul_54 = convert_element_type_11 = None
        addmm_62 = torch.ops.aten.addmm.default(arg151_1, add_38, permute_64);  arg151_1 = add_38 = permute_64 = None
        addmm_63 = torch.ops.aten.addmm.default(arg153_1, addmm_62, permute_65);  arg153_1 = addmm_62 = permute_65 = None
        return (addmm_63,)

args = [((256,), (1,), torch.float32, 'cuda'), ((7616,), (1,), torch.float32, 'cuda'), ((256,), (1,), torch.float32, 'cuda'), ((7616,), (1,), torch.float32, 'cuda'), ((8, 7616), (7616, 1), torch.float32, 'cuda'), ((7616, 256), (1, 7616), torch.float32, 'cuda'), ((256, 7616), (1, 256), torch.float32, 'cuda'), ((7616, 256), (1, 7616), torch.float32, 'cuda'), ((256, 7616), (1, 256), torch.float32, 'cuda')]
args = [rand_strided(sh, st, dt, dev) for (sh, st, dt, dev) in args]
mod = make_fx(Repro().to(device="cuda"))(*args)
from torch._inductor.compile_fx import compile_fx_inner
from torch._dynamo.debug_utils import same_two_models
compiled = compile_fx_inner(mod, args)
assert same_two_models(mod, compiled, args, only_fwd=True), "Accuracy failed"                                                                            
```

Warning:
```
[WARNING] Could not generate fp64 outputs
```

The problem is likely coming from `torch.ops.prims.convert_element_type.default(add_37, torch.float32)` and it is not a bug, but not being able to compare with `fp64_ref` sometimes can lead minifier to a wrong direction when minifying accuracy errors.
Calling item() on symbolic shape fake tensor should give more clear error message
Labels: triaged, module: inductor

### 🐛 Describe the bug

Right now it says `RuntimeError: Cannot call numel() on tensor with symbolic sizes/strides` which is bad

### Versions

master

cc @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire
Random K compression hook in PyTorch DDP
Labels: feature, triaged, module: ddp

### 🚀 The feature, motivation and pitch

Currently Pytorch DDP supports a few default compression hooks, namely fp16, bf16, and PowerSGD. However, there are a myraid of other compression types that exist in literature with various degrees of success. Often in literature these methods are rewritten by each research team leading to a lot of duplicate work, increased difficulty of reproducibility, and also increasing the barrier for others to experiment with the same techniques.

A common and simple compression method is Random-K, which all reduces only some percent (to be determined by the user) of tensor elements. We have implemented Random-K as a DDP hook and would like to contribute it back to PyTorch so there can be a standard, reusable Random-K compression method available to all. Our implementation is based upon the implementation in [GRACE](https://github.com/sands-lab/grace).

[EDIT]: We are also interested in adding other compression hooks, such as Top-K and [Deep Gradient Compression](https://github.com/synxlin/deep-gradient-compression) to the DDP library if the maintainers are interested.

Let me know what you think!

Best,
Harrison

### Alternatives

_No response_

### Additional context

_No response_
nn.CrossEntropy/nn.NLLLoss : Request for option to specify invalid ignore_index for perf. optimization
Labels: module: loss, triaged

### 🚀 The feature, motivation and pitch

nn.CrossEentropy and nn.NLLLoss functions have the argument "ignore_index" to specify the target index to be ignored during loss calculation
Eg: Pls ref:
https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html 

The default value is -100 and as per the definition at the above link, any value (even outside the usual class range) can be specified as ignore_index value.
And there is no option to specify that ingore_index value is not valid.( i.e, All elements of target are to be considered for loss calculation)
In fact some models seem to intentionally  set targets to be ignored as -100 since default ignore_index is -100.
(https://github.com/huggingface/transformers/blob/main/src/transformers/models/layoutxlm/tokenization_layoutxlm.py)

Looking for whether the target tensor contains any element with ignore_index value can have computational overhead especially with large target tensors.
But there is no target to be ignored in the case of many models.
So the current API results in unnecessary overhead in many cases.

Can PyTorch provide an option to indicate the backends whether the inore_index argument is invalid
(so that sifting though the target tensor for ignore_index can be avoided)
Say, with an API update with a "specific" ingore_index value or an additional argument?

This can improve the training times if going forward backends and training scripts can adopt this change.

When the default value was set as -100, was it intended to be used as as an invalid_ignore index?
Can models use -100 as invalid ignore_index? If yes, may be just a document update is enough from Pytorch side.
And hopefully existing models can make corresponding change when upgrading for a Pytorch version with this specification.

### Alternatives

_No response_

### Additional context

_No response_
[Dynamo] Examples that recompile beyond cache size limit  
Labels: triaged

Collecting small repros that demonstrate cache size limit violation.

**Case 1** - From `vision_maskrcnn`
**Solution** - Could be solved by Unspec work
~~~
import torch
import torch._dynamo as dynamo
import logging


dynamo.config.cache_size_limit = 2
dynamo.config.log_level = logging.DEBUG
dynamo.config.print_graph_breaks = True

def fn(box):
    w = int(box[2] + box[0])
    h = int(box[3] - box[1])
    return w + h


torch.manual_seed(0)
args = [torch.randint(0, 10000, size=(4,), dtype=torch.int64) for _ in range(10)]
opt_fn = dynamo.optimize("eager")(fn)
res = [opt_fn(arg) for arg in args]
print(res)
~~~

**Case 2** - From `tactotron2`
There is a temporary fix - https://github.com/pytorch/pytorch/pull/88857 that skips the frame entirely, but it might be suboptimal

~~~
        def inner(x):
            return torch.sin(x)

        def fn(x):
            for _ in range(100):
                inner(x)
                torch._dynamo.graph_break()
            return x
~~~

**Case 3** - Just trying on my own
**Solution** - Could be solved by Unspec work, however 1 to 5 are common numbers. If I change the n from 1 to 5, to 200 to 205, unspec would work.
~~~
import torch
import torch._dynamo as dynamo
import logging


dynamo.config.cache_size_limit = 2
dynamo.config.log_level = logging.DEBUG
dynamo.config.print_graph_breaks = True

def fn(x, n):
    return x + n

x = torch.randn(4)
opt_fn = dynamo.optimize("eager")(fn)
for n in range(5):
    opt_fn(x, n)
~~~
Way to run accuracy minifier on only one particular subgraph
Labels: triaged, bug

### 🐛 Describe the bug

Accuracy minifier may get stuck on an early subgraph. In this case, it may be helpful to ask the minifier to not run on those subgraphs, and only run on the one that you suspect is problematic (or you can do some sort of ablation and find all the subgraphs that are failing accuracy, so you can then discard the ones that are just because the accuracy minifier doesn't work.)

### Error logs

_No response_

### Minified repro

_No response_
Wrong output type hint for `F.one_hot`
Labels: module: typing, triaged

### 🐛 Describe the bug

Type hint for `torch.nn.functional.one_hot` seems not to work properly.

```python
# one_hot_type.py
import torch
import torch.nn.functional as F

x = torch.tensor([0, 1, 2]).long()
reveal_type(F.one_hot(x))
```

```shell
$ mypy one_hot_type.py
one_hot_type.py:5: note: Revealed type is "Any"
Success: no issues found in 1 source file
```

Expected Tensor, got Any.

### Versions

PyTorch version: 1.14.0.dev20221117
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.0.1 (x86_64)
GCC version: Could not collect
Clang version: 15.0.3
CMake version: version 3.24.2
Libc version: N/A

Python version: 3.9.13 (main, May 24 2022, 21:28:31)  [Clang 13.1.6 (clang-1316.0.21.2)] (64-bit runtime)
Python platform: macOS-13.0.1-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.982
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.3
[pip3] torch==1.14.0.dev20221117
[pip3] torch-tb-profiler==0.4.0
[pip3] torchaudio==0.14.0.dev20221117
[pip3] torchvision==0.15.0.dev20221117
[conda] Could not collect

cc @ezyang @malfet @rgommers @xuzhao9 @gramster
update transformer init function
Labels: oncall: transformer/mha

### 🚀 The feature, motivation and pitch

Once a transformer instance is created, like below:
`
model_pytorch = nn.Transformer(nhead=16, num_encoder_layers=2, num_decoder_layers=2)
`
I cannot access it's num_encoder_layers by **model_pytorch.num_encoder_layers**, TransformerEncoder, TransformerDecoder, TransformerEncoderLayer, TransformerDecoderLayer alike.  
This is extremely suffering when doing quantization, like below:
`
Transformer.from_torch(model_pytorch)
`
say I have a self-defined Transformer class, it's really hard to create my instance from original pytorch instance, because I cannot  access init value elegantly.

### Alternatives

_No response_

### Additional context

_No response_

cc @jbschlosser @bhosmer @cpuhrsch @erichan1
The current example for `torch.mode` is IMHO confusing and has room for improvement.
Labels: module: docs, triaged

### 📚 The issue

I believe that example for any torch operator should be in such a way that it is obvious what to see what the operator does. This is more important for an operator that is as simple as `torch.mode`. The current example does not show the tensor and it is constructed in such a way that all the modes occur together.  So, the indices returned are all the same and leads to confusion (e.g. [a post on StackOverflow](https://stackoverflow.com/questions/74608692/how-yo-get-the-torch-mode-s-output-indeces-can-details/74608850#74608850)).

### Potential alternative

Instead I suggest something like this:
```python
>>> b = torch.randint(4, (5, 7))
>>> b
tensor([[0, 0, 0, 2, 0, 0, 2],
        [0, 3, 0, 0, 2, 0, 1],
        [2, 2, 2, 0, 0, 0, 3],
        [2, 2, 3, 0, 1, 1, 0],
        [1, 1, 0, 0, 2, 0, 2]])
>>> torch.mode(b, 0)
torch.return_types.mode(
values=tensor([0, 2, 0, 0, 0, 0, 2]),
indices=tensor([1, 3, 4, 4, 2, 4, 4]))
```

Users can see the tensor, it is clear what the modes are and where they occur.

cc @svekars @carljparker
InvokeAI using MPS is broken by torch nightlies since torch-1.14.0.dev20221104 inclusive 
Labels: triaged, module: correctness (silent), module: mps

### 🐛 Describe the bug

I'm sorry this  is going to be very generic, but I've been occasionally testing InvokeAI, a Stable Diffusion front end with the pytorch nightlies to see if the speed lost since 13.0 has returned. 
Today is the first time I've tested in a while and it was broken, generating colour noise not an image,
so I've been going back through the nightlies to see where the breakage is.

torch-1.14.0.dev20221102 works, 
torch-1.14.0.dev20221103 works if https://github.com/pytorch/pytorch/pull/87974 is reverted
torch-1.14.0.dev20221104 is broken if https://github.com/pytorch/pytorch/pull/87974 is reverted
torch-1.14.0.dev20221105 is broken
 
 That in theory narrows it down to https://github.com/pytorch/pytorch/pull/82809 and https://github.com/pytorch/pytorch/pull/88319 if its a MPS only issue

### Versions

PyTorch version: 1.14.0.dev20221103
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.0.1 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.202)
CMake version: version 3.25.0
Libc version: N/A

Python version: 3.10.8 (main, Nov  9 2022, 16:53:27) [Clang 12.0.5 (clang-1205.0.22.9)] (64-bit runtime)
Python platform: macOS-13.0.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.4
[pip3] pytorch-lightning==1.7.7
[pip3] torch==1.14.0.dev20221103
[pip3] torch-fidelity==0.3.0
[pip3] torchdiffeq==0.2.3
[pip3] torchmetrics==0.10.0
[pip3] torchsde==0.2.5
[pip3] torchvision==0.15.0.dev20221128
[conda] Could not collect


cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
addcmul on CUDA does not have the correct FMA behavior
Labels: module: numerical-stability, module: cuda, triaged

### 🐛 Describe the bug

To reproduce the result:

``` python
import torch

x = torch.randn(1000)
y = torch.randn(1000)

result = x*y
diff = torch.addcmul(-result, x, y)
print("diff_cpu:", diff.abs().max().item())

x = x.cuda()
y = y.cuda()

result = x*y
diff = torch.addcmul(-result, x, y)
print("diff_cuda:", diff.abs().max().item())

# obtaining reference using double precision
xDouble = x.double()
yDouble = y.double()
diffDouble = xDouble*yDouble - result
print("diff_double:", diffDouble.abs().max().item())

```

Outputs:
``` 
diff_cpu: 2.250976649520453e-07
diff_cuda: 0.0
diff_double: 2.250976649520453e-07
```

### Versions

Pytorch installed via PIP

Outputs from collect_env.py:

Collecting environment information...
PyTorch version: 1.13.0+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: Could not collect
CMake version: version 3.22.1
Libc version: glibc-2.35

Python version: 3.10.6 (main, Nov  2 2022, 18:53:38) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.15.0-53-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: 
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090

Nvidia driver version: 520.61.05
cuDNN version: Probably one of the following:
/usr/local/cuda-9.1/targets/x86_64-linux/lib/libcudnn.so.6
/usr/local/cuda-9.1/targets/x86_64-linux/lib/libcudnn.so.7
/usr/local/lib/libcudnn.so.6
/usr/local/lib/libcudnn.so.7
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] memory-efficient-attention-pytorch==0.0.27
[pip3] numpy==1.23.4
[pip3] torch==1.13.0
[pip3] torch-tb-profiler==0.4.0
[pip3] torchaudio==0.13.0
[pip3] torchvision==0.14.0
[conda] Could not collect


cc @ngimel
DISABLED test_hf_bert_ddp_inductor (__main__.TestFakeDistributedSingleProc)
Labels: triaged, skipped, module: inductor

Platforms: linux

This test was disabled because it is failing on master ([recent examples](http://torch-ci.com/failure/test_hf_bert_ddp_inductor)).

This is a test on inductor that started failing over Thanksgiving break

cc @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire
third-order gradient of torch.pow with tensor args and certain input returns NaN
Labels: module: autograd, triaged, module: NaNs and Infs

Hi, `torch.pow` with two tensor arguments results in wrong higher order derivatives.

```python

# functorch.__version__
# '1.13.0+cu117'

import torch
import functorch

jacobian = functorch.jacrev

x = torch.tensor([0.0, 0.0], dtype=torch.float64)
y = torch.tensor([0.0, 0.0], dtype=torch.float64)

def foo(x, y):
    x1, x2 = x
    y1, y2 = y
    return torch.stack([x1*x2*y1*y2])

def bar1(x, y):
    power = torch.tensor([1, 1, 1, 1], dtype=torch.int64)
    state = torch.cat([x, y])
    state = torch.pow(state, power)
    return torch.stack([state.prod(-1)])

def bar2(x, y):
    power = [1, 1, 1, 1]
    # power = torch.tensor([1, 1, 1, 1], dtype=torch.int64) # -- will also break
    state = torch.cat([x, y])
    state = torch.stack([s**p for s, p in zip(state, power)])
    return torch.stack([state.prod(-1)])

print(foo(x, y).numpy())
print(bar1(x, y).numpy())
print(bar2(x, y).numpy())

# d^2(d^2(f)/d(y^2))/d(x^2)

print(jacobian(jacobian(lambda x: jacobian(jacobian(lambda y: foo(x, y)))(y)))(x).flatten().numpy())
print(jacobian(jacobian(lambda x: jacobian(jacobian(lambda y: bar1(x, y)))(y)))(x).flatten().numpy())
print(jacobian(jacobian(lambda x: jacobian(jacobian(lambda y: bar2(x, y)))(y)))(x).flatten().numpy())

# jacobian = functorch.jacrev
# [0.]
# [0.]
# [0.]
# [0. 0. 0. 0. 0. 1. 1. 0. 0. 1. 1. 0. 0. 0. 0. 0.]
# [nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan]
# [0. 0. 0. 0. 0. 1. 1. 0. 0. 1. 1. 0. 0. 0. 0. 0.]

# jacobian = functorch.jacfwd
# [0.]
# [0.]
# [0.]
# [0. 0. 0. 0. 0. 1. 1. 0. 0. 1. 1. 0. 0. 0. 0. 0.]
# [0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
# [0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]

```

cc @ezyang @gchanan @zou3519 @albanD @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7 @Chillee @samdow @soumith
[MPS] Add support for aten::repeat_interleave.self_Tensor for MPS backend
Labels: triaged, module: mps

### 🚀 The feature, motivation and pitch


detectron2/modeling/poolers.py:66: UserWarning: The operator 'aten::repeat_interleave.self_Tensor' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications. (Triggered internally at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1666646703877/work/aten/src/ATen/mps/MPSFallback.mm:11.)
  indices = torch.repeat_interleave(

### Alternatives

_No response_

### Additional context

_No response_

cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
torch.addbmm throws different exception differences on CPU and GPU.
Labels: module: cuda, module: error checking, triaged

### 🐛 Describe the bug

torch.addbmm runs under the pytorch1.8 version. The exceptions thrown by the test code on the CPU and GPU are very different. 

 Test on the CPU:
          
    import torch
    input = torch.rand([5], dtype=torch.float64)
    batch1 = torch.randint(-128, 1024, [10, 3, 4], dtype=torch.int64)
    batch2 = torch.randint(-64, 2, [10, 4, 5], dtype=torch.int8)
    out = torch.addbmm(input, batch1, batch2)
          
result: RuntimeError: expected scalar type Double but found Long
          
Test on the GPU:
          
    import torch
    input = torch.rand([5], dtype=torch.float64).cuda()
    batch1 = torch.randint(-128, 1024, [10, 3, 4], dtype=torch.int64).cuda()
    batch2 = torch.randint(-64, 2, [10, 4, 5], dtype=torch.int8).cuda()
    out = torch.addbmm(input, batch1, batch2)
          
result: RuntimeError: expected scalar type Double but found Char

torch.baddbmm has the same problem.

### Versions

pytorch: 1.8
Python version: 3.8
CUDA/cuDNN version: cuDNN 11.3
GPU models and configuration: RTX3060
Operating System：Windows

cc @ngimel
Sample Weighted BatchNorm1d
Labels: feature, module: nn, triaged, needs research, module: norms and normalization

### 🚀 The feature, motivation and pitch

I recently worked on a project involving graph neural networks where batch normalization with weighted samples (nodes) helped improve prediction performance. This was especially useful in my use case where graphs could vary quite significantly in size and weights reduce influence of larger graphs (those having more nodes) within the batch. Weights can also be used to compress graphs, improving performance. I don't believe there is a way currently to pass sample weights to the batch norm layer, so I wrote my own. This version of course, doesn't have the performance of the original BatchNorm1d layer. I was wondering whether it would be worthwhile for me to implement this directly. 

### Alternatives

The only feasible alternative I could come up with was repeating samples such that the behavior was replicated through the current BatchNorm1d i.e. if a sample contained [0, 1] with respective weights [1/3, 2/3] the sample could be expanded to [0, 1, 1]. Though, this is obviously inefficient.

### Additional context

There is a weighted form of Welford's algorithm: [https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Weighted_incremental_algorithm](url), so performance can likely be retained (though this will of course be tested).

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
`torch.Tensor.flatten` Trigger Segmentation Fault when trying to provide and output named dim
Labels: module: crash, triaged, module: named tensor

## 🐛 Describe the bug

A test for `torch.Tensor.flatten` triggers segmentation fault when named dim is provided.

### Test
```python
import torch
def test():
    tensor = torch.rand([2, 3, 5, 7, 11], dtype=torch.float32)
    _ = torch.Tensor.flatten(tensor, 2, 9, 'features')

test()
```

### Error log
`segmentation fault`

## Versions

```
PyTorch version: 1.14.0a0+gitbdc9911
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: 11.1.0-6
CMake version: version 3.22.1
Libc version: glibc-2.35

Python version: 3.10.6 (main, Nov  2 2022, 18:53:38) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.15.0-52-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.6.124
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: 
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090
GPU 2: NVIDIA GeForce RTX 3090

Nvidia driver version: 515.65.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.4.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.5
[pip3] torch==1.14.0a0+gitbdc9911
[pip3] torchvision==0.13.1
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               11.3.1               h2bc3f7f_2  
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py39h7f8727e_0  
[conda] mkl_fft                   1.3.1            py39hd3c417c_0  
[conda] mkl_random                1.2.2            py39h51133e4_0  
[conda] numpy                     1.21.5           py39he7a7128_1  
[conda] numpy-base                1.21.5           py39hf524024_1  
[conda] numpydoc                  1.2                pyhd3eb1b0_0  
[conda] torch                     1.14.0a0+gitce2f870          pypi_0    pypi
```

cc @ezyang @gchanan @zou3519
Segfault on torch.nn.functional.one_hot with large tensor on Python 3.9
Labels: needs reproduction, module: crash, triaged

### 🐛 Describe the bug

Running `torch.nn.functional.one_hot(...)` on a large tensor (like 10,000 elements) segfaults:

```
import torch
t = torch.randint(0, 6, size=(10000, 1))
torch.nn.functional.one_hot(t)
```

Results in a segfault (no stack trace)
```
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
```

### Versions

Collecting environment information...
PyTorch version: 1.13.0
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: macOS 10.15.7 (x86_64)
GCC version: Could not collect
Clang version: 12.0.0 (clang-1200.0.32.29)
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.13 (main, Aug 25 2022, 18:29:29)  [Clang 12.0.0 ] (64-bit runtime)
Python platform: macOS-10.15.7-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: Could not collect
Nvidia driver version: 1.1.0
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.13.0
[pip3] torchaudio==0.13.0
[pip3] torchvision==0.14.0
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.4.0           hecd8cb5_637  
[conda] mkl-service               2.4.0            py39h9ed2024_0  
[conda] mkl_fft                   1.3.1            py39h4ab4a9b_0  
[conda] mkl_random                1.2.2            py39hb2f4e1b_0  
[conda] numpy                     1.23.3                   pypi_0    pypi
[conda] numpy-base                1.23.1           py39h3b1a694_0  
[conda] torch                     1.13.0                   pypi_0    pypi
[conda] torchaudio                0.13.0                   pypi_0    pypi
[conda] torchvision               0.14.0                   pypi_0    pypi

cc @ezyang @gchanan @zou3519