gopikrsmscs/torch-issues huggingface dataset:
Minimal example for torch.optim.SparseAdam
Labels: module: sparse, module: docs, triaged

### 📚 The doc issue

Hey, 
it is a bit confusing for me and my group for which cases [SparseAdam](https://pytorch.org/docs/stable/generated/torch.optim.SparseAdam.html) can be used: 

There is a lone saying
```
Implements lazy version of Adam algorithm suitable for sparse tensors.
```
but it is unclear wether the parameters are allowed to be sparse (With sparse parameters, it throws the error 
```
ValueError: Sparse params at indices [0]: SparseAdam requires dense parameter tensors
```

Could you provide more details and a minimal example? 
Thank you

### Suggest a potential alternative/fix

Give a minimal example of how to use this optimizer.

cc @nikitaved @pearu @cpuhrsch @amjames @bhosmer @svekars @holly1238
`tensordot` not working for dtype int32 and lower when there is only 1 element in the given axis
Labels: triaged, module: linear algebra, actionable, bug

### 🐛 Describe the bug

`tensordot` seems to be failing when there is only one element in the given axis and dtype is `int32` or lower. This happens when providing explicit lists of dimensions for `a` and `b`. 

```
import torch

x = torch.randint(1, 10, ([1, 2, 3]), dtype=torch.int32)
axis = 0
torch.tensordot(x, x, dims=([axis], [axis]))
```

with the following error:

`RuntimeError: expected scalar type Long but found Int`

But it works completely fine with `int64` and float dtypes. It also works fine for `int32` but only when there are multiple elements in the given axis. For instance, the following works fine:
```
x = torch.randint(1, 10, ([2, 2, 3]), dtype=torch.int32)
axis = 0
torch.tensordot(x, x, dims=([axis], [axis]))
```


### Versions

```
PyTorch version: 1.12.1+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
CMake version: version 3.22.6
Libc version: glibc-2.26

Python version: 3.7.13 (default, Apr 24 2022, 01:04:09)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
Is CUDA available: True
CUDA runtime version: 11.1.105
GPU models and configuration: GPU 0: Tesla T4
Nvidia driver version: 460.32.03
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.0.5
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.6
[pip3] torch==1.12.1+cu113
[pip3] torchaudio==0.12.1+cu113
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.13.1
[pip3] torchvision==0.13.1+cu113
[conda] Could not collect
```

cc @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano
test_prims.py:test_nvfuser_no_args_cuda, memory leak
Labels: triaged, module: primTorch

### 🐛 Describe the bug

```py
2022-09-02T16:37:13.3046215Z ERROR [0.094s]: test_nvfuser_no_args_cuda (__main__.TestPrimsCUDA)
2022-09-02T16:37:13.3046792Z ----------------------------------------------------------------------
2022-09-02T16:37:13.3047015Z Traceback (most recent call last):
2022-09-02T16:37:13.3047567Z   File "/opt/conda/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1940, in wrapper
2022-09-02T16:37:13.3048068Z     method(*args, **kwargs)
2022-09-02T16:37:13.3048458Z   File "/opt/conda/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1939, in wrapper
2022-09-02T16:37:13.3049178Z     with policy():
2022-09-02T16:37:13.3049613Z   File "/opt/conda/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1530, in __exit__
2022-09-02T16:37:13.3050013Z     raise RuntimeError(msg)
2022-09-02T16:37:13.3050584Z RuntimeError: CUDA driver API confirmed a leak in __main__.TestPrimsCUDA.test_nvfuser_no_args_cuda! Caching allocator allocated memory was 2048 and is now reported as 2560 on device 0. CUDA driver allocated memory was 520880128 and is now 522977280.
```

### Versions

The test is added in https://github.com/pytorch/pytorch/pull/84416.

cc @ezyang @mruberry @ngimel
nn.Softmax should not allow default/implicit/unset dim constructor argument
Labels: module: nn, triaged, needs research, module: deprecation

### 🐛 Describe the bug

Originally discussed in https://github.com/pytorch/pytorch/issues/84290#issuecomment-1232133690, more links to related comments / issues in this cited comment

The F.softmax / softmin / log_softmax have implicit dim support deprecated since a few years ago, so nn.Softmax and friends nn.LogSoftmax / nn.Softmin should also remove this support for reducing possible confusion

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are @alband @jerryzh168 

### Versions

N/A
Issue with MPS ops lead to make_grid broken with mps device Tensors, whole grid is the 'first' image
Labels: triaged, module: mps

### 🐛 Describe the bug

This is a upstreaming of the following bug in torch vision that I've been asked to dos due to the lack of access to MPS hardware.

https://github.com/pytorch/vision/issues/6533

### 🐛 Describe the bug

When calling make_grid on Tensor and List of Tensors where the device in 'mps'  it returns a grid where all the Tensors are the same.

Simple example
create a venv and activate
cd into the venv directory
install the nightlies
pip3 install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cpu

unzip the attached zip file, it contains a python script and two images.
validate the python script and run it.

```
from PIL import Image
from torchvision.utils import make_grid
import torchvision.transforms as transforms
import torch
 

img1 = Image.open('00035.png')
img2 = Image.open('00036.png')

transform = transforms.Compose([
    transforms.PILToTensor()
])


t_img1 = transform(img1).to(torch.device("mps"), dtype=torch.float32)  / 256.0
t_img2 = transform(img2).to(torch.device("mps"), dtype=torch.float32)  / 256.0

grid = make_grid([t_img1, t_img2], nrow=1)

gridImage =  transforms.ToPILImage()(grid.cpu());

gridImage.save('mps_grid.png')

t_img1 = transform(img1).to(torch.device("cpu"), dtype=torch.float32)  / 256.0
t_img2 = transform(img2).to(torch.device("cpu"), dtype=torch.float32)  / 256.0

grid = make_grid([t_img1, t_img2], nrow=1)

gridImage =  transforms.ToPILImage()(grid.cpu());

gridImage.save('cpu_grid.png')
```

It generate two images, you'll see the cpu_grid image are the two images vertically stacked, the mps_grid image has the same image vertically stacked twice.

[make_grid_bug.zip](https://github.com/pytorch/vision/files/9473078/make_grid_bug.zip)


### Versions

PyTorch version: 1.13.0.dev20220901
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.5.1 (arm64)
GCC version: Could not collect
Clang version: 13.1.6 (clang-1316.0.21.2.5)
CMake version: version 3.22.4
Libc version: N/A

Python version: 3.10.4 (main, May 10 2022, 03:52:14) [Clang 13.0.0 (clang-1300.0.29.30)] (64-bit runtime)
Python platform: macOS-12.5.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.2
[pip3] torch==1.13.0.dev20220901
[pip3] torchaudio==0.13.0.dev20220901
[pip3] torchvision==0.14.0.dev20220901
[conda] Could not collect


cc @kulinseth @albanD
MPS backend appears to be limited to 32 bits
Labels: triaged, module: mps

### 🐛 Describe the bug

Create a large job for MPS to work on and we hit an inbuilt limit that bombs out with...

/AppleInternal/Library/BuildRoots/20d6c351-ee94-11ec-bcaf-7247572f23b4/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:705: failed assertion `[MPSTemporaryNDArray initWithDevice:descriptor:] Error: product of dimension sizes > 2**31'

This only occurs on MPS, I can easily run larger jobs on Cuda

### Versions

Not overly helpful...

Collecting environment information...
Traceback (most recent call last):
  File "/Users/ec2-user/Library/Lartis/collect_env.py", line 492, in <module>
    main()
  File "/Users/ec2-user/Library/Lartis/collect_env.py", line 475, in main
    output = get_pretty_env_info()
  File "/Users/ec2-user/Library/Lartis/collect_env.py", line 470, in get_pretty_env_info
    return pretty_str(get_env_info())
  File "/Users/ec2-user/Library/Lartis/collect_env.py", line 319, in get_env_info
    pip_version, pip_list_output = get_pip_packages(run_lambda)
  File "/Users/ec2-user/Library/Lartis/collect_env.py", line 301, in get_pip_packages
    out = run_with_pip(sys.executable + ' -mpip')
  File "/Users/ec2-user/Library/Lartis/collect_env.py", line 289, in run_with_pip
    for line in out.splitlines()
AttributeError: 'NoneType' object has no attribute 'splitlines'

cc @kulinseth @albanD
Torch.FX work with autograd.Function
Labels: triaged, module: fx, fx

### 🚀 The feature, motivation and pitch

Dear fx guys,

Making Torch.FX work with `autograd.Function` is important when transform or fuse custom implemented operators, there's someone else meet the same problem at https://discuss.pytorch.org/t/how-can-torch-fx-work-with-autograd-function/145922.

For now, Torch.JIT can trace `autograd.Function` like this: `%x : Float(a, b, c, strides=[a, b, c], requires_grad=1, device=cpu) = ^CustomFunction()(...)`.
While Torch.FX trying to call the custom function with proxied `fx.Proxy` parameters, that causes errors.


### Alternatives

_No response_

### Additional context

_No response_

cc @ezyang @SherlockNoMad @soumith
functionalize: Does not compose cleanly with torch.jit.script/torch.jit.trace
Labels: oncall: jit, module: functionalization

### 🐛 Describe the bug

```python

import torch
from functorch.experimental import functionalize

class Net(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.fc0 = torch.nn.Linear(12, 17)

    def forward(self, x):
        return self.fc0(x).relu_()

net = Net()

input = torch.randn([1, 12])
# These work.
torch.jit.script(net)
torch.jit.trace(net, input)
# Both of these fail with different errors (see below).
#torch.jit.script(functionalize(net))
#torch.jit.trace(functionalize(net), input)
```

For script the error is:
```
TypeError: module, class, method, function, traceback, frame, or code object was expected, got Net
```
([full traceback](https://gist.github.com/silvasean/6cfa2510a010c4f8b07721b96caa4509))


For trace the error is:
```
RuntimeError: Cannot insert a Tensor that requires grad as a constant. Consider making it a parameter or input, or detaching the gradient
```
([full traceback](https://gist.github.com/silvasean/833df0af01bdf489529713d4684e30ca))



### Versions

torch                   1.13.0.dev20220830+cpu

cc @bdhirsh @ezyang @soumith
Embedding scale_grad_by_freq should probably shrink by sqrt(count)
Labels: triaged, module: embedding, oncall: pt2

Given the analysis and experiments in "Frequency-aware SGD for Efficient Embedding Learning with Provable Benefits" (https://openreview.net/pdf?id=ibqTBNfJmi), it would seem that it would be better for `scale_grad_by_freq` to divide by sqrt(count) rather than count.

https://github.com/pytorch/pytorch/blob/0a07488ed2c47765e337e290bd138c0e6e459cbd/aten/src/ATen/native/Embedding.cpp#L133

Also, what is the reason for initializing the counter to zeros, then incrementing by 1 [here](https://github.com/pytorch/pytorch/blob/0a07488ed2c47765e337e290bd138c0e6e459cbd/aten/src/ATen/native/Embedding.cpp#L120), rather than initializing to 1?



cc @ezyang @soumith
A little improvement to torch.nn.ReflectionPad2d
Labels: triaged, module: padding, oncall: pt2

### 🚀 The feature, motivation and pitch

A little improvement, so little that I think it does not deserve to post about my intended feature and later discuss the design and implementation. Probably you can consider this little extension directly.

torch.nn.ReflectionPad2d is limited to pad to a new size less than or equal to the double of the original size.

I want to propose a new MultipleReflectionPad2d without this limitation.

```
class MultipleReflectionPad2d():
    def __init__(self, out_height, out_width):
        self.out_height = out_height
        self.out_width  = out_width
    def __call__(self, image):
        height, width = image.shape[-2:]
        while height<self.out_height or width<self.out_width:
            new_height = self.out_height
            new_width  = self.out_width
            if new_height > 2*height : new_height = 2*height
            if new_width  > 2*width  : new_width  = 2*width
            padding_top    = (new_height-height)//2
            padding_bottom = new_height-height-padding_top
            padding_left   = (new_width-width)//2
            padding_right  = new_width-width-padding_left
            padder = torch.nn.ReflectionPad2d((padding_left, padding_right, padding_top, padding_bottom))
            image  = padder(image)
            height, width = image.shape[-2:]
        return image
```

### Alternatives

May be you can rewrite ReflectionPad2d without this limitation


### Additional context

_No response_

cc @ezyang @soumith
Install LibTorch by Conan or other C++ package manager
Labels: module: cpp, triaged, topic: binaries

### 🚀 The feature, motivation and pitch

At present, There is no detailed guide for how to install LibTorch or construct the environment. The official tutorial does not show where should I put the downloaded library, or how to write a correct `CMakeList.txt` file.

### Alternatives

I find out that there are some good C++ package managers like [Conan](https://conan.io). It is kind of like `pip` and `conda` for Python. It would be easier to install and use LibTorch if we can access it by a C++ package manager, say [Conan](https://conan.io).

### Additional context

_No response_

cc @jbschlosser
[c10d] Support a public API to retrieve default process group
Labels: oncall: distributed, feature, triaged, pt_distributed_rampup

### 🚀 The feature, motivation and pitch

- We should provide a public API to retrieve the default process group, as per the comment in https://github.com/pytorch/pytorch/pull/84105#issuecomment-1230519909, there is no documented way to get this, and as a result it makes it hard to use APIs that require a process group to be passed in.

Currently, we have `_get_default_group` as a private API, we should consider whether it is okay to make this a public API and if we can provide BC guarantees especially given the work around dispatchable collectives.

### Alternatives

_No response_

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501
Strange cuda illegal memory allocation error
Labels: module: cuda, triaged

### 🐛 Describe the bug

I am trying to run the following GitHub repository.

https://github.com/neuroailab/EISEN

 However, I am receiving a strange Cuda illegal memory allocation error. 
![image](https://user-images.githubusercontent.com/52120948/188027404-12098a5c-bcc2-4af9-ba7b-ceaaa01ab117.png)
It does not seem to be the code since the repo author states that the code is running fine on his machine. It will be great if someone can help to verify if this is specific to my machine or not. Any idea if this is an issue with my Cuda/torch version? I have tried to debug this but was not successful. I did find that the error seems to come from this line: `indices = sample_inds[-1].view(B, T * K).unsqueeze(-1).expand(-1, -1, D)` from core/utils/utils.py line 97.

When trying to print the max value of the indices variable, it also throws the cuda illegal memory allocation error. The error is gone when I remove the `.expand(-1, -1, D)`, but it leads to a dimension issue further down the code.

The following is my environment.
![image](https://user-images.githubusercontent.com/52120948/188026199-9b1d8661-0709-44df-bb0e-0b505158d586.png)
I am also running this on wsl2 with the following nvidia-smi output:
![image](https://user-images.githubusercontent.com/52120948/188027473-74c14413-b74b-473d-b599-72d4052aa1c8.png)


### Versions

![image](https://user-images.githubusercontent.com/52120948/188027978-e81a3420-bcee-48c2-8396-48a8fd850a8a.png)


cc @ngimel
Set up tests to run periodically and surface them on HUD
Labels: module: ci, triaged

This task is composed of following subtasks:

- [x] #84763

- [x] #84764
   Execute nightly channel tests 1 time a day after the nightly build is completed. 
   Execute release channel test 2 times a day at morning, afternoon
   Execute test channel after rc has been cut

- [x] #85295

- [x] #84765

- [x] #86370

cc @seemethere @malfet @pytorch/pytorch-dev-infra
Deepcopy of FX graph fails with nested make_fx and constant tensors
Labels: triaged, module: fx, fx

### 🐛 Describe the bug

Deepcopy of `make_fx` produced graphs is needed when using `torch.fx.passes.infra.partitioner.CapabilityBasedPartitioner` because that partitioner modifies the graph in-place.

```python
import torch
from copy import deepcopy
from torch.fx.experimental.proxy_tensor import make_fx

class Test(torch.overrides.TorchFunctionMode):
    def __torch_function__(
        self,
        f,
        types,
        args = (),
        kwargs = {},
    ):
        gm = make_fx(f)(*args)
        return f(*args, **(kwargs or {}))

a = torch.randn(3, 3)
b = torch.randn(3, 3)

def func(b):
    # note a is constant
    return a + b

# This works
gm = make_fx(func)(b)
deepcopy(gm)

with Test():
    gm = make_fx(func)(b)

# This fails
try:
    deepcopy(gm)
except Exception as e:
    print(e)
    pass

# But normal make_fx also fails now
try:
    gm = make_fx(func)(b)
    deepcopy(gm)
except Exception as e:
    print(e)
    pass
```
```py
File ~/dev/pytorch/master/torch/fx/experimental/proxy_tensor.py:383, in PythonKeyTracer.create_arg(self, a)
    381     assert a.get_pyobj().constant is not None
    382     return a.get_pyobj().constant
--> 383 return super().create_arg(a)

File ~/dev/pytorch/master/torch/fx/_symbolic_trace.py:344, in Tracer.create_arg(self, a)
    340     setattr(self.root, qualname, a)
    342     return self.create_node("get_attr", qualname, (), {})
--> 344 return super().create_arg(a)

File ~/dev/pytorch/master/torch/fx/proxy.py:166, in TracerBase.create_arg(self, a)
    163 elif isinstance(a, base_types) or a is None or a is ...:
    164     return a
--> 166 raise NotImplementedError(f"argument of type: {type(a)}")

NotImplementedError: argument of type: <class 'torch.storage.UntypedStorage'>
```

### Versions

Latest master.

cc @ezyang @SherlockNoMad @soumith
several questions about pytorch DDP
Labels: triaged, module: nccl, module: ddp

Hi there, I have several questions about pytorch DDP impletion. 

1. Which method allreduce op used by nccl in pytorch c10d, ring allreducer or double binary tree allreduce? better point out the code.
2. What data type is the gradient transfer between the processes when use amp, float32 or float16 or same data type as the parameter tensor?

### Versions

- pytorch version is  1.10.0
- nccl version is 2.10.3
Odd type-casting behaviour in prims.div
Labels: triaged, module: type promotion, module: primTorch

### 🐛 Describe the bug

We have that:
```python
>>> prims.div(torch.tensor([3]), torch.tensor([2.]))
```
Errors with a 
> RuntimeError: Tensor with dtype torch.float32 is not the expected dtype of torch.int64!

but
```python
>>> prims.div(torch.tensor([3]), 2.)
tensor([1.])
>>> prims.div(torch.tensor([3.]), 2)
tensor([1.5000])
```

I know that scalars have some funny type casting rules. Is this one of them? Or are we missing a kind check in the scalar case in `div_meta`? I am leaning more towards the latter one, as the check for integer inputs to see whether to perform the true division or truncated division is performed just on the first input (as if it was already known that the inputs have the same type). 

### Versions

master

cc @nairbv @mruberry @ezyang @ngimel
Installation prefix is not passed to CMake appropriately
Labels: needs reproduction, module: build, triaged

### 🐛 Describe the bug

When installing py-torch using the setup.py, giving it an installation prefix that is not where the source has been downloaded fails to pass the installation prefix to the CMake call.  This wound up with many paths to the source location hard-coded into the .so libraries, which are not necessarily available when running.  The issue was only uncovered when trying to run in distributed mode using more than one processor.  Interrogating `libtorch_cuda.so` with the `strings` program shows 444 instances of paths to files in the source prefix that aren't part of error messages.

### Versions

```
Collecting environment information...
PyTorch version: 1.12.0
Is debug build: True
CUDA used to build PyTorch: 11.4
ROCM used to build PyTorch: N/A

OS: Red Hat Enterprise Linux Server release 7.9 (Maipo) (ppc64le)
GCC version: (GCC) 4.9.3
Clang version: 12.0.1
CMake version: version 3.14.5
Libc version: glibc-2.17

Python version: 3.8.13 (default, Aug 11 2022, 15:49:12)  [GCC 8.3.1 20190311 (Red Hat 8.3.1-3)] (64-bit runtime)
Python platform: Linux-4.14.0.ppc64le-ppc64le-with-glibc2.17
Is CUDA available: True
CUDA runtime version: 11.4.100
GPU models and configuration:
GPU 0: Tesla V100-SXM2-16GB
GPU 1: Tesla V100-SXM2-16GB
GPU 2: Tesla V100-SXM2-16GB
GPU 3: Tesla V100-SXM2-16GB

Nvidia driver version: 510.47.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: False

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.1
[pip3] torch==1.12.0
[pip3] torchvision==0.13.0
[conda] Could not collect
```

cc @malfet @seemethere
explain() has confusing explanation of graph breaks
Labels: triaged, oncall: pt2

For full repro, see: n2409035.

The output of explain:
```
Dynamo produced 12 graphswith 11 graph break and 24 ops
 Break reasons: 

1. generic_jump TensorVariable() 
   File "/mnt/xarfuse/uid-572259/de1cb585-seed-nspid4026531836_cgpid105054292-ns-4026531840/fairseq/models/transformer/transformer_encoder.py", line 207, in forward_scriptable
    x, encoder_embedding = self.forward_embedding(src_tokens, token_embeddings)
 
2. generic_jump TensorVariable() 
   File "/mnt/xarfuse/uid-572259/de1cb585-seed-nspid4026531836_cgpid105054292-ns-4026531840/fairseq/models/transformer/transformer_encoder.py", line 127, in forward_embedding
    x = embed + self.embed_positions(src_tokens)
 
3. generic_jump 
   File "/mnt/xarfuse/uid-572259/de1cb585-seed-nspid4026531836_cgpid105054292-ns-4026531840/fairseq/modules/sinusoidal_positional_embedding.py", line 71, in forward
    if self.weights is None or max_pos > self.weights.size(0):
 
4. generic_jump 
   File "/mnt/xarfuse/uid-572259/de1cb585-seed-nspid4026531836_cgpid105054292-ns-4026531840/fairseq/models/transformer/transformer_encoder.py", line 210, in forward_scriptable
    if has_pads:
 
5. generic_jump 
   File "/mnt/xarfuse/uid-572259/de1cb585-seed-nspid4026531836_cgpid105054292-ns-4026531840/fairseq/models/transformer/transformer_encoder.py", line 225, in forward_scriptable
    x, encoder_padding_mask=encoder_padding_mask if has_pads else None
 
6. generic_jump 
   File "/mnt/xarfuse/uid-572259/de1cb585-seed-nspid4026531836_cgpid105054292-ns-4026531840/fairseq/modules/multihead_attention.py", line 506, in forward
    assert (
 
7. generic_jump 
   File "/mnt/xarfuse/uid-572259/de1cb585-seed-nspid4026531836_cgpid105054292-ns-4026531840/fairseq/modules/multihead_attention.py", line 506, in forward
    assert (
```
Two immediate issues:
- 11 graph breaks are quoted, but the list only has seven elements.
- Breaks 1 and 2 point to generic_jump but do not actually reference a control flow statement. They appear to be the callstack for break 3? If I fix break 3, breaks 1 and 2 go away.

Some more thoughts, but require more fleshing out to be actionable:
- I don't think a string is the right way to present this information; ideally it's queryable, (so I can do something like `breaks[2].source_location` in order to get more context then is provided here, including a full stack trace).
- The break reason is quite opaque if you don't understand the dynamo implementation.

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
torch.Size should convert all elements to ints
Labels: triaged, module: python frontend

### 🚀 The feature, motivation and pitch

Currently some things are kept as is. E.g., 
```py
In [227]: type(torch.Size([np.int64(3)])[0])
Out[227]: numpy.int64
```

This is usually not an issue but part of pytorch works with `np.int64` (e.g., `torch.randn(np.int64(3))`) and part don't (https://github.com/pytorch/pytorch/issues/43094). 

Either we should accept all `np.int64` as input to `int` types, which https://github.com/pytorch/pytorch/issues/43094 seems don't want to do, or we should not allow `torch.Size` to contain them.

### Alternatives

_No response_

### Additional context

_No response_
RuntimeError: "slow_conv2d_cpu" not implemented for 'Half'
Labels: triaged, module: half

### 🚀 The feature, motivation and pitch

Hi, I need this implementation to make a lot of models (resnets, efficientnets, ...) work on float16 inference.

When I run these models on GPU, it works properly.

### Alternatives

_No response_

### Additional context

_No response_
Lack of newly raised optimizers
Labels: high priority, feature, module: optimizer, triaged, needs research

### 🚀 The feature, motivation and pitch

I'm working on train Sota DNNs for research purposes. Optimizers like LAMB, Ranger and Lookahead have shown their advantage compared to SGD or AdamW in many tasks, including large batch training. However, Pytorch has stopped adding new optimizers algorithms for a long time. There are some third-party optimizers packages though, but an official implementation will be helpful.

### Alternatives

Using third-party implementations

### Additional context

_No response_

cc @ezyang @gchanan @zou3519 @vincentqb @jbschlosser @albanD @janeyx99
Floordiv is deprecated.
Labels: oncall: jit

### 🐛 Describe the bug

In torch/nn/functional.py,  __floordiv__ is used in group_norm function. __floordiv__ is deprecated, and its behavior will change in a future version of pytorch.

```
def group_norm(
    input: Tensor, num_groups: int, weight: Optional[Tensor] = None, bias: Optional[Tensor] = None, eps: float = 1e-5
) -> Tensor:
    r"""Applies Group Normalization for last certain number of dimensions.

    See :class:`~torch.nn.GroupNorm` for details.
    """
    if has_torch_function_variadic(input, weight, bias):
        return handle_torch_function(group_norm, (input, weight, bias,), input, num_groups, weight=weight, bias=bias, eps=eps)
    _verify_batch_size([input.size(0) * input.size(1) // num_groups, num_groups] + list(input.size()[2:]))
    return torch.group_norm(input, num_groups, weight, bias, eps, torch.backends.cudnn.enabled)
```

### Versions

Collecting environment information...
PyTorch version: 1.12.1+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: version 3.16.6
Libc version: glibc-2.27

Python version: 3.8.13 (default, Mar 28 2022, 11:38:47)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.4.0-124-generic-x86_64-with-glibc2.17
Is CUDA available: True
CUDA runtime version: 11.1.105
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090
Nvidia driver version: 465.19.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.2.2
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.2.2
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.2.2
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.2.2
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.2.2
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.2.2
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.2.2
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.2
[pip3] torch==1.12.1+cu113
[pip3] torchaudio==0.12.1+cu113
[pip3] torchvision==0.13.1+cu113
[conda] numpy                     1.23.2                   pypi_0    pypi
[conda] torch                     1.12.1+cu113             pypi_0    pypi
[conda] torchaudio                0.12.1+cu113             pypi_0    pypi
[conda] torchvision               0.13.1+cu113             pypi_0    pypi

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel @ezyang @gchanan @zou3519 @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
[Profiler] Snapshot CudaCachingAllocator on profile begin
Labels: module: bootcamp, oncall: profiler

### 🚀 The feature, motivation and pitch

One of the largest technical hurdles to memory profiling is deducing information about Tensors which were created before profiling began. PyTorch uses a custom [GPU allocator](https://github.com/pytorch/pytorch/blob/master/c10/cuda/CUDACachingAllocator.h) for fine grained control over GPU memory. One feature of this allocator is that it can expose a [detailed snapshot](https://github.com/pytorch/pytorch/blob/master/c10/cuda/CUDACachingAllocator.h#L151) of its current state. Moreover, [lightweight, always on source attribution](https://github.com/pytorch/pytorch/pull/82146) was recently added to CUDACachingAllocator. This information would go a long way towards disambiguating memory use for profiled regions and would provide a more complete picture of overall usage, and we should collect it in profiler.

### Alternatives

_No response_

### Additional context

_No response_

cc @chaekit @aaronenyeshi @ngimel @nbcsm @guotuofeng @guyang3532 @gaoteng-git @tiffzhaofb
[Profiler] Generic Tensor summary
Labels: module: bootcamp, oncall: profiler

### 🚀 The feature, motivation and pitch

The PyTorch profiler has several flags which enable optional functionality. One such flag is `record_shapes`, which is needed to assess kernel performance. However the scope of what is captured has gradually grown to include not only shape and dtype, but also [address, device, layout, and strides](https://github.com/pytorch/pytorch/blob/master/torch/csrc/profiler/collection.h#L51-L68). At the same time we have found applications for inspecting Tensors in other profiling contexts in the Python tracer for [`nn.Module` profiling](https://github.com/pytorch/pytorch/blob/master/torch/csrc/autograd/profiler_python.cpp#L305-L308), and soon optimizer profiling as well.

As PyTorch evolves various sharp edges have emerged: NestedTensors do not have a concept of strides, not all Tensors have storage, etc. We are also developing tooling in Python and the disparity between representations requires us to fall back to the lowest common denominator for analysis.

A more elegant and maintainable approach would be to have a single summary class which takes an `at::Tensor` and extracts a broadly useful summary which could then be conveniently bound into Python. This would harden the collection path by minimizing the boundary between profiler and TensorImpl/StorageImpl and provide a featureful analysis experience out of the box.

### Alternatives

_No response_

### Additional context

_No response_

cc @chaekit @aaronenyeshi @ngimel @nbcsm @guotuofeng @guyang3532 @gaoteng-git @tiffzhaofb
Fix convert path for fixed qparam ops (sigmoid and softmax)
Labels: oncall: quantization, triaged

### 🐛 Describe the bug

sigmoid is an op that works for both fp32 and quantized input: https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/native_functions.yaml#L4519

but we have a quantized sigomid module: https://github.com/pytorch/pytorch/blob/master/torch/ao/nn/quantized/modules/activation.py#L129-L148 this is confusing and we should remove it

## How to fix?
remove the quantized modules for all the modules in this list: https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/quantization_mappings.py#L163-L166


### Versions

master

cc @jianyuh @raghuramank100 @jamesr66a @vkuzo
torch.Tensor.to.dtype_layout overload is not available in Python
Labels: triaged, module: codegen, module: python frontend

### 🐛 Describe the bug

There's an overload `func: to.dtype_layout(Tensor(a) self, *, ScalarType? dtype=None, Layout? layout=None, Device? device=None, bool? pin_memory=None, bool non_blocking=False, bool copy=False, MemoryFormat? memory_format=None) -> Tensor(a)` 
https://github.com/pytorch/pytorch/blob/acd6ca8cfa9537284928fb5d36834d1e5ae1e6f3/aten/src/ATen/native/native_functions.yaml#L6581

But it's not used in dispatching for `torch.Tensor.to`, what's happening?


```py
In [1]: import torch

In [2]: a = torch.randn(3, 3, device="cuda")

In [3]: a.to(layout=torch.strided)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [3], in <cell line: 1>()
----> 1 a.to(layout=torch.strided)

TypeError: to() received an invalid combination of arguments - got (layout=torch.layout, ), but expected one of:
 * (torch.device device, torch.dtype dtype, bool non_blocking, bool copy, *, torch.memory_format memory_format)
 * (torch.dtype dtype, bool non_blocking, bool copy, *, torch.memory_format memory_format)
 * (Tensor tensor, bool non_blocking, bool copy, *, torch.memory_format memory_format)
 ```

### Versions

Latest master.

cc @ezyang @bhosmer @bdhirsh
would you like  upload to the cpp libtorch to  vcpkg  package repo?
Labels: module: cpp, module: ci, triaged, enhancement, topic: binaries

### 🚀 The feature, motivation and pitch

HI:
       now I use libtorch cpp in my cpp project ,my cpp project use  cmake with vcpkg, sometimes I need to defined the libtorch root path ,feel tried , if convenient  for us to upload the libtorch to the vcpkg for package manage https://vcpkg.io/en/packages.html will be great ,thanks

### Alternatives

_No response_

### Additional context

_No response_

cc @jbschlosser @seemethere @malfet @pytorch/pytorch-dev-infra
Support dict inputs and outputs when exporting to ONNX
Labels: module: onnx, triaged, onnx-needs-info

### 🚀 The feature, motivation and pitch

I have a model that takes `features: Dict[str, Tensor]` as input and produces `predictions: Dict[str, Tensor]` as output. It's straightforward to convert this model to TorchScript, but I need to export it to ONNX for deployment. Unfortunately, the PyTorch-to-ONNX converter (`torch.onnx.export`) doesn't seem to support dictionaries as inputs/outputs even though ONNX itself does.

### Alternatives

One option is to convert the dictionaries to keyword parameters, but that approach has several drawbacks:
1. More cumbersome to iterate on code -- input features may change over the course of a model's development. Adding named parameters to a function's signature is unnecessary overhead.
2. Outputs aren't named, so we'd have to return a list/tuple. In such cases, we have to be very careful about the return value order and make sure all scripts/tools that invoke the model are updated on any additions, removals, or permutations to the output.
3. We may only have access to the TorchScript model, so changing the inputs/outputs isn't even possible for us in all cases.

### Additional context

Here's a simple script that demonstrates the missing support for this feature:

```python
import torch

from typing import Dict


class A(torch.nn.Module):
  def forward(self, features: Dict[str, torch.Tensor]) -> Dict[str, torch.Tensor]:
    print(features['b'])
    return {
      'c': features['a'],
    }


module = torch.jit.script(A())
sample_a = torch.zeros([1,2], dtype=torch.float32)
sample_b = torch.zeros([2,1], dtype=torch.float32)
features = { 'a': sample_a, 'b': sample_b }
module(features)
torch.onnx.export(module, (features, {}), 'output.onnx')
```
Ensure ops account for offsets and strides
Labels: triaged, module: nestedtensor, release notes: nested tensor

# Summary

Many ops were were written for nested tensors in the context that they every nested tensor is contiguous. The only metadata needed to define the shape of nested tensor was a nested tensor size.  

This is no longer the case. nested_strides and nested_offsets were added. There may be ops that are still assuming sizes is all that is needed. One that comes to mind is: https://github.com/pytorch/pytorch/blob/372a19d2c673a20fe50955b20ea4e3685266d630/aten/src/ATen/native/nested/NestedTensorMath.cpp#L18
This should be updated to also pass in nested_strides and nested_offsets. There may be other functions that need to be update.

cc @cpuhrsch @jbschlosser @bhosmer @mikaylagawarecki
Randomness should be consistent across devices with use_deterministic_algorithms
Labels: triaged, module: random, module: determinism

### 🚀 The feature, motivation and pitch

Being able to generate reproducible results is a key part of the research process, as as a research platform, pytorch should support this goal. It currently does this by providing a "deterministic " option as described in https://pytorch.org/docs/stable/notes/randomness.html. However, even with that option enabled, random number generation is inconsistent across GPUs.

I think it would be reasonable to expect to get consistent random numbers when using the same seed, PyTorch version, and CUDA version even when running on a different device (whether CPU or physical GPU). From what I understand of random number generation, hardware differences such as processor counts or FPU precision don't need to play a role in RNG. And if they do, I'm guessing a slower, deterministic pathway could be chosen when `torch.use_deterministic_algorithms()` is used.

### Alternatives

The current alternatives I know of are:

1. Always use CPU (which seems to be consistent, although I am not sure that is guaranteed) as the target for random number generation 
2. Use a different, deterministic generator (e.g., numpy)
3. Use the same hardware devices everywhere

Options 1 and 2 require all researchers to know about this issue and write their code accordingly. Unfortunately, people are not aware of it. When they write code with determinism in mind, they typically allow for a torch generator to be provided as an API argument, assuming this will guarantee determinism. This is unfortunately not the case (and is further compounded by #62451). 

Option 3 is unfortunately unrealistic and can't be sustained over time as hardware becomes obsolete.

### Additional context

Related issues: #62467 #79496 #62451

cc @pbelevich @mruberry @kurtamohler
Pytorch gets small bias on the result of different types of divisors while doing floating point division.
Labels: module: numerical-stability, module: cuda, triaged

### 🐛 Describe the bug
results are different, which may arise from rounding.

```python
a1 = torch.tensor(858.8350830078125, dtype=torch.float32).cuda()/100.0

a2 = torch.tensor(858.8350830078125, dtype=torch.float32).cuda()/torch.tensor(100.0).cuda()

print(a1 - a2)
# out: tensor(-9.5367e-07, device='cuda:0')
```

### Versions

python: 3.8.10
pytorch: 1.8.1+cu111


While using tensorflow, the results are equal.

```python
with tf.device('/GPU:0'):
    a1 = tf.constant([858.8350830078125], dtype=tf.float32) /100.0
    a2 = tf.constant([858.8350830078125], dtype=tf.float32) / tf.constant(100.0, dtype=tf.float32)
    print(a1 - a2)

# out: tf.Tensor([0.], shape=(1,), dtype=float32)
```

cc @ngimel
Attach execution time to each node in an fx trace
Labels: triaged, fx, hacktoberfest

## Proposal description

A proposal for HacktoberFest. 
The idea is to plot a graph of a model that shows the execution time of each node or op.

Most likely, this project should be implemented as a standalone repo rather than a feature in PyTorch.

## Background

TBD

## Details
You would need to go over:
- torch.fx Documentation: https://pytorch.org/docs/stable/fx.html
- torch.fx Profiler Tutorial: https://pytorch.org/tutorials/intermediate/fx_profiling_tutorial.html
- Printing torch.fx Graph: https://github.com/pytorch/pytorch/blob/aec0f98e7088b68ba119becffd4fdbd64b4d75a8/torch/fx/passes/graph_drawer.py#L98C1-L120C16

cc @ezyang @SherlockNoMad @soumith
RuntimeError: outputs_[i]->uses().empty() INTERNAL ASSERT FAILED at /pytorch/torch/csrc/jit/ir.cpp:1027, please report a bug to PyTorch.  (eraseOutput at /pytorch/torch/csrc/jit/ir.cpp:1027)
Labels: triage review, oncall: jit

![图片](https://user-images.githubusercontent.com/27761585/187156882-cdb5d910-dba4-41f2-a135-63a29cf44b15.png)
Session of Google Colab crashes when `torch.utils::SummaryWriter` is called after importing `torchaudio`
Labels: high priority, module: crash, triaged, module: tensorboard

### 🐛 Describe the bug

I faced a session crash by running the following script on Google Colaboratory:

```python
import torchaudio
from torch.utils.tensorboard import SummaryWriter

writer = SummaryWriter() # session crashes
```

I'm not sure this bug is due to `pytorch`,  `torchaudio`, or Google Colab.

You can reproduce the crash by running https://gist.github.com/tky823/838b9bf3722c686213853e31aa3f1c1e

### Notes
I didn't face this bug by changing the order of imports, i.e.

```python
from torch.utils.tensorboard import SummaryWriter
import torchaudio

writer = SummaryWriter() # session works!
```

### Versions

```
Collecting environment information...
PyTorch version: 1.12.1+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
CMake version: version 3.22.6
Libc version: glibc-2.26

Python version: 3.7.13 (default, Apr 24 2022, 01:04:09)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
Is CUDA available: False
CUDA runtime version: 11.1.105
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.0.5
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.6
[pip3] torch==1.12.1+cu113
[pip3] torchaudio==0.12.1+cu113
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.13.1
[pip3] torchvision==0.13.1+cu113
[conda] Could not collect
```

cc @ezyang @gchanan @zou3519
Support setting strides on quantized weights of Embedding
Labels: triage review, oncall: quantization, triaged

### 🚀 The feature, motivation and pitch

For quantized Embedding, [deberta in transformers](https://github.com/huggingface/transformers/blob/21f6f58721dd9154357576be6de54eefef1f1818/src/transformers/models/deberta/modeling_deberta.py#L694) will choose weights from original weights, but I get: 
`*** RuntimeError: Setting strides is possible only on uniformly or per channel quantized tensors`.

Below are the details of quantized weights of Embedding. torch.per_channel_affine_float_qparams should be a per channel quantization schema, right?
---------
tensor([[ 0.0064, -0.0069, -0.0082,  ..., -0.0215, -0.0056, -0.0003],
        [-0.0069,  0.0087, -0.0225,  ..., -0.0136, -0.0069,  0.0065],
        [-0.0071,  0.0225, -0.0071,  ..., -0.0182, -0.0219,  0.0151],
        ...,
        [-0.0037,  0.0060, -0.0203,  ..., -0.0176,  0.0088,  0.0102],
        [-0.0126, -0.0350, -0.0074,  ..., -0.0284,  0.0032, -0.0218],
        [ 0.0065, -0.0123,  0.0018,  ..., -0.0170,  0.0065,  0.0018]],
       size=(50265, 768), dtype=torch.quint8,
       quantization_scheme=torch.per_channel_affine_float_qparams,
       scale=tensor([0.0013, 0.0022, 0.0037,  ..., 0.0014, 0.0013, 0.0047]),
       zero_point=tensor([ 87.2115,  50.1056,  93.9225,  ...,  77.6577,  81.5797, 115.6229]),
       axis=0)


### Alternatives

No

### Additional context

_No response_

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel
`linux-bionic-cuda10.2-py3.9-gcc7`  multigpu test are broken
Labels: oncall: distributed, module: ci, module: regression

### 🐛 Describe the bug

With the following signature:
```
 File "/var/lib/jenkins/workspace/test/distributed/test_c10d_gloo.py", line 2357, in test_work_wait_gpu
    self._test_work_wait(torch.ones(2, 2, device=self.rank) * self.rank)
  File "/var/lib/jenkins/workspace/test/distributed/test_c10d_common.py", line 1565, in _test_work_wait
    traced_fn.graph.print_tabular()
  File "/opt/conda/lib/python3.9/site-packages/torch/fx/graph.py", line 1237, in print_tabular
    print(tabulate(node_specs,
UnboundLocalError: local variable 'tabulate' referenced before assignment
 exiting process 0 with exit code: 10
ERROR:torch.testing._internal.common_distributed:Caught exception: 
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/torch/testing/_internal/common_distributed.py", line 622, in run_test
    getattr(self, test_name)()
  File "/opt/conda/lib/python3.9/site-packages/torch/testing/_internal/common_distributed.py", line 503, in wrapper
    fn()
  File "/opt/conda/lib/python3.9/site-packages/torch/testing/_internal/common_distributed.py", line 145, in wrapper
    return func(*args, **kwargs)
  File "/var/lib/jenkins/workspace/test/distributed/test_c10d_gloo.py", line 2357, in test_work_wait_gpu
    self._test_work_wait(torch.ones(2, 2, device=self.rank) * self.rank)
  File "/var/lib/jenkins/workspace/test/distributed/test_c10d_common.py", line 1595, in _test_work_wait
    y = fn(x)
  File "/var/lib/jenkins/workspace/test/distributed/test_c10d_common.py", line 1554, in fn
    work.wait()
RuntimeError: [/var/lib/jenkins/workspace/third_party/gloo/gloo/transport/tcp/pair.cc:598] Connection closed by peer [172.17.0.2]:5258
 exiting process 1 with exit code: 10
Process 0 terminated with exit code 10, terminating remaining processes.
ERROR (5.020s)
    test_work_wait_gpu errored - num_retries_left: 3
```

### Versions

CI

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501 @seemethere @pytorch/pytorch-dev-infra
[nvfuser] view size is not compatible with input tensor's size and stride
Labels: triaged, oncall: pt2

Instructions 
* Uncompress Repro file - [repro.tar.gz](https://github.com/pytorch/torchdynamo/files/9436625/repro.tar.gz)
* Run `python repro/repro.py`

The repro is minified using https://github.com/pytorch/torchdynamo/pull/1056
This only fail with `aot_nvfuser` backend.


 

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh