gopikrsmscs/torch-issues huggingface dataset:
build fail when using lto with gcc
Labels: module: build, triaged

### 🐛 Describe the bug

The bug is reported to gentoo at https://bugs.gentoo.org/862672

During the build I get

_/var/tmp/portage/sci-libs/caffe2-1.12.0/work/pytorch-1.12.0/aten/src/ATen/native/cpu/moments_utils.h:76: error: type of ‘c_vecs’ does not match original declaration [-Werror=lto-type-mismatch]
   76 |     static std::array<Vec, kChunkSize> c_vecs = ([]() {
      | 
/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/include/g++-v11/array:95: note: type name ‘std::array<at::vec::DEFAULT::Vectorized<c10::BFloat16>, 16ul>’ should match type name ‘std::array<at::vec::AVX2::Vectorized<c10::BFloat16>, 16ul>’
   95 |     struct array
      | 
/var/tmp/portage/sci-libs/caffe2-1.12.0/work/pytorch-1.12.0/aten/src/ATen/native/cpu/moments_utils.h:76: error: ‘c_vecs’ violates the C++ One Definition Rule [-Werror=odr]
   76 |     static std::array<Vec, kChunkSize> c_vecs = ([]() {
      | 
/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/include/g++-v11/array:95: note: type name ‘std::array<at::vec::DEFAULT::Vectorized<c10::BFloat16>, 16ul>’ should match type name ‘std::array<at::vec::AVX512::Vectorized<c10::BFloat16>, 16ul>’
   95 |     struct array
      | 
/var/tmp/portage/sci-libs/caffe2-1.12.0/work/pytorch-1.12.0/aten/src/ATen/native/cpu/moments_utils.h:76: note: ‘c_vecs’ was previously declared here
   76 |     static std::array<Vec, kChunkSize> c_vecs = ([]() {
      | 
/var/tmp/portage/sci-libs/caffe2-1.12.0/work/pytorch-1.12.0/aten/src/ATen/native/cpu/moments_utils.h:76: note: code may be misoptimized unless ‘-fno-strict-aliasing’ is used_

my guess is that the compiler has problem (in lto mode) to distinguish the various static variable c_vecs created on different instance of the template

### Versions

version is 1.12.0

Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: Gentoo Base System release 2.8 (x86_64)
GCC version: (Gentoo 11.3.0 p4) 11.3.0
Clang version: 14.0.4
CMake version: version 3.22.4
Libc version: glibc-2.34

Python version: 3.10.5 (main, Jun 29 2022, 11:04:31) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.15.32-gentoo-r1-x86_64-x86_64-Intel-R-_Core-TM-_i5-4570_CPU_@_3.20GHz-with-glibc2.34
Is CUDA available: N/A
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] numpy==1.22.3
[conda] Could not collect

cc @malfet @seemethere
Move nested-tensor tutorial from prototype
Labels: triaged, module: nestedtensor

### 📚 The doc issue

Nested-tensor tutorial involves ops that are nightly-only now, so it is put under prototype/ and have _tutorial suffix removed.

### Suggest a potential alternative/fix

Once new PyTorch release comes out, we should move it back to beginner/ and append _tutorial suffix.

cc @cpuhrsch @jbschlosser @bhosmer @drisspg @mikaylagawarecki
SequentialLR does not work correctly with multiple ConstantLR
Labels: triaged, module: LrScheduler

In combination with multiple ConstantLR schedulers, SequentialLR should use one at a time (depending whether the current epoch and milestones), but it instead applies several at the same time. For the example below:

```
import torch.optim
from torch import nn
from torch.optim.lr_scheduler import ConstantLR
from torch.optim.lr_scheduler import SequentialLR

model = nn.Linear(10, 10)
optimizer = torch.optim.Adam(model.parameters(), lr=1.0)

scheduler1 = ConstantLR(optimizer, factor=0.5, total_iters=3)
scheduler2 = ConstantLR(optimizer, factor=0.3, total_iters=4)
scheduler = SequentialLR(optimizer, schedulers=[scheduler1, scheduler2], milestones=[3])

for step in range(6):
    scheduler.step()
    print(step, scheduler.get_last_lr())
```

The output is:
0 [0.15]
1 [0.15]
2 [0.3]
3 [0.3]
4 [0.3]
5 [0.3]

While the correct output should be:
0 [0.5]
1 [0.5]
2 [0.3]
3 [0.3]
4 [0.3]
5 [0.3]

### Versions

1.12
RReLU doc doesn't specify the eval mode behaving just like LeakyReLU
Labels: module: docs, module: nn, triaged, actionable, topic: docs

### 📚 The doc issue

[RReLU](https://pytorch.org/docs/stable/generated/torch.nn.RReLU.html) behaves the same as LeakyReLU when it's on eval mode, but the documentation doesn't seem to provide such information.

### Suggest a potential alternative/fix

Maybe add a link or a notice specifying this would be better.

cc @svekars @holly1238 @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
unittest.subTest and way to selectively mark subTests as expected failures
Labels: triaged, better-engineering, module: testing

### 🚀 The feature, motivation and pitch

When testing if something like vmap works on an operator, we test the following things:
1. if it errors out (it shouldn't)
2. if the output of vmap matches the output of a for loop (it should)
3. if there is a batching rule implemented for the operation. We do this by running the vmap and checking if it raises any "batching rule not implemented" warnings.

We have two separate tests, test_vmap_exhaustive, and test_op_has_batch_rule. The former does (1) and (2), and the latter does (1), (2), (3) (because (1) and (2) are almost required for (3)). We could cut down the test time if we have one test and used something like unittest.subTest in a way so that each test gets 3 subtests.

Furthermore, if a vmap test fails for e.g. torch.searchsorted, we have an expected failure for it. It would be nice to be able to distinguish between if it failed because of (1) or if it failed because of (2); (2) is a silent correctness issue and is much more hi-pri to fix.

### Alternatives

n/a

### Additional context

cc @jbschlosser who has worked on many testing improvements in the past
Schema information for torch.* operations
Labels: triaged, module: __torch_function__, module: testing

### 🚀 The feature, motivation and pitch

My main motivation for this is for in-place vmap testing. In order to do vmap in-place testing, we must, given an OpInfo, construct some sample inputs, some of which are batched or not. For example, given `Tensor.add_(x, y)`, we would need to generate x and y.

There is a problem if we generate x to be a regular Tensor and y to be the Tensor being vmapped over, because that errors out. If we knew that the first argument to `Tensor.add_` is the argument that gets mutated, then we can avoid generating the aforementioned case.

I don't know if anything else would find this useful, but just putting it out there.

### Alternatives

- Assume that the first argument of an in-place operation gets mutated :) (this is not always true)
- Try to "guess" what the aten operator that corresponds to a python torch in-place operation is. e.g. Tensor.add_ -> at::add_. This does not always work.

### Additional context

_No response_

cc @hameerabbasi @rgommers @peterbell10 @ezyang
in-place variants should get their own OpInfos
Labels: triaged, better-engineering, module: testing

### 🚀 The feature, motivation and pitch

Today, to write an OpInfo test to check that a subsystem (let's say vmap) works on all operations, we write something like the following:
```
@ops(op_db)
def test_vmap(self, device, dtype, op):
  test_op(op)
  if op.inplace_variant:
    test_op(op.inplace_variant)
```
Let's say that we have correct vmap support for torch.add, but incorrect support for `Tensor.add_`. Then there is no way to specify "I want to skip this test for Tensor.add_ but have the test run for torch.add" without adding additional infrastructure.

If we made in-place variants their own OpInfos, we could rewrite the above test as the following. And, since the OpInfo for "add" would be separate from "add_", then we would be able to skip them separately from each other.
```
@ops(op_db)
def test_vmap(self, device, dtype, op):
  test_op(op)
```

### Alternatives

Split test_vmap into the following:
```
@ops(op_db)
def test_vmap(self, device, dtype, op):
  test_op(op)

@ops(op_db)
def test_vmap_inplace(self, device, dtype, op):
  if op.inplace_variant:
    test_op(op.inplace_variant)
 ```
 but keep in mind that everyone who uses OpInfos and cares about out-of-place and in-place operations now needs to write two tests.

### Additional context

cc @mruberry @ngimel what are your thoughts?
contiguous() not work for rank 1 length 1 tensor.
Labels: triaged, module: dlpack

### 🐛 Describe the bug

When I try `torch.tensor([1.+2.j, 3.+4.j]).real.contiguous().stride()`, it returns `(1,)` as expected.

But if the length is only 1, the stride is not 1 but 2, that is to say `torch.tensor([1.+2.j]).real.contiguous().stride()` gives `(2,)`

Although stride of length 1 tensor has no effect. But when I try to use mpi4py to bcast/allreduce tensor, it will throw an error because of wrong stride.

### Versions

Collecting environment information...
PyTorch version: 1.12.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Arch Linux (x86_64)
GCC version: (GCC) 12.1.0
Clang version: 14.0.6
CMake version: version 3.23.3
Libc version: glibc-2.35

Python version: 3.10.5 (main, Jun  6 2022, 18:49:26) [GCC 12.1.0] (64-bit runtime)
Python platform: Linux-5.18.12-arch1-1-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.961
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.1
[pip3] torch==1.12.0
[conda] Could not collect

cc @ezyang @anjali411 @dylanbespalko @mruberry @Lezcano @nikitaved
Deep copy models with `create_feature_extractor` produces different parameters
Labels: triage review, triaged, module: vision, oncall: fx

### 🐛 Describe the bug

When I define a model using the new feature `create_feature_extractor` and then deep copy the model, the parameters of the original model and the new models are different.

Here is an example:

```python
from torchvision.models.feature_extraction import create_feature_extractor
import torch.nn as nn
import torchvision
import copy

class cusResNet18(nn.Module):    
    def __init__(self, n_classes, pretrained = True):
        super(cusResNet18, self).__init__()
        self.resnet = torchvision.models.resnet18(pretrained=pretrained)
        self.resnet.fc = nn.Linear(512, n_classes)
        
        self.avgpool = self.resnet.avgpool
        
        self.returnkey_avg = 'avgpool'
        self.returnkey_fc = 'fc'
        self.body = create_feature_extractor(
            self.resnet, return_nodes={'avgpool': self.returnkey_avg, 'fc': self.returnkey_fc})

    def forward(self, x):
        outputs = self.body(x)
        return outputs[self.returnkey_fc], outputs[self.returnkey_avg].squeeze()
    
model = cusResNet18(n_classes=1)
copied_model = copy.deepcopy(model)

print(len(list(model.parameters())), len(list(copied_model.parameters())))
```
The output is ```62 124```.

If printing out the named_parameters, the differences are from the defined self.body.
```
print(dict(model.named_parameters()).keys(), dict(copied_model.named_parameters()).keys())
```

I'm wondering why there is a difference after using deepcopy and how can I deepcopy a model that has the create_feature_extractor feature inside?

### Versions


torch==1.10.2
torchvision==0.11.3

cc @fmassa @vfdev-5 @pmeier @ezyang @SherlockNoMad
DataLoader parameter pin_memory_device should accept torch.device type
Labels: module: dataloader, triaged

### 🚀 The feature, motivation and pitch

Currently, the DataLoader class parameter pin_memory_device only accepts a device in string format. It should be possible to pass a torch.device instead of a string. 

### Alternatives

_No response_

### Additional context

_No response_

cc @SsnL @VitalyFedyunin @ejguan @NivekT
PyTorch for quantum mechanics
Labels: feature, triaged, function request, module: scientific computing

### 🚀 The feature, motivation and pitch

As I begin working with PyTorch to simulate and optimize quantum systems, I propose to open this issue to list features that would be helpful.

Note that the goal is **not** to do quantum machine learning (e.g. https://www.tensorflow.org/quantum), but rather to simulate quantum system time evolution (using Schrödinger equation to begin with) and to perform gradient-based optimal control.

### Alternatives

_No response_

### Additional context

Related to #71446, but it is not active and specific to MIT's library.

_Don't hesitate to close this issue if you find it inappropriate, and prefer that I post individual feature requests._
make_fx is broken for all tracing modes
Labels: high priority, module: crash, triaged, module: fx, fx

### 🐛 Describe the bug

Running the script from https://gist.github.com/zou3519/3869d460f8bcb12799967e08a5998d9c raises an error on the `make_fx` call for `symbolic` and `real` tracing modes and segfaults for `tracing_mode="fake"`.

`tracing_mode="real"` :
```py
File ~/dev/pytorch/master/torch/fx/experimental/proxy_tensor.py:134, in proxy_call(func_overload, args, kwargs)
    132     if t.constant is not None:
    133         with maybe_disable_fake_tensor_mode():
--> 134             return t.constant.item()
    135     raise RuntimeError("It appears that you're trying to get value out of a tracing tensor - erroring out! "
    136                        "It's likely that this is caused by data-dependent control flow or similar."
    137                        "Try torch.fx.experimental.proxy_tensor.enable_strict(False) to disable this check")
    139 def unwrap_proxy(e):

AttributeError: 'list' object has no attribute 'item'
```
using `tracing_mode="symbolic"` raises another error:
```py
File ~/dev/pytorch/master/functorch/functorch/_src/vmap.py:484, in _flat_vmap(func, batch_size, flat_in_dims, flat_args, args_spec, out_dims, randomness, **kwargs)
    483 def _flat_vmap(func, batch_size, flat_in_dims, flat_args, args_spec, out_dims, randomness, **kwargs):
--> 484     vmap_level = _vmap_increment_nesting(batch_size, randomness)
    485     try:
    486         batched_inputs = _create_batched_inputs(flat_in_dims, flat_args, vmap_level, args_spec)

TypeError: _vmap_increment_nesting(): incompatible function arguments. The following argument types are supported:
    1. (arg0: int, arg1: str) -> int

Invoked with: <torch.SymIntNode object at 0x7f6275eec3f0>, 'error'
```
and `tracing_mode="fake"` segfaults.

### Versions

Latest master branch.

cc @ezyang @gchanan @zou3519 @SherlockNoMad
Libtorch C++ torch::stack error
Labels: needs reproduction, module: cpp, triaged

#include <iostream>
#include <torch/torch.h>

int main()
{
	torch::Tensor a = torch::rand({ 1, 4 });
	torch::Tensor b = torch::rand({ 1, 4 });
	std::cout << " a = " << a << std::endl;
	std::cout << " b = " << b << std::endl;
	torch::Tensor c = torch::stack({ a, b }, 0);
	std::cout << " c = " << c << std::endl;
	return 0;
}
![image](https://user-images.githubusercontent.com/52875069/182019591-5d343c91-faa5-4d4e-b02d-3fbd8664ab27.png)


cc @jbschlosser
Incorrect CPU implementation of CTCLoss backward step
Labels: module: autograd, module: loss, triaged

### 🐛 Describe the bug

The ATen CTCLoss backward step seems to produce incorrect gradients. The following code snippet reproduces this issue. It computes the differences of the output with respect to every input parameter `input`, and compares them with the backpropagated gradients. The results shall be approximately equal. 

``` python
#!/usr/bin/env python3
import torch

torch.manual_seed(0)
torch.set_printoptions(precision=10)

T, C, S = 5, 3, 4

input = torch.randn(T, C).log_softmax(1).detach().requires_grad_()
target = torch.randint(low=1, high=C, size=(S,), dtype=torch.long)

loss = torch.nn.functional.ctc_loss(
    input, target, torch.tensor(T), torch.tensor(S))
loss.backward()

for i in range(T):
    for j in range(C):
        new_input = input.clone().detach()
        new_input[i][j] += 0.01

        new_loss = torch.nn.functional.ctc_loss(
            new_input, target, torch.tensor(T), torch.tensor(S))

        print((new_loss - loss).detach(), input.grad[i][j] * 0.01) # expected to be approximately equal
```

The actual output is shown below, and the two columns differ significantly.

```
tensor(0.) tensor(0.0021115856)
tensor(0.) tensor(0.0003372314)
tensor(-0.0024995804) tensor(-0.0024488156)
tensor(0.) tensor(0.0018777853)
tensor(-0.0024995804) tensor(-0.0021404340)
tensor(0.) tensor(0.0002626498)
tensor(0.) tensor(0.0008711107)
tensor(0.) tensor(0.0013454026)
tensor(-0.0024995804) tensor(-0.0022165121)
tensor(-0.0025000572) tensor(-0.0018093735)
tensor(0.) tensor(0.0005692409)
tensor(0.) tensor(0.0012401351)
tensor(0.) tensor(0.0002813108)
tensor(0.) tensor(0.0019916897)
tensor(-0.0024998188) tensor(-0.0022729994)
```

It seems that 

https://github.com/pytorch/pytorch/blob/4bb7e148c46167cb2b0beedf4332eb6eae5b03cc/aten/src/ATen/native/LossCTC.cpp#L330

shall be corrected to

``` c++
res = -std::exp(res + nll - lp) * gr;
```

### Versions

PyTorch version: 1.12.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.5 (x86_64)
GCC version: Could not collect
Clang version: 13.1.6 (clang-1316.0.21.2.5)
CMake version: version 3.18.2
Libc version: N/A

Python version: 3.8.5 (default, Jul 21 2020, 10:48:26)  [Clang 11.0.3 (clang-1103.0.32.62)] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.19.2
[pip3] torch==1.12.0
[conda] Could not collect

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
Is there Doc that explains how to call an extension op in another extension implementation?
Labels: module: docs, module: cpp, triaged

### 📚 The doc issue

For example, there is an extension op which is installed from public repo via `pip install torch-scatter`, and in Python code, it's easy to use this extension:

```py
import torch
output = torch.ops.torch_scatter.scatter_max(x, index)
```

However, I'm writing an C++ extension and want to call this extension as well, but I cannot find any doc that guides how to do this, or I don't know whether Pytorch C++ extension can even support it or not. Briefly, this is something I'd like to do in extension function:

```cpp
torch::Tensor my_op(torch::Tensor x, torch::Tensor y, torch::Tensor z) {
  auto temp = torch::ops::torch_scatter::scatter_max(z, y.view(-1)); // not working
  ..
  return temp;
}
```

### Suggest a potential alternative/fix

_No response_

cc @svekars @holly1238 @jbschlosser
Use NestedTensor in RNN models
Labels: triaged, enhancement, module: nestedtensor

### 🚀 The feature, motivation and pitch

Now that NestedTensor is in core, I believe a high-impact application would be to replace many RNN utils, such as:

```
torch.nn.utils.rnn.pad_sequence
torch.nn.utils.rnn.pad_packed_sequence
torch.nn.utils.rnn.pack_sequence
```
with nested tensors. LSTM/GRUs are still heavily used in RL, where long episodes can result in a batch of sequences from 1 to 10,000 timesteps. The entire batch must be zero-padded to the length of the longest episode, which uses a ton of memory (or perhaps the RNN utils do something smarter). The RNN utils also pass around list of indices and sort the outputs, which the user must unsort. Furthermore, the above methods are not flexible -- they do not allow a sequence of zero length, for example.

It would be fantastic to stack a bunch of episodes/rollouts into a ragged NestedTensor and pass this straight into an LSTM. I believe all that's needed are sigmoid, tanh, and linear operations which I believe NestedTensor already supports.

I don't think you would have to worry about backwards compatibility. You can check `instanceof(lstm_input, torch.NestedTensor)` and branch from there.

### Alternatives

_No response_

### Additional context

cc @cpuhrsch @jbschlosser @bhosmer @drisspg
Split up `common_methods_invocations.py`?
Labels: triaged, needs research, better-engineering, module: testing

`common_methods_invocations.py` has grown to 22K lines and over 1MB in file size. One implication of this is you can't open it in the GitHub UI or link to specific lines of code.

I propose creating an opinfos folder and where there are currently different categories, such as `UnaryUfuncInfo` or `ReductionOpInfo`, these could be their own file.
Symbolic tensors are not printable
Labels: module: printing, triaged, module: dynamic shapes

### 🐛 Describe the bug

```
  File "/data/users/ezyang/pytorch-tmp/test/test_dynamic_shapes.py", line 258, in test_aten_ops
    print(x)
  File "/data/users/ezyang/pytorch-tmp/torch/_tensor.py", line 423, in __repr__
    return torch._tensor_str._str(self, tensor_contents=tensor_contents)
  File "/data/users/ezyang/pytorch-tmp/torch/_tensor_str.py", line 591, in _str
    return _str_intern(self, tensor_contents=tensor_contents)
  File "/data/users/ezyang/pytorch-tmp/torch/_tensor_str.py", line 535, in _str_intern
    if self.numel() == 0 and not self.is_sparse:
RuntimeError: Tensors of type TensorImpl do not have numel

```

Maybe this will get fixed when we get sym numel landed

### Versions

master
Implement torch.clamp() on sparse tensors with SparseCPU backend
Labels: module: sparse, triaged

### 🚀 The feature, motivation and pitch

torch.clamp() fails on sparse tensors using SparseCPU backend. Among other things, this breaks gradient clipping with sparse tensors. 

```
import torch

sparse_tensor = torch.sparse_coo_tensor([[1,2]], [1,5], (3,))
torch.clamp(sparse_tensor, -1, 1)
```

Fails with error: `Could not run 'aten::clamp' with arguments from the 'SparseCPU' backend.`

### Alternatives

It's not exactly difficult to manually clamp sparse tensors, but we shouldn't have to when a function exists.

### Additional context

_No response_

cc @nikitaved @pearu @cpuhrsch @amjames @bhosmer
Cloning conjugate tensor in torch_dispatch context produces non equality.
Labels: triaged, module: complex, module: __torch_dispatch__

### 🐛 Describe the bug

```python
class TestMode(TorchDispatchMode):
        def __torch_dispatch__(self, func, types, args=(), kwargs=None):
                args2 = clone_inputs(args)
                out = func(*args, **kwargs)
                for i in range(len(args2)):
                        print(f'before {args2[i]}, after {args[i]}, {torch.equal(args2[i], args[i])}')
                return out

a = torch.rand((3,3), dtype=torch.complex64)
b = torch.rand((3,3), dtype=torch.complex64)
b = torch.conj(b)
with enable_torch_dispatch_mode(TestMode()):
        torch.mm(a, b)
```
This prints:

```python
before tensor([[0.0290+0.4019j, 0.2598+0.3666j, 0.0583+0.7006j],
        [0.0518+0.4681j, 0.6738+0.3315j, 0.7837+0.5631j],
        [0.7749+0.8208j, 0.2793+0.6817j, 0.2837+0.6567j]]), after tensor([[0.0290+0.4019j, 0.2598+0.3666j, 0.0583+0.7006j],
        [0.0518+0.4681j, 0.6738+0.3315j, 0.7837+0.5631j],
        [0.7749+0.8208j, 0.2793+0.6817j, 0.2837+0.6567j]]), True
before tensor([[0.2388-0.7313j, 0.6012-0.3043j, 0.2548-0.6294j],
        [0.9665-0.7399j, 0.4517-0.4757j, 0.7842-0.1525j],
        [0.6662-0.3343j, 0.7893-0.3216j, 0.5247-0.6688j]]), after tensor([[0.2388-0.7313j, 0.6012-0.3043j, 0.2548-0.6294j],
        [0.9665-0.7399j, 0.4517-0.4757j, 0.7842-0.1525j],
        [0.6662-0.3343j, 0.7893-0.3216j, 0.5247-0.6688j]]), False
```
Which indicates that `mm` mutates its second input which is not the case. 



### Versions

N/A

cc @ezyang @anjali411 @dylanbespalko @mruberry @Lezcano @nikitaved @Chillee @zou3519 @albanD @samdow
Guide for diagnosing excess graph breaks
Labels: module: docs, triaged, oncall: pt2, module: dynamo

Something a user using torchdynamo might want to do is figure out why they are having lots of graph breaks. We have tools for doing this, but they are not documented. Prior to the PT2 release it would be good to document them. cc @svekars @carljparker @soumith @msaroufim @wconstab @ngimel @bdhirsh @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire
SyncBatchNorm does not work on CPU
Labels: oncall: distributed, module: nn

Using Pytorch 1.12 on Debian Linux (conda env).
SyncBatchNorm does not seem to work with jit.trace. On a mobilenet V3 model, I first do:

```
model = torch.nn.SyncBatchNorm.convert_sync_batchnorm(model)
```

After training, normally, with my models I do this:

```
 traced_script_module = torch.jit.trace(swa_model, (data))
 traced_script_module.save("./swa_model.tar")
```

both model and data are on cpu - the plan is to use the model for inference on cpu only.

However, it throws me this:

```
    raise ValueError("SyncBatchNorm expected input tensor to be on GPU")
ValueError: SyncBatchNorm expected input tensor to be on GPU
```

Is there a workaround for this? or is this some weird edge case which is not supported? 

Ofcourse, without SyncBatchNorm, everything is fine - ie the trace is produced and gives me the same result as the original model.

Thank you.




cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501 @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
add support for bitwise operations with floating point numbers
Labels: oncall: jit

### 🚀 The feature, motivation and pitch

I was rewriting hash nerf so I could export a jitted version for training in c++, when I came across this limitation. Doing bitwise operation with floats is easy in eager mode with torch.tensor.view(torch.int32). However currently TorchScript does not support reinterpret_casting tensors, and jitting is a must.

### Alternatives

Support for reinterpret_casting with tensors in TorchScript  would be a better solution but I assume that would take a long time to optimize well.

### Additional context

_No response_
Minor inconsistency in description of `attn_output_weights` in MultiheadAttention docs
Labels: module: docs, module: nn, triaged, actionable

### 📚 The doc issue

On the documentation of [MultiheadAttention](https://pytorch.org/docs/stable/generated/torch.nn.MultiheadAttention.html?highlight=multihead#torch.nn.MultiheadAttention) of version 1.12 on the description of `attn_output_weights` argument, in the last sentence, it is mentioned that:

 > If **average_weights=False**, returns attention weights per head of shape [...].

I think that the correct argument on the forward method is `average_attn_weights` and not `average_weights`. 

### Suggest a potential alternative/fix

Can be fixed by replacing `average_weights=False` to `average_attn_weights`.

cc @svekars @holly1238 @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
The torch::deploy document is not updated
Labels: triaged, module: deploy

### 📚 The doc issue

Hi, 

I was trying the tutorial example in torch::deploy document, more specifically (deploy with C++)[https://pytorch.org/docs/stable/deploy.html#building-and-running-the-application]. However, the example doesn't work. Here are the details.

It looks like it is because of compilation problem in pytorch. After seemingly sucuessful compilation, the `libtorch_deployinterpreter.o` file (or similar files) cannot be found in the directory. This blocks the next steps of the example. Another related (compiling issue)[https://github.com/pytorch/pytorch/issues/82382] shows that the compilation is even not successful in Linux (due to missing file). 

Could you update this part of document or point to me other working demo examples? 

---
Here is the environment of my system
```
Collecting environment information...
PyTorch version: 1.13.0a0+git1a9317c
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.4 (x86_64)
GCC version: Could not collect
Clang version: 13.1.6 (clang-1316.0.21.2.5)
CMake version: version 3.19.6
Libc version: N/A

Python version: 3.9.12 (main, Apr  5 2022, 01:53:17)  [Clang 12.0.0 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.5
[pip3] numpydoc==1.2
[pip3] torch==1.13.0a0+git1a9317c
[pip3] torchaudio==0.12.0
[pip3] torchvision==0.13.0
[conda] blas                      1.0                         mkl  
[conda] ffmpeg                    4.3                  h0a44026_0    pytorch
[conda] mkl                       2021.4.0           hecd8cb5_637  
[conda] mkl-include               2022.0.0           hecd8cb5_105  
[conda] mkl-service               2.4.0            py39h9ed2024_0  
[conda] mkl_fft                   1.3.1            py39h4ab4a9b_0  
[conda] mkl_random                1.2.2            py39hb2f4e1b_0  
[conda] numpy                     1.21.5           py39h2e5f0a9_1  
[conda] numpy-base                1.21.5           py39h3b1a694_1  
[conda] numpydoc                  1.2                pyhd3eb1b0_0  
[conda] pytorch                   1.12.0                  py3.9_0    pytorch
[conda] torch                     1.13.0a0+git1a9317c          pypi_0    pypi
[conda] torchaudio                0.12.0                 py39_cpu    pytorch
[conda] torchvision               0.13.0                 py39_cpu    pytorch
```


### Suggest a potential alternative/fix

_No response_

cc @wconstab
[JIT] _unsafe_view returns alias when size(input) = size argument
Labels: oncall: jit

### 🐛 Describe the bug

aten::_unsafe_view seems to return an alias of input as output when the size argument is equal to the size of the initial input. This isn't currently marked on native_functions.yaml, but seems to be intended due to:
https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/TensorShape.cpp#L2728
Keeping this here just to track this possible issue.

### Versions

N/A
Bilinear interpolation with antialiasing is slow in performance
Labels: module: performance, triaged

### 🐛 Describe the bug

Bilinear interpolation with antialiasing is significantly slower than without.
Is this intended behavior or am I missing something?

Code to reproduce
```python
import torch
import torch.nn.functional as F
from tqdm import tqdm

x = torch.randn((7, 3, 256, 256), device='cuda')
start = torch.cuda.Event(enable_timing=True)
end = torch.cuda.Event(enable_timing=True)

start.record()
for i in enumerate(tqdm(range(10000))):
  F.interpolate(x, (64, 64), mode='bilinear')
end.record()
torch.cuda.synchronize()
print(start.elapsed_time(end))
# 310.09381103515625

start.record()
for i in enumerate(tqdm(range(10000))):
  F.interpolate(x, (64, 64), mode='bilinear', antialias=True)
end.record()
torch.cuda.synchronize()
print(start.elapsed_time(end))
# 1558.083984375
```

### Versions

```
Collecting environment information...
PyTorch version: 1.12.0+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
CMake version: version 3.22.5
Libc version: glibc-2.26

Python version: 3.7.13 (default, Apr 24 2022, 01:04:09)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
Is CUDA available: True
CUDA runtime version: 11.1.105
GPU models and configuration: GPU 0: Tesla T4
Nvidia driver version: 460.32.03
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.0.5
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.6
[pip3] torch==1.12.0+cu113
[pip3] torchaudio==0.12.0+cu113
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.13.0
[pip3] torchvision==0.13.0+cu113
[conda] Could not collect

```

cc @VitalyFedyunin @ngimel
masked_scatter_ is very lacking
Labels: module: docs, triaged, module: scatter & gather ops

### 📚 The doc issue

The documentation for `masked_scatter_` says

```
Tensor.masked_scatter_(mask, source):
Copies elements from source into self tensor at positions where the mask is True. 
```

Which suggests it might work similar to

```
tensor[mask] = source[mask]
```

however, the actual semantics are quite different, which leads to confusion as evidenced here: https://stackoverflow.com/questions/68675160/torch-masked-scatter-result-did-not-meet-expectations/73145411#73145411

### Suggest a potential alternative/fix

The docs should specify that the way scatter works is that the `i`th `True` in a row is given the `i`th value from the source. So not the value corresponding to the position of the `True`.
Maybe an example of the difference between the two methods described in the issue.


cc @svekars @holly1238 @mikaylagawarecki
Offer a way to really force merges via pytorchbot
Labels: module: ci, triaged

### 🚀 The feature, motivation and pitch

I had a fun situation in https://github.com/pytorch/pytorch/pull/82236. Here's the summary:
- I created PR C, and then imported it into fbcode to check test signal.
- Next, I created PRs A, B, D, which were stacked like the following: A <- B <- C <- D
- Then, I merged A and B via GHF
- PR C refused to merge via GHF due to "This PR has internal changes and must be landed via Phabricator". 

We tried a couple of things:
1. resync the PR to fbcode. The resync failed because commits A and B had not yet made it into fbcode
2. unlink the PR from the fbcode diff and then attempting land via GHF. The unlink didn't seem to do anything.

What I described above probably isn't a common workflow, but it would be nice to have a "ignore all checks and force merge" button for GHF for the cases where tooling didn't work out

cc @seemethere @malfet @pytorch/pytorch-dev-infra 

### Alternatives

Alternatively, using "unlink" on a PR should make it so that this check doesn't appear anymore. This was not the case for https://github.com/pytorch/pytorch/pull/82236 -- after unlink, the PR still could not be merged.

### Additional context

_No response_
[JIT] SchemaInfo warning appears out in the wild
Labels: oncall: jit

### 🐛 Describe the bug

![294585988_473395204623132_7838843822650475571_n](https://user-images.githubusercontent.com/33938358/181291450-5dbbb407-f929-40d7-84da-ad2d817f90c4.png)
^ Warning for SchemaInfo having multiple input arguments with the same alias set appears in the wild when there shouldn't be any ops that trigger this warning. 

### Versions

n/a
test_make_fx_symbolic_exhaustive should pass dynamic ints for shape arguments
Labels: triaged, fx, module: dynamic shapes

### 🐛 Describe the bug

```
diff --git a/test/test_proxy_tensor.py b/test/test_proxy_tensor.py
index 9e22de81529..f3809fc22d9 100644
--- a/test/test_proxy_tensor.py
+++ b/test/test_proxy_tensor.py
@@ -844,6 +844,9 @@ def _test_make_fx_helper(self, device, dtype, op, tracing_mode):
         except DynamicOutputShapeException as e:
             self.skipTest("Dynamic output shape operation in trace")
 
+        print(new_f)
+        print(new_f.shape_env.guards)
+
         for arg in args:
             if isinstance(arg, torch.Tensor) and arg.dtype == torch.float:
                 arg.uniform_(0, 1)
```

then

```
python test/test_proxy_tensor.py -k test_make_fx_symbolic_exhaustive_new_zeros_cpu_float32
```

gives, for example, 

```
def forward(self, args, kwargs):
    args_1, args_2, args_3, args_4, kwargs_1, kwargs_2, = fx_pytree.tree_flatten_spec([args, kwargs], self._in_spec)
    size = args_1.size(0)
    _tensor_constant0 = self._tensor_constant0
    device_1 = torch.ops.prim.device.default(_tensor_constant0);  _tensor_constant0 = None
    new_zeros_default = torch.ops.aten.new_zeros.default(args_1, [2, 2, 2], dtype = torch.float64, layout = torch.strided, device = device(type='cpu'), pin_memory = False);  args_1 = None
    return pytree.tree_unflatten([new_zeros_default], self._out_spec)
    
[]
```

the call to `new_zeros` is clearly specialized but there are no guards.

cc @ezyang @SherlockNoMad @Chillee 

### Versions

master
Add more Vulkan operations
Labels: triaged, module: vulkan, ciflow/periodic

### 🚀 The feature, motivation and pitch

I got an error while trying to host my model into mobile using Vulkan
`Could not run aten::max_pool2d_with_indices`

And I found the list of operations of vulkan quite short.

### Alternatives

_No response_

### Additional context

_No response_

cc @jeffdaily @sunway513
`torch.matrix_exp` doesn't handle NaN properly
Labels: module: cuda, triaged, module: NaNs and Infs, module: linear algebra

### 🐛 Describe the bug

When calling `torch.matrix_exp`  on a value containing `nan` values, the result is strange and different on CPU and on CUDA with the same input.

Based on similar discussion in https://github.com/pytorch/pytorch/issues/61251 , I think `torch.matrix_exp` should also check input for `nan`.

```
import torch
torch.random.manual_seed(420)
input = torch.randn(3, 3, dtype=torch.float32)
print("Intermediate: ", torch.log(input * 2 - 1)) # Contains nan
output = torch.matrix_exp(torch.log(input * 2 - 1))
print("cpu output: ", output)
input = input.cuda()
output = torch.matrix_exp(torch.log(input * 2 - 1))
print("gpu output: ", output) 
```

Outputs:
```
Intermediate:  tensor([[    nan, -1.2915,     nan],
        [    nan,  0.7102,     nan],
        [-0.3755,     nan,     nan]])
cpu output:  tensor([[ 7.2619e+11,  0.0000e+00, -8.4379e-01],
        [-1.8280e+00,  2.0343e+00, -9.0547e-01],
        [ 6.8693e-01, -1.4521e+00, -9.3093e-01]])
gpu output:  tensor([[-4.3953,  0.2749, -0.8438],
        [-1.8280,  2.0343, -0.9055],
        [ 0.6869, -1.4521, -0.9309]], device='cuda:0')
```

### Versions

pytorch: 1.12.0

cc @ngimel @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano
DEBUG=1 env var doesn't actually set DEBUG preprocessor macro
Labels: module: build, triaged, enhancement

### 🐛 Describe the bug

What it says on the tin. There are a lot of uses of this dead macro.

```
aten/src/ATen/BatchedTensorImpl.cpp:#ifdef DEBUG
aten/src/ATen/BatchedTensorImpl.h:#ifdef DEBUG
aten/src/ATen/OpaqueTensorImpl.h:#ifdef DEBUG
aten/src/ATen/SparseTensorImpl.cpp:#ifdef DEBUG
aten/src/ATen/SparseTensorImpl.h:#ifdef DEBUG
aten/src/ATen/core/dispatch/Dispatcher.h:#ifdef DEBUG
aten/src/ATen/core/dispatch/Dispatcher.h:#ifdef DEBUG
aten/src/ATen/core/dispatch/Dispatcher.h:#ifdef DEBUG
aten/src/ATen/core/dispatch/Dispatcher.h:#ifdef DEBUG
aten/src/ATen/native/Resize.h:#ifdef DEBUG
aten/src/ATen/native/vulkan/api/Allocator.h:#ifdef DEBUG
c10/core/TensorImpl.h:#ifdef DEBUG
c10/core/TensorImpl.h:#ifdef DEBUG
c10/core/TensorImpl.h:#ifdef DEBUG
c10/core/TensorImpl.h:#ifdef DEBUG
c10/core/UndefinedTensorImpl.cpp:#ifdef DEBUG
c10/core/UndefinedTensorImpl.h:#ifdef DEBUG
c10/util/Registry.h:#ifdef DEBUG
functorch/functorch/csrc/BatchedTensorImpl.cpp:#ifdef DEBUG
functorch/functorch/csrc/BatchedTensorImpl.h:#ifdef DEBUG
```

cc @malfet @seemethere @zou3519 @bdhirsh 

### Versions

master
[Reproducibility] Make tests say when unusual environment variables are set that change behavior of the test
Labels: module: ci, triaged

### 🐛 Describe the bug

E.g. as seen in https://github.com/pytorch/pytorch/pull/81999 developers don't necessarily know how to apply environment variables in unconventional test configurations to reproduce CI failures. The test error messages should detect and report when an envvar is necessary to reproduce the behavior.

Currently relevant envvars:

```
PYTORCH_TEST_WITH_SLOW
PYTORCH_TEST_WITH_CROSSREF
PYTORCH_TEST_WITH_DYNAMO
```

### Versions

master

cc @seemethere @malfet @pytorch/pytorch-dev-infra
logspace inconsistently casts inputs to int before performing computation
Labels: triaged, module: tensor creation

When `steps = 1`, the inputs are not casted to integers before performing the pow:
```python
>>> torch.logspace(4.3, 5, 2, dtype=torch.int32)
tensor([ 10000, 100000], dtype=torch.int32)
>>> torch.logspace(4.3, 5, 1, dtype=torch.int32)
tensor([19952], dtype=torch.int32)
>>> torch.logspace(4.3, 5, 1, dtype=torch.int32, device="cuda")
tensor([19952], device='cuda:0', dtype=torch.int32)
>>> torch.logspace(4.3, 5, 2, dtype=torch.int32, device="cuda")
tensor([ 10000, 100000], device='cuda:0', dtype=torch.int32)
```

cc @gchanan @mruberry
primtorch refs should be composite compliant
Labels: triaged, module: __torch_dispatch__, tensor subclass, module: primTorch

## Issue description

Primtorch refs should be composite compliant. That is, given a ref for an operator like `new_empty`, if the input to new_empty is a Tensor subclass, then the output should also be a Tensor subclass. Otherwise, the ref doesn't capture the semantics of the operation and cannot be used as a "decomposition" for a Tensor Subclass.

We should add explicit tests that refs are composite compliant.

## Code example

E.g. https://github.com/pytorch/pytorch/pull/82237




cc @Chillee @ezyang @zou3519 @albanD @samdow @mruberry @ngimel @kshitij12345
logspace and linspace off by one on cuda for integer dtypes for some inputs
Labels: triaged, module: tensor creation

As expected,
```python
>>> torch.logspace(1., 2., 2, dtype=torch.int32, device="cpu")
tensor([ 10, 100], dtype=torch.int32)
>>> torch.logspace(1., 2., 2, dtype=torch.float32, device="cuda")
tensor([ 10.0000, 100.0000], device='cuda:0')
```
But when output dtype is specified to be int:
```python
>>> torch.logspace(1., 2., 2, dtype=torch.int32, device="cuda")
tensor([10, 99], device='cuda:0', dtype=torch.int32)
```

Off-by-one also appears when casting is performed after - but only happens on float32.
```python
>>> torch.logspace(1., 2., 2, 10, dtype=torch.float16, device="cuda").to(dtype=torch.int32)
tensor([ 10, 100], device='cuda:0', dtype=torch.int32)
>>> torch.logspace(1., 2., 2, dtype=torch.float32, device="cuda").to(dtype=torch.int32)
tensor([10, 99], device='cuda:0', dtype=torch.int32)
>>> torch.logspace(1., 2., 2, dtype=torch.float64, device="cuda").to(dtype=torch.int32)
tensor([ 10, 100], device='cuda:0', dtype=torch.int32)
```
Does not happen consistently on all inputs
```python
tensor([ 9, 81], device='cuda:0', dtype=torch.int32)
>>> torch.logspace(1., 2., 2, 10, dtype=torch.float32, device="cuda").to(dtype=torch.int32)
tensor([10, 99], device='cuda:0', dtype=torch.int32)
>>> torch.logspace(1., 2., 2, 11, dtype=torch.float32, device="cuda").to(dtype=torch.int32)
tensor([ 11, 121], device='cuda:0', dtype=torch.int32)
```
I would expect the produced binary representation of the output to be greater than the integral value so that when casted it does not produce a large discrepancy with the before-casted value.

cc @gchanan @mruberry
[Profiler] Allow profiler to gracefully fail without interrupting workflow.
Labels: oncall: profiler

### 🚀 The feature, motivation and pitch

The pytorch profiler works very hard to minimize work on the hot path to reduce overhead. However, one of the consequences of this choice is that there is a fairly significant post processing step to convert these raw events into a coherent program representation. To ensure correctness this post processing makes liberal use of asserts. However this is causing a non-trivial number of post processing calls to fail and as a result crashes the entire workload. Which is not ideal. How we want to handle this depends on the context. In debug mode it should be a hard failure. In opt mode we should emit a warning (to console in OSS and to a remote logger in FBCode), record it in the trace file, and gracefully drop the offending event or do whatever else is needed to return to a somewhat sane state. (E.g. drop the event)

As a concrete example, we currently have a guard to ensure that an event produces sane timestamps before marking it finished and incorporating it in the tree:
```
void mark_finished(std::shared_ptr<Result>& r) {
  TORCH_INTERNAL_ASSERT(!r->finished_, r->name());
  r->finished_ = true;
  TORCH_INTERNAL_ASSERT(r->endTimeNS() >= r->start_time_ns_, r->name());
}
```
However corner cases (such as is asynchronously stopped when an op is in progress) breaks this. We want to correct the logic, but it's not necessary to crash an entire run. A better structure would be something like:
```
if (SOFT_ASSERT(!r->finished_, r->name()) {
  r->finished_ = true;
  return SOFT_ASSERT(r->endTimeNS() >= r->start_time_ns_, r->name());
}
return false;
```
And then the caller should act appropriately.

This will consist of two parts:
1) Implement `SOFT_ASSERT`. (Working title)
2) Update profiler to gracefully recover from invariant violations.

### Alternatives

_No response_

### Additional context

_No response_

cc @ilia-cher @robieta @chaekit @gdankel @bitfort @ngimel @nbcsm @guotuofeng @guyang3532 @gaoteng-git
[feature request] DataLoader to accept num_threads argument to auto-set number of threads for OpenMP / intra-op parallelism
Labels: module: dataloader, triaged, module: openmp

### 🚀 The feature, motivation and pitch

In the typical single-node multi-gpu DDP + multi-threaded DataLoader + intra-op OpenMP parallelism, there might be CPU thread oversubscription leading to slower perf compared to no intra-op parallelism.

Probably a better choice would be to support a helper argument for DataLoader that would call torch.set_num_threads at thread init. Currently this is possible via custom worker_init_fn, but having it as constructor argument would be simpler and bring more visibility.

Currently there is a debatable workaround: DDP sets OMP_NUM_THREADS=1 by default: https://github.com/pytorch/pytorch/blob/master/torch/distributed/run.py#L653 assuming CPU intra-op parallelism almost doesn't happen at all (and not only in DataLoader) - this is rather strange and can be unexpected



### Alternatives

_No response_

### Additional context

_No response_

cc @SsnL @VitalyFedyunin @ejguan @NivekT