gopikrsmscs/torch-issues huggingface dataset:
Documentation and typing hints for RProp
Labels: module: docs, module: optimizer, triaged, actionable

It seems that the RProp's `foreach` parameter is not well documented, and also the `rprop.pyi` file does not reflect the additional `foreach` parameter available in `rprop.py`.

https://github.com/pytorch/pytorch/blob/3a2cfbb813e19c1648b23079f704829f9997425d/torch/optim/rprop.pyi#L5

Seems that `foreach` controls whether to use `_multi_tensor_rprop` or `_single_tensor_rprop`. But their difference isn't obvious to me. My guess is that for the "multi" case, each parameter has its own $\eta_t$?

cc @svekars @holly1238 @vincentqb @jbschlosser @albanD
"upsample_nearest2d_out_frame" not implemented for 'BFloat16'
Labels: triaged, module: bfloat16, module: interpolation

### 🐛 Describe the bug

Nearest upsampling with `torch.nn.functional.interpolate` does not work in `bfloat16`. Minimal code to reproduce.

```
import torch
import torch.nn.functional as F

image = torch.randn(1, 4, 32, 32).to(device="cuda", dtype=torch.bfloat16)
out = F.interpolate(image, size=(64, 64), mode="nearest")
```

This throws an error

```
File ~/.pyenv/versions/3.9.14/envs/diffusers-env/lib/python3.9/site-packages/torch/nn/functional.py:3910, in interpolate(input, size, scale_factor, mode, align_corners, recompute_scale_factor, antialias)
   3908     return torch._C._nn.upsample_nearest1d(input, output_size, scale_factors)
   3909 if input.dim() == 4 and mode == "nearest":
-> 3910     return torch._C._nn.upsample_nearest2d(input, output_size, scale_factors)
   3911 if input.dim() == 5 and mode == "nearest":
   3912     return torch._C._nn.upsample_nearest3d(input, output_size, scale_factors)

RuntimeError: "upsample_nearest2d_out_frame" not implemented for 'BFloat16'
```

`F.interpolate` with `nearest` mode is used a lot unets which are the backbone diffusion models like stable diffusion. Due to this at the moment it's not possible to use Stable Diffusion with `bfloat16` without manual casting. cf https://github.com/huggingface/diffusers/pull/792

### Versions

```
PyTorch version: 1.12.1+cu116
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux 10 (buster) (x86_64)
GCC version: (Debian 8.3.0-6) 8.3.0
Clang version: Could not collect
CMake version: version 3.13.4
Libc version: glibc-2.28

Python version: 3.9.14 (main, Sep 22 2022, 15:50:51)  [GCC 8.3.0] (64-bit runtime)
Python platform: Linux-4.19.0-22-cloud-amd64-x86_64-with-glibc2.28
Is CUDA available: True
CUDA runtime version: 11.0.221
GPU models and configuration:
GPU 0: NVIDIA A100-SXM4-40GB
GPU 1: NVIDIA A100-SXM4-40GB

Nvidia driver version: 510.47.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.12.1+cu116
[pip3] torchaudio==0.12.1+cu116
[pip3] torchvision==0.13.1+cu116
[conda] numpy                     1.19.5           py37h3e96413_3    conda-forge
```
Pytorch built for Jetson errors if CUDA is not found
Labels: module: build, module: cuda, triaged, module: arm

### 🐛 Describe the bug

I've been building torch wheel from source for arm/jetson using instructions here: https://forums.developer.nvidia.com/t/pytorch-for-jetson/72048

It works fine on Jetson, but it doesn't work on ARM machine without Nvidia GPU:

```
(py39) [ubuntu@arm-dev ~]: python
Python 3.9.13 | packaged by conda-forge | (main, May 27 2022, 17:32:26)
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/.miniforge3/envs/py39/lib/python3.9/site-packages/torch/__init__.py", line 196, in <module>
    _load_global_deps()
  File "/home/ubuntu/.miniforge3/envs/py39/lib/python3.9/site-packages/torch/__init__.py", line 149, in _load_global_deps
    ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
  File "/home/ubuntu/.miniforge3/envs/py39/lib/python3.9/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libcurand.so.10: cannot open shared object file: No such file or directory
```

For the sake of the simplicity, it'd be useful if we can have one single wheel that works seamlessly b/w CPU and GPU on arm servers. Like how it does for x86.

@albanD suggests that It looks like our main shared library has a dependency on some CUDA stuff (which it shouldn't).

### Versions

Here's the env from jetson TX2. `import torch` works fine here but not on ARM server without cuda.

```
(base) [qure@orbitty ssd]: python collect_env.py 
Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (aarch64)
GCC version: (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: version 3.10.2
Libc version: glibc-2.27

Python version: 3.10.6 | packaged by conda-forge | (main, Aug 22 2022, 20:27:34) [GCC 10.4.0] (64-bit runtime)
Python platform: Linux-4.9.140-tegra-aarch64-with-glibc2.27
Is CUDA available: N/A
CUDA runtime version: 10.2.89
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/aarch64-linux-gnu/libcudnn.so.8.0.0
/usr/lib/aarch64-linux-gnu/libcudnn_adv_infer.so.8.0.0
/usr/lib/aarch64-linux-gnu/libcudnn_adv_train.so.8.0.0
/usr/lib/aarch64-linux-gnu/libcudnn_cnn_infer.so.8.0.0
/usr/lib/aarch64-linux-gnu/libcudnn_cnn_train.so.8.0.0
/usr/lib/aarch64-linux-gnu/libcudnn_ops_infer.so.8.0.0
/usr/lib/aarch64-linux-gnu/libcudnn_ops_train.so.8.0.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[conda] No relevant packages
```

cc @malfet @seemethere @ngimel
[TorchInductor] Add support for Pascal GPUs (P100, GTX 1080, etc)
Labels: triaged, oncall: pt2

According to @ptillet supporting Pascal GPUs with Triton should be possible if we:
1) Disable float16 support (and give a good error message saying fp16 requires a newer GPU)
2) Use `aten` backend for matmul/conv
3) May need to remove an assert on the Triton side?
4) Test it to make sure things work

This might be a good first issue if someone has a Pascal GPU handy to test on.

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
RFC(from users): nn.Module behavior with in-place changes
Labels: triaged, needs design, module: functorch

## Background
As a part of [functorch rationalization](https://docs.google.com/document/d/1vL9mjHKeRR-uzBazt4W-N4NftqR5I1IDlszHUUanAIw/edit#heading=h.gngvuf6wr2h2) (integrating functorch into pytorch/pytorch), we're considering how to consolidate [`make_functional`](https://pytorch.org/functorch/stable/generated/functorch.make_functional.html?highlight=make_functional#functorch.make_functional) from functorch with [`stateless.functional_call`](https://pytorch.org/docs/stable/generated/torch.nn.utils.stateless.functional_call.html?highlight=functional_call#torch.nn.utils.stateless.functional_call). We weren't certain what the expected behavior here is and wanted to get some user feedback

## What should the following produce?
Consider that we have some model Foo that updates the number of iterations it has run whenever the forward is called
```python
class Foo(nn.Module):
  def __init__(self):
    self.iter = 0
    self.weight = torch.randn(3)

  def forward(self, x):
    self.iter += 1
    out = x * self.weight
    return out.sum()
```

### Question 1
In this example, should the last line print out "has run 0 time(s)" or "has run 1 time(s)"?
```python
model = Foo()
model_fn, params = make_functional(model)

# note since the API might change, please ignore what this currently does in practice 
# and just share what you would expect
#
# here we get the grad wrt parameters of running with a random input
grad(model_fn, argnums=0)(params, torch.randn(3))

print(f"has run {model.iter} time(s)")
```

### Question 2
Does it change anything if iter is a tensor? Should we treat ints differently?
```python
class Foo(nn.Module):
  def __init__(self):
    self.iter = torch.tensor([0])
    self.weight = torch.randn(3)

  def forward(self, x):
    self.iter.add_(1)
    out = x * self.weight
    return out.sum()

model_fn, params, buffers = make_functional_with_buffers(model)
grad(model_fn, argnums=0)(params, buffers, torch.randn(3))

print(f"has run {model.iter.item()} time(s)")
```

## Ask
As users of functorch, it would be great to know your thoughts on either or both questions (partial answers are still valuable)!

cc @zou3519 @Chillee @soumith @ain-soph (if you're still willing to give feedback on the nn API)
Move functorch tests from functorch/test/* to test/*; delete functorch CI configs
Labels: module: ci, triaged, module: functorch

## Issue description

We're at the tail end of moving functorch code out of the functorch/ folder and into the pytorch directory structure. One of the last remaining items is to move the functorch tests from `functorch/test/*` to `test/functorch/*`. The main motivation for this change is that pytorch testing infrastructure generally assumes that PyTorch tests are under `test/*`.

In addition to moving `functorch/test/*` to under `test/*`, we are also proposing deleting the functorch CI configurations and running functorch tests as a part of regular PyTorch CI tests. functorch is no longer a separate library from PyTorch and the tests for its features should run wherever PyTorch core tests run. Concretely, this would involve deleting the functorch shards and increasing the shard count for PyTorch configs to maintain TTS if appropriate.

I'm opening this issue to ask for feedback from the pytorch-dev-infra folks and if there are any other considerations; cc @seemethere @malfet @pytorch/pytorch-dev-infra @Chillee @samdow @soumith and folks we've worked with around this (@malfet, @huydhn, @janeyx99).
Decomposition table is ignored with use_functionalize=True in AOT Autograd
Labels: triaged, module: functorch, module: aotdispatch

### 🐛 Describe the bug

```py
import torch
from functorch.compile import aot_function
from functorch.compile import make_boxed_compiler
import functorch
functorch.compile.config.use_functionalize = True # default

def func(a):
    return torch.nn.functional.silu(a)

def raise_error(*args):
    raise RuntimeError("Expected error")

d = {torch.ops.aten.silu_backward.default: raise_error}

def fw_compiler(gm, args):
    return gm

aot_fn = aot_function(func, fw_compiler=make_boxed_compiler(fw_compiler), decompositions=d)
a = torch.randn(3, 3, device="cuda", requires_grad=True)

try:
    aot_fn(a).backward(torch.ones_like(a))
    print("No error!")
except RuntimeError as e:
    print(e)
```
```
No error!
```
Setting `functorch.compile.config.use_functionalize = False` works as expected (specified table is used).

### Versions

Latest master.

cc @zou3519 @Chillee @samdow @soumith
Nonoptimal trace of silu_backward with AOT Autograd
Labels: triaged, module: functorch, module: aotdispatch

### 🐛 Describe the bug

The backward pass of `silu` creates an unnecessary tensor full of ones to do subtraction:
```py
import torch
from functorch.compile import aot_function
from functorch.compile import make_boxed_compiler
from functorch.compile import get_aot_graph_name

def func(a):
    return torch.nn.functional.silu(a)

def fw_compiler(gm, args):
    print(get_aot_graph_name())
    gm.graph.print_tabular()
    return gm

aot_fn = aot_function(func, fw_compiler=make_boxed_compiler(fw_compiler))
a = torch.randn(3, 3, device="cuda", requires_grad=True)

aot_fn(a).backward(torch.ones_like(a))
```
Output:
```py
model_forward_0
opcode         name       target             args                  kwargs
-------------  ---------  -----------------  --------------------  --------
placeholder    primals_1  primals_1          ()                    {}
call_function  silu       aten.silu.default  (primals_1,)          {}
output         output     output             ([silu, primals_1],)  {}
model_backward_0
opcode         name        target                   args                 kwargs
-------------  ----------  -----------------------  -------------------  ----------------------------------------
placeholder    primals_1   primals_1                ()                   {}
placeholder    tangents_1  tangents_1               ()                   {}
call_function  sigmoid     aten.sigmoid.default     (primals_1,)         {}
call_function  empty_like  aten.empty_like.default  (sigmoid,)           {'memory_format': torch.preserve_format}
call_function  fill        aten.fill.Scalar         (empty_like, 1)      {}
call_function  sub         aten.sub.Tensor          (fill, sigmoid)      {}
call_function  mul         aten.mul.Tensor          (primals_1, sub)     {}
call_function  add         aten.add.Scalar          (mul, 1)             {}
call_function  mul_1       aten.mul.Tensor          (sigmoid, add)       {}
call_function  mul_2       aten.mul.Tensor          (tangents_1, mul_1)  {}
output         output      output                   ([mul_2],)           {}
```
Here's the C++ code:
https://github.com/pytorch/pytorch/blob/4a5fdc56ec692fe5e39b8f5d2da6be16434c5a02/aten/src/ATen/native/Activation.cpp#L487
`1 - input_sigmoid` is recorded as a sequence of empty_like, fill, and sub.

### Versions

Latest master.

cc @zou3519 @Chillee @samdow @soumith @ezyang
NVFuser batch norm with prims: internal assert failure from test suite
Labels: module: tests, triaged, module: nvfuser, module: primTorch

I noticed that when I run the following test locally:
```
python test/test_ops.py TestCommonCUDA.test_python_ref_executor_ops_nvprims_native_batch_norm_executor_nvfuser_cuda_float32
```

I get this error:
```
  File "/raid/hirsheybar/pytorch/torch/_prims/nvfuser_executor.py", line 205, in nvfuser_execute
    fusion, unflatten_spec = make_nvfuser_fusion(gm, *nv_template_args)  # type: ignore[misc]
  File "/raid/hirsheybar/pytorch/torch/_prims/nvfuser_executor.py", line 178, in make_nvfuser_fusion
    fd.add_output(o)
RuntimeError: downcast_ptr != nullptr INTERNAL ASSERT FAILED at "../torch/csrc/jit/codegen/cuda/utils.h":136, please report a bug to PyTorch.
```

I'm not sure why that test isn't failing in CI - one possibility is that I have a debug build locally, and the failure only manifests in a debug build.

cc @mruberry @ezyang @ngimel @Lezcano @fdrocha
`squeeze_` fails with JIT but succeeds without it
Labels: oncall: jit

### 🐛 Describe the bug

`squeeze_` fails with JIT but succeeds without it

```py
import torch

def fn(input):
    return input.squeeze_(dim=3)
input = torch.empty([5, 1, 5, 1], dtype=torch.float64, device='cpu')
input.uniform_(-64, 127)

print(fn(input.clone()))

jit_fn = torch.jit.trace(fn, input.clone())
```

```
tensor([[[ 57.4385,  -4.1074, -42.2959, -59.3506, -37.6112]],

        [[ -1.8995, 122.6732, -19.9084, 105.3027,  46.4563]],

        [[ 39.0169, -47.2261, -43.9141, -56.1353,  71.8304]],

        [[106.2245,  -1.1742,  27.4605,  41.4658,  82.8431]],

        [[119.8366,  66.7670,   5.1189,  17.5239, -60.9487]]],
       dtype=torch.float64)

IndexError: Dimension out of range (expected to be in range of [-3, 2], but got 3)
```

### Versions

```
PyTorch version: 1.13.0.dev20221001
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.9.13 (main, Aug 25 2022, 23:26:10)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.19.5-051905-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.99
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3060
Nvidia driver version: 515.65.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.13.0.dev20221001
[pip3] torchaudio==0.13.0.dev20221001
[pip3] torchvision==0.14.0.dev20221001
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py39h7f8727e_0  
[conda] mkl_fft                   1.3.1            py39hd3c417c_0  
[conda] mkl_random                1.2.2            py39h51133e4_0  
[conda] numpy                     1.23.3                   pypi_0    pypi
[conda] numpy-base                1.23.1           py39ha15fc14_0  
[conda] pytorch                   1.13.0.dev20221001 py3.9_cuda11.7_cudnn8.5.0_0    pytorch-nightly
[conda] pytorch-cuda              11.7                 h67b0de4_0    pytorch-nightly
[conda] pytorch-mutex             1.0                        cuda    pytorch-nightly
[conda] torch                     1.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchaudio                0.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchvision               0.14.0.dev20220923+cu116          pypi_0    pypi
```
`CTCLoss` returns a different value with JIT on cuda
Labels: oncall: jit, module: cuda, module: correctness (silent)

### 🐛 Describe the bug

`CTCLoss` returns a different value with JIT on cuda

```py
import torch

torch.random.manual_seed(3564)
def get_fn():
    arg_class = torch.nn.CTCLoss()
    inp1 = torch.randint(-16, 1, [16, 30], dtype=torch.int64, device='cuda')
    inp2 = torch.randint(-4, 2, [16], dtype=torch.int64, device='cuda')
    inp3 = torch.randint(-1, 1, [16], dtype=torch.int64, device='cuda')
    def fn(inp):
        fn_res = arg_class(inp, inp1, inp2, inp3)
        return fn_res
    return fn

fn = get_fn()
inp = torch.empty([1, 16, 20], dtype=torch.float32)
inp.uniform_(-32, 15)
inp_a = inp.clone().to('cuda')
inp_b = inp.clone().to('cuda')
inp_c = inp.clone().to('cuda')

res = fn(inp_a)

jit_fn = torch.jit.trace(fn, (inp_b, ))
jit_res = jit_fn(inp_c)

print(res) # tensor(12.9426, device='cuda:0')
print(jit_res) # tensor(14.2359, device='cuda:0')
```

### Versions

```
PyTorch version: 1.13.0.dev20221001
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.9.13 (main, Aug 25 2022, 23:26:10)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.19.5-051905-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.99
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3060
Nvidia driver version: 515.65.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.13.0.dev20221001
[pip3] torchaudio==0.13.0.dev20221001
[pip3] torchvision==0.14.0.dev20221001
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py39h7f8727e_0  
[conda] mkl_fft                   1.3.1            py39hd3c417c_0  
[conda] mkl_random                1.2.2            py39h51133e4_0  
[conda] numpy                     1.23.3                   pypi_0    pypi
[conda] numpy-base                1.23.1           py39ha15fc14_0  
[conda] pytorch                   1.13.0.dev20221001 py3.9_cuda11.7_cudnn8.5.0_0    pytorch-nightly
[conda] pytorch-cuda              11.7                 h67b0de4_0    pytorch-nightly
[conda] pytorch-mutex             1.0                        cuda    pytorch-nightly
[conda] torch                     1.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchaudio                0.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchvision               0.14.0.dev20220923+cu116          pypi_0    pypi
```

cc @ngimel
JIT model with `relu+div+sgn` will crash when computing the gradient
Labels: oncall: jit, module: crash, module: functorch

### 🐛 Describe the bug

JIT model with `relu+div+sgn` will crash (segmentation fault (core dumped)) when computing the gradient by using `torch.autograd.functional.jacobian` after running the model

```py
import torch

class M(torch.nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, input):
        input = torch.nn.functional.relu(input)
        input = torch.div(input, torch.tensor(2, dtype=torch.float32, device='cpu'))
        fn_res = torch.sgn(input, )
        return fn_res

fn = M().to('cpu')
torch.random.manual_seed(47202)
input = torch.rand([5, 5, 5], dtype=torch.float64)

jit_fn = torch.jit.trace(fn, input)

from torch.autograd.functional import jacobian

jit_fn(input.clone().requires_grad_())
jacobian(jit_fn, input.clone().requires_grad_())
```

```
segmentation fault (core dumped)
```

Interestingly, it will not crash if we directly call the `jacobian` without running the model at first

### Versions

```
PyTorch version: 1.13.0.dev20221001
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.9.13 (main, Aug 25 2022, 23:26:10)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.19.5-051905-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.99
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3060
Nvidia driver version: 515.65.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.13.0.dev20221001
[pip3] torchaudio==0.13.0.dev20221001
[pip3] torchvision==0.14.0.dev20221001
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py39h7f8727e_0  
[conda] mkl_fft                   1.3.1            py39hd3c417c_0  
[conda] mkl_random                1.2.2            py39h51133e4_0  
[conda] numpy                     1.23.3                   pypi_0    pypi
[conda] numpy-base                1.23.1           py39ha15fc14_0  
[conda] pytorch                   1.13.0.dev20221001 py3.9_cuda11.7_cudnn8.5.0_0    pytorch-nightly
[conda] pytorch-cuda              11.7                 h67b0de4_0    pytorch-nightly
[conda] pytorch-mutex             1.0                        cuda    pytorch-nightly
[conda] torch                     1.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchaudio                0.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchvision               0.14.0.dev20220923+cu116          pypi_0    pypi
```

cc @zou3519 @Chillee @samdow @soumith
Easy way to "freeze" BatchNorm running_mean/running_var
Labels: module: nn, triaged, needs research

### 🚀 The feature, motivation and pitch

In most transfer learning applications, it is often useful to freeze some layers of the CNN (e.g. resnet50) up to the last convolutional layers in order to train only the last layers. Usually, I simply set requires_grad=False to all parameters in a simple for loop:
for param in net.parameters():
  param.requires_grad = False

This way, I can alternate freely between model.train() and model.eval(), knowing which weights will get updated. However, since batchnorm parameters (running_mean/running_var) are not updated during backprop, the requires_grad attribute set on them is pretty much ignored, contrary to the training attribute of the layer which is used to compute bn_training => after training, I end up with a model where all batchnorm params have changed.

Here's my request: could bn_training somehow also depend on the requires_grad attribute of running_mean/running_var, i.e. something like:

if self.training and self.running_mean.requires_grad and self.running_var.requires_grad:
  bn_training = True
else:
  bn_training = (self.running_mean is None) and (self.running_var is None)

Note that I've seen a lot of codes where people don't know that their feature extractor is not entirely frozen, which can lead to a lot of incorrect/non reproducible results in research papers for example.


### Alternatives

_No response_

### Additional context

_No response_

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
Instructions for Selective Build for Mobile Linux Platform
Labels: oncall: mobile

### 📚 The doc issue

1. The docs. at https://pytorch.org/tutorials/prototype/tracing_based_selective_build.html include instructions for iOS/Android, but not for Linux. Please could you add instructions for Linux?
2. The instructions on that page don't include a call to `optimize_for_mobile()`. Is it a required step or can selective build work without `optimize_for_mobile()`?
3. I also want to know a quick way to be able to check the approximate size change for my PyTorch build for the set of models I want to work with - is there any way to do that w/o doing a complete selective build?

The page above says:

> The custom build is still under development, and we will continue improving its size in the future. Note, however, that the APIs are subject to change in future versions.

4. What other size improvements can be expected in the future?


### Suggest a potential alternative/fix

1. Inclde steps for Linux based build
2. Clarofy if `optimize_for_mobile()` is a required step and what it does - I can't find documentation about this
3. ??
4. ??
Data conversion ops ignore `memory_format=torch.contiguous_format` 
Labels: triaged, module: primTorch

### 🐛 Describe the bug

**Bug**: a _contiguous_ format is requested for a _non-contiguous_ input, but a _non-contiguous view_ is returned.

Likely this is the same for all data conversion ops:
`bool`, `bfloat16`, `byte`, `char`, `double`, `float`, `half`, `int`, `long`, `short`.

Probably only happens when input dtype matches the conversion op (like `torch.bool` and `.bool()`).

**Expected**: a _contiguous copy_ is returned instead. Note: in primTorch, `to` returns a  _contiguous copy_ in this case.

```py
#!/usr/bin/env python3

import torch

input = torch.tensor([[True, False, False], [False, False, False]])
args = ()
kwargs = {'memory_format': torch.contiguous_format}

print("is contiguous?", input.is_contiguous())
input = input.t()
print("is contiguous after transpose?", input.is_contiguous())
print("is view?", input._is_view())

op=lambda x, *args, **kwargs: x.bool(*args, **kwargs)

torch_result = op(input, *args, **kwargs)

print()
print("torch is view?", torch_result._is_view())
print("torch is contiguous?", torch_result.is_contiguous())
```

Output:
```
is contiguous? True
is contiguous after transpose? False
is view? True

torch is view? True
torch is contiguous? False
```



### Versions

master (67434c70df5df353944f6ba876d9dd06b669bacd)

cc @ezyang @mruberry @ngimel @Lezcano @fdrocha
`topk` will return the wrong value and could read out-of-bound value after jit
Labels: oncall: jit

### 🐛 Describe the bug

`topk` will return the wrong value and could read out-of-bound value after jit

```py
import torch

input = torch.tensor(5.0, dtype=torch.float64)

device = 'cuda'
m = lambda inp: inp.topk(0)

print(m(input.clone().to(device)))

jit_m = torch.jit.trace(m, input.clone().to(device))

print(m(input.clone().to(device)))
print(jit_m(input.clone().to(device)))
print(m(input.clone().to(device)))
```

```
torch.return_types.topk(
values=tensor(0., device='cuda:0', dtype=torch.float64),
indices=tensor(0, device='cuda:0'))

torch.return_types.topk(
values=tensor(1., device='cuda:0', dtype=torch.float64),
indices=tensor(4532020583610935537, device='cuda:0'))
(tensor(1., device='cuda:0', dtype=torch.float64), tensor(4532020583610935537, device='cuda:0'))

torch.return_types.topk(
values=tensor(1., device='cuda:0', dtype=torch.float64),
indices=tensor(4532020583610935537, device='cuda:0'))
```

### Versions

```
PyTorch version: 1.13.0.dev20221001
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.9.13 (main, Aug 25 2022, 23:26:10)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.19.5-051905-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.99
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3060
Nvidia driver version: 515.65.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.13.0.dev20221001
[pip3] torchaudio==0.13.0.dev20221001
[pip3] torchvision==0.14.0.dev20221001
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py39h7f8727e_0  
[conda] mkl_fft                   1.3.1            py39hd3c417c_0  
[conda] mkl_random                1.2.2            py39h51133e4_0  
[conda] numpy                     1.23.3                   pypi_0    pypi
[conda] numpy-base                1.23.1           py39ha15fc14_0  
[conda] pytorch                   1.13.0.dev20221001 py3.9_cuda11.7_cudnn8.5.0_0    pytorch-nightly
[conda] pytorch-cuda              11.7                 h67b0de4_0    pytorch-nightly
[conda] pytorch-mutex             1.0                        cuda    pytorch-nightly
[conda] torch                     1.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchaudio                0.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchvision               0.14.0.dev20220923+cu116          pypi_0    pypi
```
Segmentation fault (core dumped) in RTX3090
Labels: needs reproduction, module: cuda, triaged

### 🐛 Describe the bug

When I train my Neural Network in a RTX3090 with latest drivers 520.61.05 and CUDA Version: 11.8 and AMD Ryzen 9 5950X 16-Core Processor I get the error `Segmentation fault (core dumped)` and the training stop. It is frustrating.

I have installed the latest pytorch version with 
`pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116
`

FYI: To make sure the issue is not due to my code, I tried several times the same training script in a A100 GCP instance and it works fine

Thanks

### Versions

Collecting environment information...
PyTorch version: 1.12.1+cu116
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.10.6 (main, Aug 10 2022, 11:40:04) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.15.0-48-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.8.89
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090
Nvidia driver version: 520.61.05
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.12.1+cu116
[pip3] torch-tensorrt==1.1.0
[pip3] torchvision==0.13.1+cu116
[conda] No relevant packages

cc @ngimel
cuda.list_gpu_processes() uses the 'wrong' device order (PCI_BUS_ID)
Labels: module: cuda, triaged

### 🐛 Describe the bug

As far as I can tell, all other torch.cuda functions use the same device order (where the largest or most capable GPU is ordered first, versus when it was found on the PCI bus). However, list_gpu_processes uses the bus order. 

This is easy to test if you have 2 GPUs, with the least capable one listed first in nvidia-smi. For me:

    # nvidia-smi order
    # 0. GTX 745
    # 1. GTX 1080 Ti 

    import torch
    
    # demonstrate the PyTorch cuda order
    print(torch.cuda.get_device_name(0))  # prints: NVIDIA GeForce GTX 1080 Ti
    print(torch.cuda.get_device_name(1))  # prints: NVIDIA GeForce GTX 745
    
    # specify the 0th device
    dev = torch.device("cuda", 0)

    # verify no processes
    print(torch.cuda.list_gpu_processes(0))  # prints: GPU:0 no processes are running
    print(torch.cuda.list_gpu_processes(1))  # prints: GPU:1 no processes are running
    
    # add a variable to the device
    tempvar = torch.zeros([10,10]).to(dev)
    
    # verify wrong order processes
    print(torch.cuda.list_gpu_processes(0))  # prints: GPU:0 no processes are running
    print(torch.cuda.list_gpu_processes(1))  # prints: GPU:1 process. 100492 uses 437 MB GPU memory

    
    

### Versions

Collecting environment information...
PyTorch version: 1.12.1
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.10.6 (main, Oct  7 2022, 20:19:58) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.0-48-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration:
GPU 0: NVIDIA GeForce GTX 745
GPU 1: NVIDIA GeForce GTX 1080 Ti

Nvidia driver version: 510.85.02
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.1
[pip3] torch==1.12.1
[pip3] torchvision==0.13.1
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               11.3.1               h2bc3f7f_2
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.4.0           h06a4308_640
[conda] mkl-service               2.4.0           py310h7f8727e_0
[conda] mkl_fft                   1.3.1           py310hd6ae3a3_0
[conda] mkl_random                1.2.2           py310h00e6091_0
[conda] numpy                     1.23.1          py310h1794996_0
[conda] numpy-base                1.23.1          py310hcba007f_0
[conda] pytorch                   1.12.1          py3.10_cuda11.3_cudnn8.3.2_0    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torchvision               0.13.1              py310_cu113    pytorch

cc @ngimel
[NvFuser] JIT model with `mul+atan+sgn` will access illegal memory on cuda when computing gradient
Labels: triaged, module: nvfuser

### 🐛 Describe the bug

JIT model with `mul+atan+sgn` will access illegal memory on cuda when computing gradient

```py
import torch

class M(torch.nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, input):
        input = torch.mul(input, torch.tensor(13, dtype=torch.float32, device='cuda'))
        input = torch.atan(input)
        fn_res = torch.sgn(input, )
        return fn_res

fn = M().to('cuda')
torch.random.manual_seed(18986)
inp = torch.empty([5, 5, 5], dtype=torch.float64)
inp.uniform_(-1, 127)
inp = inp.to('cuda')

from torch.autograd.functional import jacobian

jit_fn = torch.jit.script(fn)
jit_fn(inp.clone().requires_grad_())
print(jacobian(jit_fn, inp.clone().requires_grad_()))
```

### Versions

```
PyTorch version: 1.13.0.dev20221001
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.9.13 (main, Aug 25 2022, 23:26:10)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.19.5-051905-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.99
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3060
Nvidia driver version: 515.65.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.13.0.dev20221001
[pip3] torchaudio==0.13.0.dev20221001
[pip3] torchvision==0.14.0.dev20221001
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py39h7f8727e_0  
[conda] mkl_fft                   1.3.1            py39hd3c417c_0  
[conda] mkl_random                1.2.2            py39h51133e4_0  
[conda] numpy                     1.23.3                   pypi_0    pypi
[conda] numpy-base                1.23.1           py39ha15fc14_0  
[conda] pytorch                   1.13.0.dev20221001 py3.9_cuda11.7_cudnn8.5.0_0    pytorch-nightly
[conda] pytorch-cuda              11.7                 h67b0de4_0    pytorch-nightly
[conda] pytorch-mutex             1.0                        cuda    pytorch-nightly
[conda] torch                     1.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchaudio                0.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchvision               0.14.0.dev20220923+cu116          pypi_0    pypi
```
Tooling Issue Tracking
Labels: triaged, oncall: pt2

### Minifier
- [x] [Accuracy minifier](https://github.com/pytorch/pytorch/issues/93715)
- [x] Ensure FX graphs dumped by hooks and `TORCHINDUCTOR_TRACE=1` are runnable in the minifier
- [ ] https://github.com/pytorch/pytorch/issues/93696
- [ ] https://github.com/pytorch/pytorch/issues/93694
- [x] https://github.com/pytorch/pytorch/issues/93673 @williamwen42 
- [x] Cleanup minifier print outs, currently a lot of exceptions are printed @mlazos https://github.com/pytorch/pytorch/pull/87537
- [ ] Ensure minifier runs are sensible in torchbench (check that the output isn't hidden somehwere)

### General UX
- [x] Single debug folder @mlazos https://github.com/pytorch/pytorch/pull/87438
- [ ] Enable debug dump of ir/log/minifier into single folder with one flag when there is an issue @mlazos 
Automatically run minifier with correct mode setting (aot autograd or dynamo) on exception
- [x] https://github.com/pytorch/torchdynamo/issues/1383 @mlazos 
- [x] https://github.com/pytorch/torchdynamo/issues/1376 @mlazos 
- [x] Indicator that dynamo is still running @msaroufim 
- [x] Always have next step for user in error messages
- [ ] Single config for all components (`torch.compile.config`)

### Documentation
- [ ] Stride check example in debug runbook
- [ ] Accuracy minifier example in debug runbook
- [x] FAQ once people have started to try these tools out and we have early feedback @msaroufim 

### Compilation Statistics
- [ ] https://github.com/pytorch/pytorch/issues/93675

### PyTorch Profiler Integration ([main issue](https://github.com/pytorch/pytorch/issues/93717))
- [ ] Fix links missing between CPU and GPU traces (figure out which metadata is required for this)
- [x] Add fused kernel naming to make it clearer which kernels are the results of fusion @mlazos (https://github.com/pytorch/pytorch/pull/88624)
- [ ] Add support in TorchDynamo to handle profiler context (currently they are ignored)
Currently disabled because most backends don't support it
- [ ] Identify additional requirements for integrating with the profiler infra

### Exceptions
- [ ] Fix handling of RAISE_VARARGS to show a nicer error message than Unsupported

### Logging
- [x] [Log steps of compilation](https://github.com/pytorch/pytorch/issues/93674)
- [x] Hooks for logging FX graph to external source
- [x] https://github.com/pytorch/torchdynamo/issues/1462 @mlazos 
- [x] https://github.com/pytorch/torchdynamo/issues/1167 @williamwen42 
- [x] Add logging to AOT and make it interoperate with TorchInductor trace @mlazos 


cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
[DDPOptimizer] Compiled subgraphs sometimes return lists
Labels: triaged, oncall: pt2

```python
            def compile_submod(self, submod, args, kwargs):
                """
                Compile the submodule,
                using a wrapper to make sure its output is always a tuple,
                which is required by AotAutograd based compilers
                """
                assert len(kwargs) == 0, "We assume only args for these modules"

                class WrapperModule(torch.nn.Module):
                    def __init__(self, compiled_submod, unwrap_singleton_tuple):
                        super().__init__()
                        self.compiled_submod = compiled_submod
                        self.unwrap_singleton_tuple = unwrap_singleton_tuple

                    def forward(self, *args):
                        x = self.compiled_submod(*args)
                        # TODO(whc)
                        # for some reason the isinstance check is necessary if I split one node per submod
                        # - even though I supposedly wrapped the output in a tuple in those cases, the real
                        # compiled module was still returning a tensor
                        if self.unwrap_singleton_tuple and (isinstance(x, tuple) or isinstance(x, list)): # <- HERE
                            return x[0]
                        return x

                unwrap_singleton_tuple = False
                for sn in submod.graph.nodes:
                    if sn.op == "output":
                        if not isinstance(sn.args[0], tuple):
                            unwrap_singleton_tuple = True
                            sn.args = (sn.args,) # <- Make it a tuple
                submod.recompile()

                wrapper = WrapperModule(
                    self.compiler(submod, args),
                    unwrap_singleton_tuple,
                )
                return wrapper
```

See the line marked `HERE`. We need to add an option for `isinstance(x, list)` for reasons I don't understand. We can see on the line marked "Make it a tuple" that we are already wrapping the output as a tuple, so why is the output sometimes appearing as a list?

Appears to be an inductor-only issue, I can't repro with aot_eager. aot_nvfuser fails but it appears to be for unrelated jit scripting issues...

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
Inductor doesn't fuse outer dimension softmax into a single kernel.
Labels: triaged, oncall: pt2

http://ix.io/4cog

We generate two kernels:

![image](https://user-images.githubusercontent.com/6355099/194673608-c62b6f4e-3be9-4872-a7d8-0ae2b01bc44e.png)

The issue seems to lie in the memory dependencies not matching up.

cc: @jansel @ngimel

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
[ONNX] Create an input adapter for suppling torch module input to onnxruntime
Labels: module: onnx, triaged, needs design, onnx-triaged, onnx-needs-info

ONNX runtime does not take inputs the same way as pytorch does. Currently there is a step in the verifier that massages the input into what onnx runtime expects. It would be helpful to extract this into a standalone adapter.
Automatic broadcasting for batch addition for sparse tensors
Labels: module: sparse, triaged, enhancement

### 🚀 The feature, motivation and pitch

```python
import torch
#torch.utils.backcompat.broadcast_warning.enabled=True
m1 = torch.ones(3,4)
m2 = torch.ones(4)
torch.add(m1, m2)
print("This is working.")


m1 = torch.rand((6,10),
                out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)
m1 = m1.to_sparse()

m2 = torch.rand((1,10),
                out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)
m2 = m2.to_sparse()
print("This will not work")
torch.add(m1,m2)
#m1 + m2
```

It would be great to have the same dimension broadcasting for sparse tensor as we have for dense tensors.

### Alternatives

_No response_

### Additional context

_No response_

cc @nikitaved @pearu @cpuhrsch @amjames @bhosmer
onnx.export make size operations return Tensor instead of int
Labels: module: onnx, triaged

### 🐛 Describe the bug

`Tensor.size` [is documented](https://pytorch.org/docs/stable/generated/torch.Tensor.size.html#torch.Tensor.size) to return an `int` if you pass the `dim` argument. Similarly, the Tensor indexing operations return an `int` most of the time. However, when tracing via `torch.onnx.export` these somehow do not work as advertised and instead return a `Tensor` that is a scalar.

Given:

```python
import torch

import torch.nn as nn


class ExampleIdentity(nn.Module):
    def __init__(self) -> None:
        super().__init__()

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        print('Indexing:', type(x.shape[-1]))
        print('Using size:', type(x.size(-1)))
        return x


x = torch.rand((1, 3, 224, 244))
ei = ExampleIdentity()

print('\n\nRunning normally:')
ei(x)

print('\n\nVia onnx.export:')
with open('/tmp/output.onnx', 'wb') as outf:
    torch.onnx.export(ei, x, outf)
```

And running that:

```bash
$ python3.10 /tmp/example.py


Running normally:
Indexing: <class 'int'>
Using size: <class 'int'>


Via onnx.export:
Indexing: <class 'torch.Tensor'>
Using size: <class 'torch.Tensor'>
```

### Versions

```
Collecting environment information...
PyTorch version: 1.12.1+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux 11 (bullseye) (x86_64)
GCC version: (Debian 10.2.1-6) 10.2.1 20210110
Clang version: Could not collect
CMake version: version 3.18.4
Libc version: glibc-2.31

Python version: 3.10.6 (main, Sep 20 2022, 13:16:58) [GCC 10.2.1 20210110] (64-bit runtime)
Python platform: Linux-5.10.0-18-amd64-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: Quadro P520
Nvidia driver version: 470.141.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.12.1
[pip3] torchvision==0.13.1
[conda] Could not collect
```
FSDP support to load DDP optim checkpoints
Labels: oncall: distributed, triaged, better-engineering, module: fsdp

### 🚀 The feature, motivation and pitch

FSDP optimizer checkpoint loading expects params to be keyed by FQN, but DDP saves checkpoints with param IDs.

FSDP does provide `rekey_optim_state_dict` to change the key type appropriately, but user has to manually call this code when attempting to load in a checkpoint. This makes integration with higher level trainer libs a bit more challenging. 

Propose for FSDP to just inspect the state_dict and rekey appropriately if needed. 

### Alternatives

_No response_

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501