gopikrsmscs/torch-issues huggingface dataset:
"Attempted to resize a view tensor to a larger size. This is not allowed in the functionalization pass" reported on non view tensor
Labels: triaged, module: functionalization

### 🐛 Describe the bug

Repro test:

```
    def test_resize_intermediate(self):
        def f(x):
            y = x.clone()
            z = y.view(0)
            y.resize_(20)
            return y

        self.assert_functionalization(f, torch.ones(0))
```

this raises

```
======================================================================
ERROR: test_resize_intermediate (__main__.TestFunctionalization)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_functionalization.py", line 120, in test_resize_intermediate
    self.assert_functionalization(f, torch.ones(0))
  File "test/test_functionalization.py", line 93, in assert_functionalization
    out_functional = func(input_functional)
  File "test/test_functionalization.py", line 117, in f
    y.resize_(20)
RuntimeError: Attempted to resize a view tensor to a larger size. This is not allowed in the functionalization pass
```

However, y is not a view.

This variant of the program does work:

```
        def f(x):
            y = x.clone()
            y.resize_(20)
            return y
```

cc @bdhirsh @ezyang this is low priority

### Versions

master
Investigate ncclRedOpCreatePreMulSum operator for gradient reduction
Labels: oncall: distributed

### 🚀 The feature, motivation and pitch

Per @kwen2501's comment here: https://github.com/pytorch/pytorch/issues/78774#issuecomment-1148272714, we should investigate this operator and see whether it is more efficient than gradient pre/post divide in DDP / FSDP.

### Alternatives

_No response_

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501
quantization: QConfigMapping should be easy to print
Labels: oncall: quantization, triaged

### 🐛 Describe the bug

Currently printing out the contents of a `QConfigMapping` object does not make it easy to see the contents:

```
import torch
qconfig_mapping = torch.ao.quantization.get_default_qconfig_mapping()
print(qconfig_mapping)

> <torch.ao.quantization.qconfig_mapping.QConfigMapping object at 0x7f29785c45e0>
```

Ideally printing this object would display the contents, so users can inspect the configs they are passing in to the quantization APIs.

### Versions

master

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @jgong5 @Xia-Weiwen @leslie-fang-intel @vkuzo
JIT trace takes forever on a simple method
Labels: oncall: jit

### 🐛 Describe the bug

```python
import torch
import torch.utils.benchmark as bench

from torch import Tensor

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

input_size = [64, 16, 2048, 2048//16]

m1 = torch.randn(input_size, device=device, dtype=torch.bfloat16)
m2 = torch.randn(input_size, device=device, dtype=torch.bfloat16)

def _matmul_max_impl(m1: Tensor, m2: Tensor) -> tuple[Tensor, Tensor]:
    _LOW = -64000.0
    out_shape = m1.shape[:-1]
    max = torch.full(out_shape, _LOW, dtype=m1.dtype, device=m1.device)
    max_i = torch.full(
        out_shape, 0, dtype=torch.long, device=m1.device
    )

    for i in range(m1.size(-1)):  # dimension m
        m = torch.matmul(m1, m2[:, :, i, :].unsqueeze(-1)).squeeze(-1)
        increased = m > max
        max = torch.where(increased, m, max)
        max_i = torch.where(increased, i, max_i)
    
    return max, max_i


def custom_matmul_max(m1: Tensor, m2: Tensor) -> tuple[Tensor, Tensor]:
    return _matmul_max_impl(m1, m2)

traced_matmul_max = torch.jit.trace(custom_matmul_max, (m1, m2))
# this next call has not finished after 1h running on a single core of 5950X
# no GPU load
traced_matmul_max(m1, m2)
```

### Versions

PyTorch version: 1.12.0+cu116
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 Pro
GCC version: Could not collect
Clang version: 13.0.0 (https://github.com/llvm/llvm-project.git d7b669b3a30345cfcdb2fde2af6f48aa4b94845d)
CMake version: version 3.22.22040401-MSVC_2
Libc version: N/A

Python version: 3.10.4 | packaged by conda-forge | (main, Mar 30 2022, 08:38:02) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19044-SP0
Is CUDA available: True
CUDA runtime version: 11.7.64
GPU models and configuration:
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090

Nvidia driver version: 516.59
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.22.3
[pip3] performer-pytorch==1.1.4
[pip3] torch==1.12.0+cu116
[pip3] torchaudio==0.12.0+cu116
[pip3] torchvision==0.13.0+cu116
[conda] numpy                     1.22.3                   pypi_0    pypi
[conda] performer-pytorch         1.1.4                    pypi_0    pypi
[conda] torch                     1.12.0+cu116             pypi_0    pypi
[conda] torchaudio                0.12.0+cu116             pypi_0    pypi
[conda] torchvision               0.13.0+cu116             pypi_0    pypi
Reductions on tensors larger than GPU memory
Labels: feature, module: cuda, triaged, needs research

### 🚀 The feature, motivation and pitch

I am interested in the ability to compute reductions on tensors larger than memory. For example:

```python
A = zeros(size=(100,100))
B = zeros(size=(10000000,100,100))
A.uniform_()
B.uniform_()
(A@B).min()
```

This falls under the scope of the concept of lazy computation. The output is a single integer, the input doesn't fit in GPU memory or CPU memory, but fits on disk. The data is juggled between disk, CPU and GPU based on whatever is available automatically.

Per this thread on dev-discuss, we were wondering if this is presently a capability to any extent with current backends: https://dev-discuss.pytorch.org/t/lazy-tensor-core/232/20

For example, perhaps computation on tensors larger than GPU memory for some reductions (but not this one - since the data is all random, it must be swapped to CPU RAM or disk) is currently possible on an existing backend.

cc @ngimel @wconstab 



### Alternatives

_No response_

### Additional context

_No response_
Incorrect results for mean or sum kernels on aarch64 when building with gcc-7
Labels: triaged, module: arm

## Issue description

Some aarch64 kernels produce wrong results when building with gcc-7 in native aarch64 environement.
Probably related to #47098 (presumably fixed in #47099 and then reverted in ##50389)

## Code example

The problem is reproducible by building with the following Dockerfile
```
FROM arm64v8/ubuntu:bionic

ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    gcc-7 \
    g++-7 \
    libstdc++-7-dev \
    make \
    cpp \
    python3.8 \
    python3.8-dev \
    python3-pip \
    git \
    curl \
    ssh \
    ca-certificates && \
    update-ca-certificates && \
    # Clean up
    apt-get autoremove -y && \
    apt-get clean -y && \
    rm -rf /var/lib/apt/lists/*

# Installing cmake from a GitHub release && \
RUN export CMAKE_VERSION="3.21.4" && \
    curl -L https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/cmake-${CMAKE_VERSION}-linux-aarch64.sh -o cmake-${CMAKE_VERSION}-linux-aarch64.sh && \
    chmod +x cmake-${CMAKE_VERSION}-linux-aarch64.sh && \
    ./cmake-${CMAKE_VERSION}-linux-aarch64.sh --skip-license --prefix=/usr/local/

RUN git clone https://github.com/pytorch/pytorch.git -b v1.12.0 --recursive

WORKDIR /pytorch

RUN python3.8 -m pip install -U setuptools wheel pip && \
    python3.8 -m pip install pyyaml numpy typing_extensions
RUN USE_CUDA=0 \
    USE_CUDNN=0 \
    USE_MKLDNN=0 \
    USE_DISTRIBUTED=0 \
    USE_OPENMP=1 \
    USE_MAGMA=0 \
    USE_NCCL=0 \
    USE_FBGEMM=0 \
    CC=gcc-7 \
    CXX=g++-7 \
    python3.8 setup.py bdist_wheel
RUN python3.8 -m pip install dist/torch-*.whl

WORKDIR /
RUN python3.8 -c 'import torch; m=torch.ones(5,12); print(torch.mean(m,axis=1), torch.sum(m, axis=1))'
RUN python3.8 -mtorch.utils.collect_env
```

produces the following results:
```
tensor([0.8333, 0.8333, 0.8333, 0.8333, 0.8333]) # expecting tensor([1., 1., 1., 1., 1.])
tensor([10., 10., 10., 10., 10.]) # expecting tensor([12., 12., 12., 12., 12.])
```

## System Info
```
PyTorch version: 1.12.0a0+git67ece03
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (aarch64)
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.21.4
Libc version: glibc-2.27

Python version: 3.8.0 (default, Dec  9 2021, 17:53:27)  [GCC 8.4.0] (64-bit runtime)
Python platform: Linux-5.13.0-1031-aws-aarch64-with-glibc2.27
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.1
[pip3] torch==1.12.0a0+git67ece03
[conda] Could not collect
```

cc @malfet
[Prims+NvFuser] Non-fusible ops Tracker
Labels: triaged, module: nvfuser

Following is the complete list of ops that appear in the torchbench + huggingface + TIMM models, but are not included in the nvFuser fusion group.

They are not included due to one (or more) of the following reasons: 
1. nvFuser backend does not support this op
2. aten2aten missing 
3. aten2prim ref missing 
4. nvfuser_impl missing for prims op
5. not included in NvFuserOperatorSupport dict

Tracking is moved here: https://docs.google.com/spreadsheets/d/1kmFvpIQ42XB4okN2WYrLX7D6svkRhbJX_iCFqM231YQ/edit#gid=0

  | Occurance
-- | --
aten.view.default' | 14567
aten.t.default' | 10085
aten.to.dtype' | 8595
aten.unsqueeze.default' | 6639
aten.permute.default' | 5395
aten.mm.default' | 4915
aten._unsafe_view.default' | 3585
aten.clone.default' | 2890
aten.new_zeros.default' | 2408
aten.bmm.default' | 2312
aten.reshape.default' | 1866
aten.expand.default' | 1674
aten.addmm.default' | 1423
aten.convolution.default' | 1112
aten.convolution_backward.default' | 1045
aten.slice.Tensor' | 1030
aten.native_layer_norm_backward.default' | 568
aten.mean.dim' | 436
aten.amax.default' | 362
aten.squeeze.dim' | 260
aten.cat.default' | 205
aten.narrow.default' | 192
aten.var.dim' | 114
aten.empty_like.default' | 103
aten._to_copy.default' | 75
aten.where.ScalarSelf' | 73
aten.select.int' | 70
aten.embedding_dense_backward.default' | 65
aten.zero.default' | 55
aten.fill.Scalar' | 49
aten.index_select.default' | 48
aten.select_scatter.default' | 41
aten.hardtanh.default' | 35
aten.max_pool2d_with_indices.default' | 35
aten.reflection_pad2d.default' | 31
aten.max_pool2d_with_indices_backward.default' | 28
aten.reflection_pad2d_backward.default' | 28
aten.avg_pool2d.default' | 23
aten.constant_pad_nd.default' | 21
aten.upsample_bilinear2d.vec' | 18
aten.avg_pool2d_backward.default' | 13
aten.col2im.default' | 12
aten.im2col.default' | 12
aten.index.Tensor' | 11
aten.mean.default' | 10
aten.slice_scatter.default' | 9
aten._embedding_bag.default' | 8
aten.upsample_bilinear2d_backward.vec' | 8
aten.grid_sampler_2d.default' | 6
aten.stack.default' | 6
aten._adaptive_avg_pool2d_backward.default' | 2
aten._adaptive_avg_pool2d.default' | 2
aten.any.default' | 1
aten.gather.default' | 1
aten.new_empty.default' | 1
aten.repeat.default' | 1
aten.scatter_add.default' | 1
Files downloaded with torch.hub should respect umask
Labels: triaged, module: hub

### 🐛 Describe the bug

torch.hub code uses the `download_url_to_file` function to download and cache files. This function first downloads the data to a temporary file (using `NamedTemporaryFile`), then moves it into the cache directory. As a result, the file will always have 0600 permissions (this is hard-coded for files created using `NamedTemporaryFile`, and moving the file copies these permissions). As a result, umask is not respected and there is therefore no way to control the permissions of files created in the cache. This is a problem for multi-user systems that want to share the files in the torch.hub cache directory, as they will only be user-readable.

See here for discussion of a similar issue in a different Python project: https://github.com/zarr-developers/zarr-python/issues/325

### Versions

Collecting environment information...
PyTorch version: 1.11.0
Is debug build: False
CUDA used to build PyTorch: 11.5
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:56:21)  [GCC 10.3.0] (64-bit runtime)
Python platform: Linux-5.15.0-40-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.64
GPU models and configuration: 
GPU 0: NVIDIA GeForce GTX TITAN X
GPU 1: NVIDIA GeForce GTX TITAN X

Nvidia driver version: 515.48.07
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.4
[pip3] torch==1.11.0
[pip3] torchmetrics==0.9.2
[pip3] torchvision==0.12.0
[conda] blas                      2.114                       mkl    conda-forge
[conda] blas-devel                3.9.0            14_linux64_mkl    conda-forge
[conda] cudatoolkit               11.5.1              h59c8dcf_10    conda-forge
[conda] libblas                   3.9.0            14_linux64_mkl    conda-forge
[conda] libcblas                  3.9.0            14_linux64_mkl    conda-forge
[conda] liblapack                 3.9.0            14_linux64_mkl    conda-forge
[conda] liblapacke                3.9.0            14_linux64_mkl    conda-forge
[conda] mkl                       2022.0.1           h8d4b97c_803    conda-forge
[conda] mkl-devel                 2022.0.1           ha770c72_804    conda-forge
[conda] mkl-include               2022.0.1           h8d4b97c_803    conda-forge
[conda] numpy                     1.22.4           py39hc58783e_0    conda-forge
[conda] pytorch                   1.11.0          py3.9_cuda11.5_cudnn8.3.2_0    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torchmetrics              0.9.2              pyhd8ed1ab_0    conda-forge
[conda] torchvision               0.12.0               py39_cu115    pytorch

cc @nairbv @NicolasHug @vmoens @jdsgomes
Refactor linter adapters to avoid code duplication
Labels: module: lint, triaged, enhancement

### 🚀 The feature, motivation and pitch

[Linter adapters](https://github.com/pytorch/pytorch/tree/master/tools/linter/adapters) are used by `lintrunner` to check and format pytorch code. There are close to 20 of them at the moment, and the list could grow to include many more in the future. At the moment, adding a new linter requires copy/paste the code structure from other linters, and make the necessary modification, i.e. updating `check_file`. This approach doesn't scale and leads to code duplication. It's better to have a well-defined interface for linter adapters and use that to define new linters

### Alternatives

* Define a base class for linter adapter (require design), i.e.
```
class LinterAdapter(abc)
  @abstract
  def check_file(filename: str) -> Optional[LintMessage]:
    pass

  def execute()
    # Run check_file in parallel
```
* Put as much common logic in [linter adapters](https://github.com/pytorch/pytorch/tree/master/tools/linter/adapters) to the base class as possible
* Refactor other existing linter adapters to use the base class
* Update [README.md](https://github.com/pytorch/pytorch/blob/master/tools/linter/adapters/README.md) to provide instructions and examples on how to create a new linter

### Additional context

* [lintrunner GH](https://github.com/suo/lintrunner)
* [lintrunner](https://github.com/pytorch/pytorch/wiki/lintrunner)
[FSDP] Avoid explicit replace of activation checkpoint prefixes
Labels: oncall: distributed, better-engineering, module: fsdp

### 🚀 The feature, motivation and pitch

FSDP has two instances where we explicitly replace the AC prefix: https://github.com/pytorch/pytorch/pull/80936/files and https://github.com/pytorch/pytorch/pull/81253/files. We should figure out a better solution as this is not great for composability. 

### Alternatives

_No response_

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501 @ezyang
Libtorch cannot load TrochScript Module correctly, when a network contains conv2d(inchannels=64, outchannels=128, kernelsize=1) .
Labels: oncall: jit

### 🐛 Describe the bug

Libtorch cannot load TrochScript Module correctly, when a network contains`` conv2d(inchannels=64, outchannels=128, kernelsize=1)`` .

### Versions

I need to convert a model in python to c++, so I use torch.jit.trace produce a 'pt' file:
pytorch == 1.12
python == 3.10
```python
import torch

class m(torch.nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 =  torch.nn.Conv2d(64, 128, (1, 1))
    
    def forward(self, x):
        return self.conv1(x)

a = m()
b = torch.rand((1, 64, 128, 128)).cpu().float()
x = torch.jit.trace(a, b)
x.save('model.pt')

c = torch.zeros((1, 64, 128, 128)).cpu()
print(x(c)[0,0,0,0]) # tensor(0.0301, grad_fn=<SelectBackward0>)
````
so I use libtorch==1.12 to load the pt file:
c++17

```c++
#include <iostream>
#include <torch/torch.h> 
#include <torch/script.h>

using namespace torch; 
using namespace std;

int main() {
    
    auto fuck = torch::jit::load("model.pt");
    auto t = torch::zeros({1, 64, 128, 128}).toType(torch::kFloat32);
    std::vector<torch::jit::IValue> inputs;
    inputs.push_back(t);
    auto output = fuck.forward(inputs).toTensor().squeeze();
    output.print();
    cout<<output[0][0][0]<<endl; //-8.44151e+33
    return 0;
}

```

However, when I set the in_channels=1 out_channels=128, their outputs are equal!!
```python
import torch

class m(torch.nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 =  torch.nn.Conv2d(1, 128, (1, 1)) # in_channel = 1
    
    def forward(self, x):
        return self.conv1(x)

a = m()
b = torch.rand((1, 1, 128, 128)).cpu().float()
x = torch.jit.trace(a, b)
x.save('model.pt')

c = torch.zeros((1, 1, 128, 128)).cpu()
print(x(c)[0,0,0,0]) # tensor(-0.3926, grad_fn=<SelectBackward0>)
```

```c++
#include <iostream>
#include <torch/torch.h>   
#include <torch/script.h>

using namespace torch; 
using namespace std;

int main() {
    
    auto fuck = torch::jit::load("model.pt");
    auto t = torch::zeros({1, 1, 128, 128}).toType(torch::kFloat32);
    std::vector<torch::jit::IValue> inputs;
    inputs.push_back(t);
    auto output = fuck.forward(inputs).toTensor().squeeze();
    output.print();
    cout<<output[0][0][0]<<endl; //-0.392607

    return 0;
}

```

I think this maybe a bug.
CapabilityBasedPartitioner does not work correctly with mutating operations
Labels: triaged, module: fx, module: CapabilityBasedPartitioner

### 🐛 Describe the bug

Here is a diff adding a test case that triggers the error

```
diff --git a/test/test_fx_passes.py b/test/test_fx_passes.py
index 39fe501f44..b09f0e71e5 100644
--- a/test/test_fx_passes.py
+++ b/test/test_fx_passes.py
@@ -158,11 +158,23 @@ class TestPartitionFunctions:
         out = torch.stack([add_1, add_2, add_3])
         return out
 
+    @staticmethod
+    def forward12(a):
+        b = a.clone()
+        b_1 = b.fill_(2)
+        b_2 = b.fill_(3)
+        return b
+
 # A mock OperatorSupport class, where only operator.add is supported
 class MockOperatorSupport(OperatorSupport):
     def is_node_supported(self, submodules, node: torch.fx.Node) -> bool:
         return node.op == "call_function" and node.target in {operator.add}
 
+# A mock OperatorSupport where all ops are supported
+class MockAllOperatorSupport(OperatorSupport):
+    def is_node_supported(self, submodules, node: torch.fx.Node) -> bool:
+        return node.op == "call_function"
+
 class TestFXGraphPasses(JitTestCase):
 
     @parametrize("fn, expected_partition", [
@@ -179,16 +191,20 @@ class TestFXGraphPasses(JitTestCase):
         (TestPartitionFunctions.forward9, [['add_3', 'add_2', 'add_1', 'add']]),
         (TestPartitionFunctions.forward10, [['add_3', 'add_2', 'add', 'add_1']]),
         (TestPartitionFunctions.forward11, [['add_1'], ['add']]),
+
+        (TestPartitionFunctions.forward12, [['b_2', 'b_1', 'b']]),
     ])
     def test_partitioner(self, fn, expected_partition):
         traced = symbolic_trace(fn)
 
         supported_ops = MockOperatorSupport()
+        if fn is TestPartitionFunctions.forward12:
+            supported_ops = MockAllOperatorSupport()
         partitioner = CapabilityBasedPartitioner(traced, supported_ops, allows_single_node_partition=True)
         partitions = partitioner.propose_partitions()
 
         partitions_name = [[node.name for node in partition.nodes] for partition in partitions]
-        assert len(partitions_name) == len(expected_partition)
+        assert len(partitions_name) == len(expected_partition), f"{partitions_name}, {expected_partition}"
         for i in range(len(partitions_name)):
             assert set(partitions_name[i]) == set(expected_partition[i])
 

```

cc @ezyang @SherlockNoMad 

### Versions

master
DISABLED test_trace_dependencies (test_analyze.TestAnalyze)
Labels: triaged, module: flaky-tests, skipped, module: deploy, oncall: package/deploy, imported

Platforms: linux

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_trace_dependencies&suite=test_analyze.TestAnalyze&file=package/test_analyze.py) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/7275940320).

Over the past 3 hours, it has been determined flaky in 1 workflow(s) with 1 red and 1 green.

cc @wconstab @bdhirsh @ezyang
torch._weight_norm with specified dim returns wrong output
Labels: module: nn, module: error checking, triaged, module: regression, module: norms and normalization

### 🐛 Describe the bug

# Problem
`torch._weight_norm` outputs wrong result from torch 1.12

Though `torch._weight_norm` is an undocumented API, I report this issue since the API is made public.

#### torch 1.11 and before
```python3
$ python3
Python 3.8.10 (default, Mar 15 2022, 12:22:08) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> a = torch.Tensor([[1,2],[3,4]])
>>> b = torch.Tensor([[5],[6]])
>>> torch._weight_norm(a,b,dim=1)
tensor([[1.5811, 2.2361],
        [5.6921, 5.3666]])
```


#### torch 1.12
```python3
$ python3
Python 3.8.10 (default, Mar 15 2022, 12:22:08) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> a = torch.Tensor([[1,2],[3,4]])
>>> b = torch.Tensor([[5],[6]])
>>> torch._weight_norm(a,b,dim=1)
tensor([[1.5811, 2.6833],
        [4.7434, 5.3666]])
```

#### manual computation
```python3
>>> def weight_norm_dim1(a, b):
...     norm = a.norm(2, [0], True)
...     return a * b / norm
... 
>>> weight_norm_dim1(a, b)
tensor([[1.5811, 2.2361],
        [5.6921, 5.3666]])
```

### Versions

```
Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
Clang version: Could not collect
CMake version: version 3.22.2
Libc version: glibc-2.31

Python version: 3.8.10 (default, Mar 15 2022, 12:22:08)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.4.0-100-generic-x86_64-with-glibc2.2.5
Is CUDA available: N/A
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: Tesla V100-SXM2-16GB
GPU 1: Tesla V100-SXM2-16GB
GPU 2: Tesla V100-SXM2-16GB
GPU 3: Tesla V100-SXM2-16GB
GPU 4: Tesla V100-SXM2-16GB
GPU 5: Tesla V100-SXM2-16GB
GPU 6: Tesla V100-SXM2-16GB
GPU 7: Tesla V100-SXM2-16GB

Nvidia driver version: 470.103.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.0.5
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] numpy==1.18.4
[pip3] torch==1.12.0
[pip3] torchvision==0.13.0
[conda] Could not collect
```

cc @ezyang @gchanan @zou3519 @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
grad not preserved during copying or pickling
Labels: triaged, module: python frontend

### 🐛 Describe the bug

I would like to (cloud-)pickle some Tensors. However, their `grad` attribute is lost during pickling. The following snippet 

```python
import cloudpickle
import copy
import torch

def square(x):
    return x*x

x = torch.tensor([2.0], requires_grad=True)
loss = square(x)
loss.backward()
print (type(x), x, x.grad)
y = copy.copy(x)
print (type(y), y, y.grad)

y = cloudpickle.loads(cloudpickle.dumps(x))
print (type(y), y, y.grad)

```
yields the following output:
```
<class 'torch.Tensor'> tensor([2.], requires_grad=True) tensor([4.])
<class 'torch.Tensor'> tensor([2.], requires_grad=True) None
<class 'torch.Tensor'> tensor([2.], requires_grad=True) None

```

I can work around this using a custom reduction function for `torch.Tensor` like
```python

import copyreg
def reconstruct_tensor(val, requires_grad, grad, device):
    t = torch.tensor(val, device=device, requires_grad=requires_grad)
    if grad:
        t.grad = torch.from_numpy(grad)
    return t
    
def reduce_tensor(t):
    if t.requires_grad:
        return reconstruct_tensor, (t.detach().numpy(), True, t.grad.numpy(), t.device.type)
    else:
        return reconstruct_tensor, (t.numpy(), False, None, t.device.type)

copyreg.pickle(torch.Tensor, reduce_tensor)
# Include previous snippet here
```
which yields the expected output
```
<class 'torch.Tensor'> tensor([2.], requires_grad=True) tensor([4.])
<class 'torch.Tensor'> tensor([2.], requires_grad=True) tensor([4.])
<class 'torch.Tensor'> tensor([2.], requires_grad=True) tensor([4.])

```
This is basically what [Dask does](https://github.com/dask/distributed/blob/main/distributed/protocol/torch.py). But a better place to fix this would seem to be in [`Tensor.__reduce_ex__`](https://github.com/pytorch/pytorch/blob/3c2199b159b6ec57af3f7ea22d61ace9ce5cf5bc/torch/_tensor.py#L208)

### Versions

PyTorch version: 1.12.0+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Fedora Linux 35 (Container Image) (x86_64)
GCC version: (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9)
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.34

Python version: 3.8.13 (default, Mar 28 2022, 11:38:47)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.18.9-200.fc36.x86_64-x86_64-with-glibc2.17
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.0
[pip3] torch==1.12.0+cpu
[pip3] torchaudio==0.12.0+cpu
[pip3] torchvision==0.13.0+cpu
[conda] numpy                     1.23.0                   pypi_0    pypi
[conda] torch                     1.12.0+cpu               pypi_0    pypi
[conda] torchaudio                0.12.0+cpu               pypi_0    pypi
[conda] torchvision               0.13.0+cpu               pypi_0    pypi
move bazel files out of pytorch repo root
Labels: triaged, module: bazel

https://github.com/pytorch/pytorch/commit/03847808a0f98faf91790802a9f5be49de82360b added a bunch of bazel files to pytorch repo root. Can we move these somewhere out of the way? As is, they make the "front page" of the repo really long and make it so you have to scroll to get to the readme
SparseAdam performance issue during optimizer step
Labels: module: performance, module: sparse, module: optimizer, triaged

### 🐛 Describe the bug

I read the code and found that Sparse Adam optimizer add sparse to dense tensor and called coalesce method on Sparse Tensor during step, and that operation on CPU is extremely slow. I run Node2Vec model on pubmed dataset, using Sparse Tensor with Sparse Adam optimizer is slower than using dense tensor with Adam optimizer. 
And I noticed that during tensor add and coalesce operation, only one CPU was used. Do we have some method to accelerate these processes with multiple CPU cores usage? 

### Versions

```
Collecting environment information...
PyTorch version: 1.12.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.31

Python version: 3.9.12 (main, Apr  5 2022, 06:56:58)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.15.35-1-pve-x86_64-with-glibc2.31
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] intel-extension-for-pytorch==1.12.0
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.5
[pip3] numpydoc==1.2
[pip3] torch==1.12.0
[pip3] torchaudio==0.12.0
[pip3] torchvision==0.13.0
[conda] blas                      1.0                         mkl  
[conda] cpuonly                   2.0                           0    pytorch
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] intel-extension-for-pytorch 1.12.0                   pypi_0    pypi
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py39h7f8727e_0  
[conda] mkl_fft                   1.3.1            py39hd3c417c_0  
[conda] mkl_random                1.2.2            py39h51133e4_0  
[conda] numpy                     1.21.5           py39he7a7128_1  
[conda] numpy-base                1.21.5           py39hf524024_1  
[conda] numpydoc                  1.2                pyhd3eb1b0_0  
[conda] pytorch                   1.12.0              py3.9_cpu_0    pytorch
[conda] pytorch-mutex             1.0                         cpu    pytorch
[conda] torchaudio                0.12.0                 py39_cpu    pytorch
[conda] torchvision               0.13.0                 py39_cpu    pytorch
```

cc @VitalyFedyunin @ngimel @nikitaved @pearu @cpuhrsch @amjames @bhosmer @vincentqb @jbschlosser @albanD
libprotobuf version compatibility 
Labels: triaged, module: build warnings

### 🐛 Describe the bug


Just wanna test out **classifier.py**, but failed with the following **ERROR** message

```console
[libprotobuf FATAL google/protobuf/stubs/common.cc:83] This program was compiled against version 3.9.2 of the Protocol Buffer runtime library, which is not compatible with the installed version (3.19.4).  Contact the program author for an update.  If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library.  (Version verification failed in "bazel-out/k8-opt/bin/tensorflow/core/framework/tensor_shape.pb.cc".)
```

### Versions

```console
➜  ~ python collect_env.py 
Collecting environment information...
PyTorch version: 1.13.0a0+git4b6ba34
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: 15.0.0-++20220707031229+438ffdb821bb-1~exp1~20220707151338.282
CMake version: version 3.22.1
Libc version: glibc-2.35

Python version: 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.0-40-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.64
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3080 Ti Laptop GPU
Nvidia driver version: 515.48.07
cuDNN version: Probably one of the following:
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn.so.8.4.1
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.4.1
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.4.1
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.4.1
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.4.1
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.4.1
/usr/local/cuda-11.7/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.4.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.0
[pip3] torch==1.13.0a0+git4b6ba34
[pip3] torch-model-archiver==0.6.0
[pip3] torchserve==0.6.0
[pip3] torchvision==0.14.0a0+fb7f9a1
[conda] Could not collect
```
Docker updates cause subsequent builds to fail
Labels: high priority, module: ci, triaged

### 🐛 Describe the bug

Most recent example:
https://hud.pytorch.org/pytorch/pytorch/commit/8a5d9843ff5d5dd865fc922853a15b3e7e459fdb (underlying workflows [trunk](https://github.com/pytorch/pytorch/actions/runs/2637230403/attempts/1) and [pull](https://github.com/pytorch/pytorch/actions/runs/2637230400) )

And subsequent commits after this one started to fail as well until I've manually re-run docker image build in https://github.com/pytorch/pytorch/actions/runs/2637584738


We are having those regressions periodically, despite numerous claims that race conditions have been fixed

### Versions

CI

cc @ezyang @gchanan @zou3519 @seemethere @malfet @pytorch/pytorch-dev-infra
torch.package can not be used to serialize `resnet18` from TorchVision-0.12
Labels: high priority, module: vision, module: regression, oncall: package/deploy, imported

### 🐛 Describe the bug

Running `ModelTest.test_model` will fail if PyTorch-1.12 + TorchVision are installed with the following error:
```
torch.package.package_exporter.PackagingError: 
* Module had no __file__ defined.
    sys
    _io
    _imp
    py.io
    six.moves.urllib.parse
* Module is a C extension module. torch.package supports Python modules only.
    numpy.core._multiarray_umath
    _ctypes
    numpy.linalg.lapack_lite
    numpy.linalg._umath_linalg
    psutil._psutil_osx
    psutil._psutil_posix
    numpy.random.mtrand
    numpy.random._philox
    numpy.random._pcg64
    numpy.random._sfc64
    numpy.random._generator
    numpy.random._mt19937
    numpy.random._common
    numpy.random._bounded_integers
    numpy.random.bit_generator
    numpy.fft._pocketfft_internal
    scipy._lib._ccallback_c
    _cffi_backend
    PIL._imaging
    PIL._imagingft
```
This failures have started with commit https://github.com/pytorch/vision/commit/183a722169421c83638e68ee2d8fc5bd3415c4b4

that introduced unused by generated model, but still serialized dependency on `sys` buildin module, see
```
% python3 test_package.py -v -k test_resnet
test_resnet (test_model.ModelTest) ... ERROR

======================================================================
ERROR: test_resnet (test_model.ModelTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/nshulga/git/pytorch/pytorch/test/package/test_model.py", line 45, in test_resnet
    e.save_pickle("model", "model.pkl", resnet)
  File "/Users/nshulga/git/pytorch/pytorch/torch/package/package_exporter.py", line 951, in __exit__
    self.close()
  File "/Users/nshulga/git/pytorch/pytorch/torch/package/package_exporter.py", line 1055, in close
    self._execute_dependency_graph()
  File "/Users/nshulga/git/pytorch/pytorch/torch/package/package_exporter.py", line 992, in _execute_dependency_graph
    self._validate_dependency_graph()
  File "/Users/nshulga/git/pytorch/pytorch/torch/package/package_exporter.py", line 974, in _validate_dependency_graph
    raise PackagingError(self.dependency_graph)
torch.package.package_exporter.PackagingError: 
* Module had no __file__ defined.
    sys


----------------------------------------------------------------------
Ran 1 test in 0.919s

FAILED (errors=1)
```

### Versions

Nightly, 1.12/0.12

cc @ezyang @gchanan @zou3519 @fmassa @vfdev-5 @pmeier
CI: Run cpu tests in parallel processes?
Labels: module: ci, triaged

AFAICT CI is running on multi-core workers, but only launching a single process to run the tests. Since most individual test cases have low thread utilization, this means under-utilizing the available hardware.

For example, using [`pytest-xdist`](https://pypi.org/project/pytest-xdist/) I see a 4x speedup from running 8 parallel processes on a 32-core machine
```
$ time OMP_NUM_THREADS=32 pytest test_ops.py -k cpu
real	24m13.282s
user	369m12.584s
sys	11m12.221s

$ time OMP_NUM_THREADS=4 pytest test_ops.py -k cpu -n 8
real	5m22.487s
user	61m53.781s
sys	2m21.365s
```



There is some risk of additional flakiness. For example, two tests allocating large tensors might fail if run at the same time. This could be mitigated though, by marking problematic tests as not suitable for running in parallel.


cc @seemethere @malfet @pytorch/pytorch-dev-infra
[discussion] Consolidation of audio-visual I/O in a new package
Labels: module: build, triaged, module: vision, module: third_party

### 🚀 The feature, motivation and pitch

Both torchvision (VideoReader) and torchaudio (StreamReader) have implemented different wrappers for ffmpeg for reading both video (and audio) with their own api / decisions / problems on building from source and linking: https://github.com/pytorch/vision/issues/5720#issuecomment-1171715173. And also torchdata has its own entity named StreamReader dealing with something else completely: https://github.com/pytorch/pytorch/blob/cac16e2ee293964033dffa6616f78b68603cd565/torch/utils/data/datapipes/iter/streamreader.py#L8-L9 (for StreamReader is a quite generic name), so audio's naming is also not very good IMO

I propose to consolidate this complex code in a separate package (`torchio`? `torchavio`?) and confine these issues to this package only. It would simplify building / testing of torchvision/torchaudio

It may make sense to move to this package image/audio reading/writing functionality too (although less urged because building of those seems simpler).

@ezyang @soumith @albanD 

### Alternatives

_No response_

### Additional context

_No response_

cc @malfet @seemethere @fmassa @vfdev-5 @pmeier
RuntimeError: required keyword attribute 'value' is undefined
Labels: high priority, triage review, oncall: jit

### 🐛 Describe the bug

Hi,
While loading loading a `.pt` file, I'm getting following error:
```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sort/miniconda3/envs/openmmlab/lib/python3.8/site-packages/torch/jit/_serialization.py", line 162, in load
    cpp_module = torch._C.import_ir_module(cu, str(f), map_location, _extra_files)
RuntimeError: required keyword attribute 'value' is undefined
```

Code:
```
import torch
loaded = torch.jit.load('end2end.pt')
```

### Versions

Collecting environment information...
PyTorch version: 1.12.0
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.8.13 (default, Mar 28 2022, 11:38:47)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.13.0-52-generic-x86_64-with-glibc2.17
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: NVIDIA GeForce RTX 3080 Ti
GPU 1: NVIDIA GeForce RTX 3090

Nvidia driver version: 470.103.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.2.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.2.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.2.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.2.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.2.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.2.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.2.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.0
[pip3] torch==1.12.0
[pip3] torchvision==0.13.0
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               11.3.1               h2bc3f7f_2  
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] libblas                   3.9.0            12_linux64_mkl    conda-forge
[conda] libcblas                  3.9.0            12_linux64_mkl    conda-forge
[conda] liblapack                 3.9.0            12_linux64_mkl    conda-forge
[conda] liblapacke                3.9.0            12_linux64_mkl    conda-forge
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py38h7f8727e_0  
[conda] mkl_fft                   1.3.1            py38hd3c417c_0  
[conda] mkl_random                1.2.2            py38h51133e4_0  
[conda] numpy                     1.23.0                   pypi_0    pypi
[conda] numpy-base                1.22.3           py38hf524024_0  
[conda] pytorch                   1.12.0          py3.8_cuda11.3_cudnn8.3.2_0    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torchvision               0.13.0               py38_cu113    pytorch


cc @ezyang @gchanan @zou3519
[ONNX] Exporting the operator `::svd` to ONNX opset version 13 is not supported.
Labels: module: onnx, triaged, OSS contribution wanted, onnx-triaged

### 🐛 Describe the bug

Exporting the operator ::svd to ONNX opset version 13 is not supported.

```
File "C:\Users\kimju\anaconda3\envs\HybrIK\lib\site-packages\torch\onnx\__init__.py", line 350, in export
  return utils.export(
File "C:\Users\kimju\anaconda3\envs\HybrIK\lib\site-packages\torch\onnx\utils.py", line 163, in export
  _export(
File "C:\Users\kimju\anaconda3\envs\HybrIK\lib\site-packages\torch\onnx\utils.py", line 1074, in _export
  graph, params_dict, torch_out = _model_to_graph(
File "C:\Users\kimju\anaconda3\envs\HybrIK\lib\site-packages\torch\onnx\utils.py", line 731, in _model_to_graph
  graph = _optimize_graph(
File "C:\Users\kimju\anaconda3\envs\HybrIK\lib\site-packages\torch\onnx\utils.py", line 308, in _optimize_graph
  graph = _C._jit_pass_onnx(graph, operator_export_type)
File "C:\Users\kimju\anaconda3\envs\HybrIK\lib\site-packages\torch\onnx\__init__.py", line 416, in _run_symbolic_function
  return utils._run_symbolic_function(*args, **kwargs)
File "C:\Users\kimju\anaconda3\envs\HybrIK\lib\site-packages\torch\onnx\utils.py", line 1421, in _run_symbolic_function
  raise symbolic_registry.UnsupportedOperatorError(
torch.onnx.symbolic_registry.UnsupportedOperatorError: Exporting the operator ::svd to ONNX opset version 13 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub.
```

### Versions

pytorch 1.12.0
[ONNX] Support opset 17 operators
Labels: module: onnx, good first issue, triaged, OSS contribution wanted, onnx-triaged

Tracking issue on the onnx opset 17 support. Contributions welcome.

- [x] Add version 17 as a known version #83287 
- [ ] fft
  - [ ] ifft
  - [ ] fft_fft
  - [ ] fft_ifft
  - [ ] fft_rfft
  - [ ] fft_irfft
  - [ ] fft_hfft
  - [ ] fft_ihfft
  - [ ] fft_fft2
  - [ ] fft_ifft2
  - [ ] fft_rfft2
  - [ ] fft_irfft2
  - [ ] fft_hfft2
  - [ ] fft_ihfft2
  - [ ] fft_fftn
  - [ ] fft_ifftn
  - [ ] fft_rfftn
  - [ ] fft_irfftn
  - [ ] fft_hfftn
  - [ ] fft_ihfftn
  - [ ] fft_fftfreq
  - [ ] fft_rfftfreq
  - [ ] fft_fftshift
  - [ ] fft_ifftshift
- [x] stft #83944 
- [ ] istft
- [ ] bartlett_window
- [ ] blackman_window
- [ ] hamming_window
- [ ] hann_window
- [ ] kaiser_window
- [x] LayerNormalization #84293

Closes #80834
[Releng] Improve the tutorials release process
Labels: module: build, module: ci, triaged

### 📚 The doc issue

Copying from [tutorials](https://github.com/pytorch/tutorials/issues/1964) by @b0noI' request.

We need a better way to test tutorials against the RC than we have now. Today we wait until the release to merge the tutorials which is a confusing way of doing it. We don't have a clear way to know against which dependencies the tutorial was tested against.

### Suggest a potential alternative/fix

We should be able to merge to a branch that runs against RCs and publish under pytorch.org/some-link so that we can preview and make sure the tutorials build correctly before the release.
It would also help to have some kind of printenv output that would tell exactly against which binaries for all pytorch components and related dependancies we have build the tutorials.

cc @malfet @seemethere @pytorch/pytorch-dev-infra
three typing inconsistencies on Tensor methods
Labels: module: typing, triaged

### 🐛 Describe the bug

* `Tensor.aslist` is typed to return `def tolist(self) -> List: ...` but returns a scalar if Tensor is 0-dim
* `Tensor.unbind` is typed to return `def unbind(self, ...) -> List[Tensor]: ...` but returns a tuple in 1.12
* `Tensor.chunk` is typed to return `def chunk(self, ...) -> List[Tensor]: ...` but returns a tuple in 1.12
* There's probably more like these, including the torch namespace, but these were the breakers I found in my build after upgrading
* Use case for fixing: It breaks mypyc compilation (unless hacked via getattr).

### Versions

Collecting environment information...
PyTorch version: 1.12.0+cu116
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Manjaro Linux (x86_64)
GCC version: (GCC) 12.1.0
Clang version: 13.0.1
CMake version: version 3.23.2
Libc version: glibc-2.35

Python version: 3.10.5 (main, Jun  6 2022, 18:49:26) [GCC 12.1.0] (64-bit runtime)
Python platform: Linux-5.15.50-1-MANJARO-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.64
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3080
Nvidia driver version: 515.57
cuDNN version: Probably one of the following:
/usr/lib/libcudnn.so.8.4.0
/usr/lib/libcudnn_adv_infer.so.8.4.0
/usr/lib/libcudnn_adv_train.so.8.4.0
/usr/lib/libcudnn_cnn_infer.so.8.4.0
/usr/lib/libcudnn_cnn_train.so.8.4.0
/usr/lib/libcudnn_ops_infer.so.8.4.0
/usr/lib/libcudnn_ops_train.so.8.4.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.950
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.22.3
[pip3] torch==1.12.0+cu116
[pip3] torch-lr-finder==0.2.1
[conda] Could not collect


cc @ezyang @malfet @rgommers @xuzhao9 @gramster