gopikrsmscs/torch-issues huggingface dataset:
aten::_softmax.out doesn't work with non-contiguous Tensors 
Labels: module: nn, triaged

### 🐛 Describe the bug

Repro script:

```python
import torch

from torch.testing import make_tensor

inp = torch.rand(2, 3)

out = make_tensor(inp.shape, dtype=inp.dtype, device=inp.device, noncontiguous=True)
print("BEFORE SOFTMAX: \n", out)
torch._softmax(inp, 0, half_to_float=False, out=out)

print("AFTER SOFTMAX: \n", out)

out_real = torch._softmax(inp, 0, half_to_float=False)

print("EXPECTED OUTPUT: \n", out_real)
``` 

outputs:
```
BEFORE SOFTMAX: 
 tensor([[-0.1820,  4.4953, -8.6387],
        [-2.9375,  8.6783,  7.0183]])
AFTER SOFTMAX: 
 tensor([[ 0.5257,  0.4881,  0.3627],
        [-2.9375,  8.6783,  7.0183]])
EXPECTED OUTPUT: 
 tensor([[0.5257, 0.6373, 0.4881],
        [0.4743, 0.3627, 0.5119]])
 ```

This is clearly wrong because the second row is not updated and there shouldn't be any negative numbers. 
cc: @gmagogsfm @zhxchen17 

### Versions

PyTorch version: 1.12.0a0+gitfab098d
Is debug build: True
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: CentOS Stream 8 (x86_64)
GCC version: (GCC) 8.5.0 20210514 (Red Hat 8.5.0-3)
Clang version: Could not collect
CMake version: version 3.19.6
Libc version: glibc-2.28

Python version: 3.8.10 (default, May 19 2021, 18:05:58)  [GCC 7.3.0] (64-bit runtime)
Python platform: Linux-5.6.13-0_fbk19_hardened_6064_gabfd136bb69a-x86_64-with-glibc2.10
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.812
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.20.2
[pip3] torch==1.12.0a0+gitfab098d
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.2.0           h06a4308_296  
[conda] mkl-include               2021.2.0           h06a4308_296  
[conda] mkl-service               2.3.0            py38h27cfd23_1  
[conda] mkl_fft                   1.3.0            py38h42c9631_2  
[conda] mkl_random                1.2.1            py38ha9443f7_2  
[conda] mypy                      0.812              pyhd3eb1b0_0  
[conda] mypy_extensions           0.4.3                    py38_0  
[conda] numpy                     1.20.2           py38h2d18471_0  
[conda] numpy-base                1.20.2           py38hfae3a4d_0  
[conda] torch                     1.10.0a0+gitacc9f9a          pypi_0    pypi

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345
'python setup.py build' failed but succeed using  'pip install -v .' which calls 'python setup.py build'.
Labels: module: build, triaged

## Issue description

'python setup.py build' failed but succeed using  'pip install -v .' which calls 'python setup.py build'.

Error message:
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '6']' returned non-zero exit status 2.


## Log
# TORCH_CUDA_ARCH_LIST="3.5 5.2 6.0 6.1 7.0+PTX 8.0" TORCH_NVCC_FLAGS="-Xfatbin -compress-all"     CMAKE_PREFIX_PATH="$(dirname $(which conda))/../"     python setup.py build
Building wheel torch-1.7.0a0
-- Building version 1.7.0a0
cmake --build . --target install --config Release -- -j 6
make: *** No rule to make target 'install'.  Stop.
Traceback (most recent call last):
  File "setup.py", line 717, in <module>
    build_deps()
  File "setup.py", line 308, in build_deps
    build_caffe2(version=version,
  File "/pytorch-builder/src/pytorch/tools/build_pytorch_libs.py", line 62, in build_caffe2
    cmake.build(my_env)
  File "/pytorch-builder/src/pytorch/tools/setup_helpers/cmake.py", line 345, in build
    self.run(build_args, my_env)
  File "/pytorch-builder/src/pytorch/tools/setup_helpers/cmake.py", line 141, in run
    check_call(command, cwd=self.build_dir, env=env)
  File "/opt/conda/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '6']' returned non-zero exit status 2.


## System Info
# python ./torch/utils/collect_env.py 
Collecting environment information...
PyTorch version: 1.7.0a0+bd37e59
Is debug build: True
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: version 3.10.2

Python version: 3.8 (64-bit runtime)
Is CUDA available: False
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 2070 SUPER
Nvidia driver version: 470.86
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.21.5
[pip3] torch==1.7.0a0
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py38h7f8727e_0  
[conda] mkl_fft                   1.3.1            py38hd3c417c_0  
[conda] mkl_random                1.2.2            py38h51133e4_0  
[conda] numpy                     1.21.5           py38he7a7128_1  
[conda] numpy-base                1.21.5           py38hf524024_1  
[conda] torch                     1.7.0a0                  pypi_0    pypi


cc @malfet @seemethere
[FSDP] Verify buffer checkpointing
Labels: high priority, triage review, oncall: distributed, module: fsdp

### 🚀 The feature, motivation and pitch

`test_fsdp_state_dict` currently does not contain any logic to test that buffers are checkpointed appropriately. We should add these tests and also verify that it works for rank0 and offload_to_cpu checkpoint which is being added in https://github.com/pytorch/pytorch/pull/75908

### Alternatives

_No response_

### Additional context

_No response_

cc @ezyang @gchanan @zou3519 @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang
Add batching rules for `{view}_copy` operators
Labels: triaged, module: batching

The plan for the `functionalize()` transform is that it will soon have two modes:
- `functionalize(remove='mutations')` will remove all inplace and out= ops from the function
- `functionalize(remove='mutations_and_views')` will *additionally* convert view operators into their view_copy variants.

You can't use that second version of `functionalize()` together with `vmap` today though, because none of the `view_copy` operators have batching rules.

Most of them can use the fallback batching rule, but some aren't able to because of their schemas, like `at::split_copy`. So maybe we should just ensure that the "problematic" view copy ops have batching rules first.
Move _SKIP_PYTHON_BINDINGS to native_functions.yaml
Labels: triaged, module: codegen

### 🐛 Describe the bug

@swolchok ran into a case where he named a function with `_forward` suffix and that caused it to magically not show up in Python bindings, in a way that was not debuggable until I remembered that there are name based matches. We should move this into a field on `native_functions.yaml` to make it more obvious when python bindings are being suppressed.

### Versions

master

cc @ezyang @bhosmer @bdhirsh
add -D_GLIBCXX_ASSERTIONS in debug mode
Labels: module: build, triaged

### 🚀 The feature, motivation and pitch

Internally, dev builds are built with -D_GLIBCXX_ASSERTIONS, which enables checks like index out-of-bounds in vectors.

See https://github.com/pytorch/pytorch/pull/75766 - which was built with _GLIBCXX_ASSERTIONS and found one error.

cc @malfet @seemethere @janeyx99 

### Alternatives

_No response_

### Additional context

_No response_
INTERNAL ASSERT FAILED at "vulkan_rewrite.cpp":272
Labels: oncall: mobile

### 🐛 Describe the bug

```
from torch import nn
import torch
from torch.utils.mobile_optimizer import optimize_for_mobile

model = nn.Sequential(nn.Conv2d(3, 3, kernel_size=1))
model = model.cpu()
model.eval()

example0 = torch.rand(1, 3, 4, 4)
with torch.no_grad():
    traced = torch.jit.trace(model, example0)

print('torch version is', torch.__version__)
optimized_traced = optimize_for_mobile(traced, backend='vulkan')
optimized_traced._save_for_lite_interpreter("./traced_model_vulkan.ptl")
```
Message
```
torch version is 1.11.0
Traceback (most recent call last):
  File "/home/marat/OCR/yolo3/exmobile.py", line 15, in <module>
    optimized_traced = optimize_for_mobile(traced, backend='vulkan')
  File "/home/marat/anaconda3/envs/cexp/lib/python3.7/site-packages/torch/utils/mobile_optimizer.py", line 67, in optimize_for_mobile
    optimized_cpp_module = torch._C._jit_pass_vulkan_optimize_for_mobile(script_module._c, preserved_methods_str)
RuntimeError: falseINTERNAL ASSERT FAILED at "/opt/conda/conda-bld/pytorch_1646755861072/work/torch/csrc/jit/passes/vulkan_rewrite.cpp":272, please report a bug to PyTorch. Mobile optimizaiton only available with Vulkan at the moment. Vulkan is not enabled. Please build with USE_VULKAN=1
```


### Versions

torch 1.11.0 ubuntu 18.04 installed by 
conda install pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch
Different result with JIT
Labels: oncall: jit

### 🐛 Describe the bug

```python
import torch
# Outputs 1,1,1 if the following line is commented-out, and 1,1,2 if it is not
#@torch.jit.script
def test():
    a = torch.zeros(1)
    b = torch.zeros(1)
    c = torch.zeros(1)
    n = 3
    x = torch.ones(n)
    
    for i in range(n):
        a, b, c = c, a, b
        a[0] += x[i]
        print(a[0].item())

test()
```

### Versions

I ran with PyTorch 1.11.0 on a CPU.

```
PyTorch version: 1.11.0+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: 10.0.0-4ubuntu1 
CMake version: version 3.16.3
Libc version: glibc-2.10

Python version: 3.7.3 (default, Mar 27 2019, 22:11:17)  [GCC 7.3.0] (64-bit runtime)
Python platform: Linux-5.13.0-39-generic-x86_64-with-debian-bullseye-sid
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] botorch==0.1.0
[pip3] gpytorch==0.3.2
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.20.3
[pip3] numpydoc==0.9.2
[pip3] pytorch-lightning==0.9.0rc6
[pip3] torch==1.11.0+cpu
[pip3] torchaudio==0.11.0+cpu
[pip3] torchvision==0.12.0+cpu
[conda] blas                      1.0                         mkl  
[conda] botorch                   0.1.0                    pypi_0    pypi
[conda] cpuonly                   2.0                           0    pytorch
[conda] gpytorch                  0.3.2                    pypi_0    pypi
[conda] mkl                       2020.1                      217  
[conda] mkl-include               2019.3                      199    anaconda
[conda] mkl-service               2.3.0            py37he904b0f_0  
[conda] mkl_fft                   1.0.15           py37ha843d7b_0  
[conda] mkl_random                1.1.1            py37h0573a6f_0  
[conda] mypy-extensions           0.4.3                    pypi_0    pypi
[conda] numpy                     1.20.3                   pypi_0    pypiz
[conda] numpy-base                1.18.1           py37hde5b4d6_1  
[conda] numpydoc                  0.9.2                      py_0  
[conda] pytorch-lightning         0.9.0rc6                 pypi_0    pypi
[conda] pytorch-mutex             1.0                         cpu    pytorch
[conda] torch                     1.10.0+cpu               pypi_0    pypi
[conda] torchaudio                0.11.0+cpu               pypi_0    pypi
[conda] torchvision               0.12.0+cpu               pypi_0    pypi
```
`torch.jit.script` Script functions do return `requires_grad = False` if `torch.no_grad()` has been used
Labels: oncall: jit

### 🐛 Describe the bug

Hello,
I think I found a bug with `torch.jit.script` and `torch.no_grad()`. Here is a code sample to reproduce behavior:

```
import torch


def bar(x):
    with torch.no_grad():
        y = x * 2

    z = 2 * x + y
    print("requires_grad: ", x.requires_grad, y.requires_grad, z.requires_grad)

    return z


scripted_bar = torch.jit.script(bar)

a = torch.rand(3, requires_grad=True)
b = torch.rand(3, requires_grad=True)

print("scripted_bar(a) : ", scripted_bar, scripted_bar(a).requires_grad)
print("bar(b) : ", bar, bar(b).requires_grad)

```

Output:
```
requires_grad:  True False False
scripted_bar(a) :  <torch.jit.ScriptFunction object at 0x7fd538ac63b0> False
requires_grad:  True False True
bar(b) :  <function bar at 0x7fd5ada901f0> True
```

I would expect the scripted and non-scripted function to both return tensors with `requires_grad = True`.


### Versions

Collecting environment information...
PyTorch version: 1.11.0+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.3 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.31

Python version: 3.8.13 (default, Mar 28 2022, 11:38:47)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.13.0-39-generic-x86_64-with-glibc2.17
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: Quadro T1000
Nvidia driver version: 510.54
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.22.3
[pip3] torch==1.11.0
[pip3] torchvision==0.12.0
[conda] mypy-extensions           0.4.3                    pypi_0    pypi
[conda] numpy                     1.22.3                   pypi_0    pypi
[conda] torch                     1.11.0                   pypi_0    pypi
[conda] torchvision               0.12.0                   pypi_0    pypi
`torch.matmul` produces wrong results on A4000 for matrices (n*m) with large m and small n 
Labels: high priority, triaged, module: tf32

### 🐛 Describe the bug

### Description

`torch.matmul` produces wrong result for matrices (n*m) with large m and small n, but not if you transpose it.
This bug is only seen on my A4000 devices but not on 2070 devices.

### Code to reproduce

```
import torch as th
import numpy as np


num_trials = 100
num_failures_cpu = 0
num_failures_gpu = 0

num_failures_cpu_transposed = 0
num_failures_gpu_transposed = 0

stTime = time.time()
for _ in range(num_trials):
    M = th.rand(4, 4) * 100
    X = (th.rand(1000000, 4) - 0.5) * 100


    res1 = X @ M
    res2 = X.cuda() @ M.cuda()
    res3 = X.numpy() @ M.numpy()

    res1_T = M @ X.T
    res2_T = M.cuda() @ X.T.cuda()
    res3_T = M.numpy() @ X.T.numpy()
    
    try:
        np.testing.assert_allclose(res1.cpu().numpy(), res3, atol=1e-1)
    except:
        num_failures_cpu += 1
        
    try:
        np.testing.assert_allclose(res2.cpu().numpy(), res3, atol=1e-1)
    except:
        num_failures_gpu += 1

    try:
        np.testing.assert_allclose(res1_T.cpu().numpy(), res3_T, atol=1e-1)
    except:
        num_failures_cpu_transposed += 1
        
    try:
        np.testing.assert_allclose(res2_T.cpu().numpy(), res3_T, atol=1e-1)
    except:
        num_failures_gpu_transposed += 1
    

print(f"""
CPU failures: {num_failures_cpu / num_trials}
GPU failures: {num_failures_gpu / num_trials}
CPU failures (transposed): {num_failures_cpu_transposed / num_trials}
GPU failures (transposed): {num_failures_gpu_transposed / num_trials}
Total time: {time.time() - stTime}s.
""")
```

### Results
On GTX 2070
```
CPU failures: 0.0
GPU failures: 0.0
CPU failures (transposed): 0.0
GPU failures (transposed): 0.0
```
On A4000
```
CPU failures: 0.0
GPU failures: 0.0
CPU failures (transposed): 0.0
GPU failures (transposed): 1.0
````

### Versions

pytorch 1.10.2 + cuda 11.3

cc @ezyang @gchanan @zou3519 @zasdfgbnm @ptrblck @ngimel
Handle noncontiguous inputs in distributed backend layer
Labels: oncall: distributed, triaged, module: c10d

### 🚀 The feature, motivation and pitch

As inspired by issue "torch.distributed.nn.functional.all_gather: Tensors must be contiguous" #73515

I had quick fix to force the tensors to be contiguous at the distributed.nn.functional layer, see 
https://github.com/pytorch/pytorch/pull/75276

However, I think the proper fix should be implemented in the layer of distributed backend, e.g. torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp, to support tensors that fails "is_non_overlapping_and_dense()" check. Such materialization function can be shared by all NCCL collectives, that requires dense non-overlapping buffer. 

I think there are 2 reasons that it's preferred to be done at the lower layer:
1. tensor.is_non_overlapping_and_dense() is not exposed in python API, and only the tensor that failed this check should be materialized. Calling the .contiguous() universally at the functional layer is an overkill. 
2. All the layers that are built on top of distributed backend layer will not need to worry about noncontiguous inputs. 

### Alternatives

Do it in the distributed.nn.functional layer, as https://github.com/pytorch/pytorch/pull/75276 

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang
Depthwise Conv1d performance (a naive CUDA kernel is 10x faster)
Labels: module: cuda, triaged

### 🚀 The feature, motivation and pitch

Please improve the CUDA performance of Depthwise Conv1d :)

FYI, I write a naive CUDA kernel and it's already 10x faster than pytorch:

https://github.com/BlinkDL/RWKV-CUDA

RTX3090:
pytorch = fwd 14ms bwd 65ms
CUDA kernel v3 = fwd 0.8ms bwd 5.5ms

### Alternatives

_No response_

### Additional context

_No response_

cc @ngimel
torch.device missing doctring
Labels: module: docs, triaged

There is no docstring for `torch.device` for v. 1.12.0.dev20220224.

```python
# Running on JupyterLab
import torch
torch.device(<shift + tab>)

# A pop-up shows the following
Init signature: torch.device(self, /, *args, **kwargs)
Docstring:      <no docstring>
File:           /usr/local/Caskroom/miniconda/base/envs/book/lib/python3.10/site-packages/torch/__init__.py
Type:           type
Subclasses:
```

<details>
  <summary>Versions</summary>
  
```
PyTorch version: 1.12.0.dev20220224
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.3.1 (x86_64)
GCC version: Could not collect
Clang version: 11.0.0 (clang-1100.0.33.12)
CMake version: version 3.23.0
Libc version: N/A

Python version: 3.10.0 (default, Nov 10 2021, 11:24:47) [Clang 12.0.0 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.2
[pip3] torch==1.12.0.dev20220224
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.4.0           hecd8cb5_637  
[conda] mkl-service               2.4.0           py310hca72f7f_0  
[conda] mkl_fft                   1.3.1           py310hf879493_0  
[conda] mkl_random                1.2.2           py310hc081a56_0  
[conda] numpy                     1.21.2          py310h2cbf25c_0  
[conda] numpy-base                1.21.2          py310he490955_0  
[conda] pytorch                   1.12.0.dev20220224        py3.10_0    pytorch-nightly
```

</details>

cc @brianjo @mruberry
`torch.sum, prod, cumsum, cumprod, sparse.sum` INTERNAL ASSERT FAIL
Labels: module: error checking, triaged, module: reductions

### 🐛 Describe the bug

```python
import torch

a = torch.rand([], requires_grad=True)
torch.sum(a, dtype=torch.bool)
```

When the tensor requiring grad and arg `dtype` is `torch.bool`, `torch.sum` will trigger internal assert failed.

Besides, `torch.prod`, `cumsum`, `cumprod` and `sparse.sum` also trigger such internal assert fail

```python
import torch

a = torch.rand([], requires_grad=True)
torch.prod(a, dtype=torch.bool)
```

```python
import torch

a = torch.rand([1], requires_grad=True)
torch.cumsum(a, 0, dtype=torch.int8)
```

```python
import torch

a = torch.rand([1], requires_grad=True)
torch.cumprod(a, 0, dtype=torch.int8)
```

```python
import torch

a = torch.rand([1], requires_grad=True).to_sparse()
torch.sparse.sum(a, dtype=torch.int8)
```

### Versions

1.11.0
Let's host NVIDIA dependencies in our own S3
Labels: module: ci, triaged

### 🚀 The feature, motivation and pitch

To avoid SEVs where we are unable to retrieve a dependency hosted by NVIDIA (see https://github.com/pytorch/pytorch/issues/74967), we should host copies in S3 ourselves. 

### Alternatives

_No response_

### Additional context

This was not possible before but may be possible now.

cc @seemethere @malfet @pytorch/pytorch-dev-infra
Einsum should have an `out=` parameter
Labels: triaged, module: linear algebra

### 🚀 The feature, motivation and pitch

When using `einsum` in performance sensitive code (and not caring about gradients), it is not good that it allocates a new tensor for the result. Then `numpy` version has an `out=` parameter, similar to `bmm` and many other pytorch functions, which allow specifying a piece of memory to store the result. It would be good to have this in pytorch as well.

### Alternatives

Alternatively users can avoid einsum entirely, but there is not always a pytorch function with the same functionality.

### Additional context

_No response_

cc @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano
Addressing skips in OpInfo nn.functional.binary_cross_entropy_with_logits
Labels: module: nn, module: tests, triaged

### 🐛 Describe the bug

While implementing the OpInfo for nn.functional.binary_cross_entropy_with_logits a large number of skips were needed to pass the related tests. The PR for creating this ops info can be found here: #75604 

Skips:

``` Python
 skips=(
            # The Weight tensor requires_grad = False
            DecorateInfo(
                unittest.skip("Skipped!"),
                "TestCommon",
                "test_floating_inputs_are_differentiable",
                dtypes=(torch.float32,)
            ),
            # Adding OpInfo to existing operator
            DecorateInfo(
                unittest.skip("Skipped!"),
                "TestCompositeCompliance",
                "test_backward",
                dtypes=(torch.float32,)
            ),
            # Pos Weight is required to be positve
            DecorateInfo(
                unittest.skip("Skipped!"),
                "TestMathBits",
                "test_neg_view",
                dtypes=(torch.float64,)
            ),
            # Test Gradient failures CI
            DecorateInfo(
                unittest.skip("Skipped!"),
                "TestGradients",
                "test_neg_view",
                dtypes=(torch.float64,)
            ),
            DecorateInfo(
                unittest.skip("Skipped!"),
                'TestJit',
                'test_variant_consistency_jit',
                dtypes=(torch.float32,)
            ),
            DecorateInfo(unittest.skip("Skipped!"), 'TestGradients', "test_fn_gradgrad", dtypes=(torch.float64,)),
            DecorateInfo(unittest.skip("Skipped!"), 'TestGradients', "test_forward_mode_AD", dtypes=(torch.float64,)),
            DecorateInfo(unittest.skip("Skipped!"), 'TestGradients', "test_fn_fwgrad_bwgrad", dtypes=(torch.float64,)),
        ),
```


### Versions

Not version specific

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345
Tensorboard Issue with visualizing the connections of encoder-decoder network
Labels: oncall: visualization

We are trying to visualize the connections of the intermediate layers of a deep neural network using tensorboard. Our implementation is below:

```
    model = build_model(cfg) 
    path = 'output/model_0004999.pth' 
    model.load_state_dict(torch.load(path)['model'])  

    # get some random training images
    print(trainloader)
    dataiter = iter(trainloader)
    images = dataiter.next()

    # create grid of images 
    img_grid = torchvision.utils.make_grid(images)

    writer=SummaryWriter('content/logsdir')

    # show images
    matplotlib_imshow(img_grid, one_channel=True)

    # write to tensorboard
    writer.add_image('mt_images', img_grid)
    writer.add_graph(model, images)
    writer.close()

```

However, we keep getting this error pointing to `jit`

```
Only tensors, lists, tuples of tensors, or dictionary of tensors can be output from traced functions
Error occurs, No graph saved
Traceback (most recent call last):
  File "convert_model_tensorboard.py", line 96, in <module>
    writer.add_graph(model, [images])
  File "/home/ubuntu/user/.virtualenv/lib/python3.6/site-packages/torch/utils/tensorboard/writer.py", line 723, in add_graph
    self._get_file_writer().add_graph(graph(model, input_to_model, verbose))
  File "/home/ubuntu/user/.virtualenv/lib/python3.6/site-packages/torch/utils/tensorboard/_pytorch_graph.py", line 293, in graph
    raise e
  File "/home/ubuntu/user/.virtualenv/lib/python3.6/site-packages/torch/utils/tensorboard/_pytorch_graph.py", line 286, in graph
    trace = torch.jit.trace(model, args)
  File "/home/ubuntu/user/.virtualenv/lib/python3.6/site-packages/torch/jit/_trace.py", line 742, in trace
    _module_class,
  File "/home/ubuntu/user/.virtualenv/lib/python3.6/site-packages/torch/jit/_trace.py", line 940, in trace_module
    _force_outplace,
RuntimeError: Only tensors, lists, tuples of tensors, or dictionary of tensors can be output from traced functions
```

### Versions

```
Python 3.6.9

torch==1.8.0
torchsummary==1.5.1
torchvision==0.9.0
tensorboard==2.8.0
```
Implement histc for bfloat16 on CPU
Labels: triaged, module: bfloat16, module: sorting and selection

### 🚀 The feature, motivation and pitch

Add support for `histc` using bloat16 on cpu.

Currently, this yields the error:
```
RuntimeError: "histogram_cpu" not implemented for 'BFloat16'
```
as detailed in [this issue at wandb](https://github.com/wandb/client/issues/3332)

### Alternatives

_No response_

### Additional context

_No response_
Off main thread symbolic evaluation
Labels: module: internals, triaged, enhancement, oncall: pt2

Standard JIT design has compilation run off-thread, so that normal user code can continue running while compilation is still occurring. Unfortunately, torchdynamo's implementation in Python makes this difficult to do, as in standard Python there is a GIL which prevents useful parallelization of Python work. This means that we need to think about our off main thread strategy earlier rather than later, as it may imply architectural changes that are easier to do earlier in this project's lifetime.

I can imagine a number of possible ways to skin this cat:

* Put torchdynamo evaluation in a separate process
* Put torchdynamo in a subinterpreter in the same process
  * All strategies involving a separate Python interpreter will have to somehow transmit all of the frame information as well as locals for evaluation
* Rewrite torchdynamo in C++ (ugh)
* Rewrite torchdynamo in RPython
* Do nothing and pray the host program spends enough time in PyTorch operations that we can make useful progress with standard threading
* Do nothing for symbolic evaluation, but offload FX graph compilation to another thread

It would be good to know the PoR here.

cc @bhosmer @smessmer @ljk53 @bdhirsh @soumith @msaroufim @wconstab @ngimel
multiprocessing and torch.tensor, Cannot allocate memory error
Labels: module: multiprocessing, triaged

### 🐛 Describe the bug

Consider the simple example in the following, if I return a list at `func`, there will be no error, 
if I return a `torch.tensor` I will get this error:

```sh
/work/aten/src/ATen/MapAllocator.cpp":263, please report a bug to PyTorch. unable to open shared memory object </torch_326297_506> in read-write mode')'
```
and If it use  `set_sharing_strategy`, to avoid this error:

```python
from multiprocessing import Pool
from time import sleep
import torch


class A:
    def __init__(self, n_job, N):
        self.n_job = n_job
        self.N = N

    
    def func(self, i):
        # sleep(1)
        print(i)
        return torch.as_tensor([i])
        # return [i]

    data = []

    def read(self):

        torch.multiprocessing.set_sharing_strategy('file_system')

        with Pool(processes=self.n_job) as pool:
            data = pool.map(self.func, range(self.N))
    
        return data
    
if __name__ == "__main__":
    a = A(10, 2**16)
    data = a.read()


    print(len(data))
```

I get another error says 

```sh 
MapAllocator.cpp":323, please report a bug to PyTorch. unable to mmap 72 bytes from file </torch_326588_5707>: Cannot allocate memory (12)
```

is there any solution?
I need to return the tensor, so just removing the tensor is not an option.

I am using 

### Versions

```sh 
PyTorch version: 1.10.2
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.3 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: 10.0.0-4ubuntu1 
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.9.7 (default, Sep 16 2021, 13:09:58)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.11.0-27-generic-x86_64-with-glibc2.31
Is CUDA available: False
CUDA runtime version: 11.6.112
GPU models and configuration: GPU 0: NVIDIA RTX A5000
Nvidia driver version: 510.47.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.5
[pip3] torch==1.10.2
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               11.0.3              h15472ef_10    conda-forge
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py39h7f8727e_0  
[conda] mkl_fft                   1.3.1            py39hd3c417c_0  
[conda] mkl_random                1.2.2            py39h51133e4_0  
[conda] numpy                     1.21.5                   pypi_0    pypi
[conda] numpy-base                1.21.2           py39h79a1101_0  
[conda] pytorch                   1.10.2          cpu_py39hfa7516b_0
```

cc @VitalyFedyunin
Misleading documentation for cholesky_inverse
Labels: module: docs, triaged, module: linear algebra

### 📚 The doc issue

https://pytorch.org/docs/stable/generated/torch.cholesky_inverse.html

The documentation suggests that the inputted matrix should be the original positive-definite matrix A, where as the examples show that the user must first compute the cholesky u and input that into the cholesky_inverse function.

### Suggest a potential alternative/fix

Change the parameters section so that it is clear that the input must be the cholesky decomposition.

cc @svekars @holly1238 @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano @brianjo
jit fails when trying to assign values to model via hook
Labels: oncall: jit

### 🐛 Describe the bug

I'm using hooks to assign new values to the model on forward passes. Jit script fails with:
```
RuntimeError: 
attribute assignment is not defined on python value of type 'Model Name':
```

For Example:

```python
from typing import Iterable, Callable, Tuple

from torch import Tensor, nn, jit
from torchvision.models import resnet50


class FeatureExtractor(nn.Module):
    def __init__(self, model: nn.Module, layers: Iterable[str], num):
        super().__init__()
        self.model = model
        self.num = num

        for layer_id in layers:
            layer = dict([*self.model.named_modules()])[layer_id]
            layer.register_forward_hook(self.save_outputs_hook(layer_id))

    def save_outputs_hook(self, layer_id: str) -> Callable:
        fe = self
        def fn(_, input: Tuple[Tensor], output):
            fe.num = 3
        return fn

    def forward(self, x: Tensor):
        _ = self.model(x)


if __name__ == '__main__':
    resnet_features = FeatureExtractor(resnet50(), layers=["layer4", "avgpool"], num=0)
    script = jit.script(resnet_features)
```

Fails with 
```
RuntimeError: 
attribute assignment is not defined on python value of type 'FeatureExtractor':
  File "", line 20
        def fn(_, input: Tuple[Tensor], output):
            fe.num = 3
            ~~~~~~~~~~ <--- HERE

```

### Versions

PyTorch version: 1.11.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 11.6 (x86_64)
GCC version: Could not collect
Clang version: 13.0.0 (clang-1300.0.29.30)
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.8 (main, Nov 10 2021, 09:21:22)  [Clang 13.0.0 (clang-1300.0.29.3)] (64-bit runtime)
Python platform: macOS-11.6-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] pytorch-lightning==1.6.0
[pip3] torch==1.11.0
[pip3] torchmetrics==0.7.3
[pip3] torchvision==0.12.0
[conda] Could not collect
Op segfaults with ForwardAD and Subclassed Tensor as Tangent
Labels: triaged, module: forward ad

### 🐛 Describe the bug

```python
import torch
import torch.autograd.forward_ad as fwAD
from torch.utils._pytree import tree_map
import contextlib

@contextlib.contextmanager
def no_dispatch():
    guard = torch._C._DisableTorchDispatch()
    try:
        yield
    finally:
        del guard

class CompositeCompliantTensor(torch.Tensor):
    elem: torch.Tensor

    __slots__ = ['elem']

    __torch_function__ = torch._C._disabled_torch_function_impl

    @staticmethod
    def __new__(cls, elem, *args, **kwargs):
        r = torch.Tensor._make_wrapper_subclass(  # type: ignore[attr-defined]
            cls, elem.size(),
            dtype=elem.dtype, layout=elem.layout,
            device=elem.device, requires_grad=elem.requires_grad,
            strides=elem.stride(), storage_offset=elem.storage_offset())

        if elem.requires_grad:
            r.elem = elem.detach().clone()
        else:
            r.elem = elem
        
        return r

    def __repr__(self):
        return f"CompositeCompliantTensor({self.elem})"

    @classmethod
    def __torch_dispatch__(cls, func, types, args=(), kwargs=None):
        def unwrap(e):
            return e.elem if isinstance(e, CompositeCompliantTensor) else e

        def wrap(e):
            return CompositeCompliantTensor(e) if isinstance(e, torch.Tensor) else e

        with no_dispatch():
            unwrapped_args = tree_map(unwrap, args)
            unwrapped_kwargs = tree_map(unwrap, kwargs)
            unwrapped_rs = func(*unwrapped_args, **unwrapped_kwargs)
            rs = tree_map(wrap, unwrapped_rs)

        return rs

CCT = CompositeCompliantTensor

def fn(x, y):
    return torch.add(x, y)

with fwAD.dual_level():
    x_f = CCT(torch.tensor(-8.4784, requires_grad=True))
    y_f = CCT(torch.tensor(-1.7658, requires_grad=True))

    dual_input_1 = fwAD.make_dual(x_f, torch.randn_like(x_f))
    dual_input_2 = fwAD.make_dual(y_f, torch.randn_like(y_f))
    out = fn(dual_input_1, dual_input_2)
    print("1")

    dual_input_1 = fwAD.make_dual(x_f, torch.randn_like(x_f))
    dual_input_2 = fwAD.make_dual(y_f, CCT(torch.randn_like(y_f)))
    out = fn(dual_input_1, dual_input_2)
    print("2")

```

The following script segfaults while performing the second operation.

**NOTE**: Script works if we remove `with no_dispatch` in `__torch__dispatch__`.

### Versions

master

cc: @zou3519 @soulitzer
[ONNX] Support unit tests in scripting that we already support in tracing
Labels: module: onnx, triaged, onnx-triaged

- [x] Enable scripting tests that are currently working due to recent changes in pytorch export https://github.com/pytorch/pytorch/pull/77254
- [ ] Non-scriptable operators (as_strided, transformer_encoder)
- [ ] Dict type (ONNX map) not supported https://github.com/pytorch/pytorch/issues/81482
- [ ] Shape/type inference issues
- [ ] Issues related to support for kwargs/args https://github.com/pytorch/pytorch/issues/81478
- [ ] args* issues post compilation
- [ ] Support for optional type inference
- [ ] Input/output shape issues
- [ ] Issues due to ONNX Spec Gaps
- [ ] Failures from jit_pass_lower_tuple
- [ ] Failures from jit_pass_mutation https://github.com/pytorch/pytorch/pull/79555
- [ ] User defined/custom class support
- [ ] Model code not scriptable
- [ ] Unscriptable modules
- [x] Support for Union Types https://github.com/pytorch/pytorch/pull/77254



Migrated from the work item:
<https://microsoft.sharepoint.com/:x:/r/teams/ONNX2/_layouts/15/Doc.aspx?sourcedoc=%7B2F53DDC4-54E9-488F-B5CB-C114937A3E90%7D&file=TorchScript%20Export%20Issues.xlsx&wdOrigin=OFFICECOM-WEB.MAIN.SEARCH&ct=1636674278879&action=default&mobileredirect=true&share=IQHE3VMv6VSPSLXLwRSTej6QAWZK7HjHmHw9xD4UMRwbV4U&cid=9094dc65-62dd-4898-86b4-84904add63f7>
kthvalue 20x slower than sort 
Labels: module: performance, triaged, module: sorting and selection

### 🐛 Describe the bug

An example task: find 2nd smallest element in a huge tensor on a cuda-device. 

1) First approach via `torch.sort`:
```python
import torch
import time

elapsed = 0.0
runs = 100
for i in range(runs):
    tensor = torch.rand(100_000_000, device='cuda:0')

    torch.cuda.synchronize()
    start = time.time()

    sorted, _ = tensor.sort()
    second_smallest_el = sorted[1]

    torch.cuda.synchronize()
    end = time.time()
    elapsed += end - start

print(elapsed/runs)  # 0.03 sec
```
2) Second approach via `torch.kthvalue`:
```python
import torch
import time

elapsed = 0.0
runs = 100
for i in range(runs):
    tensor = torch.rand(100_000_000, device='cuda:0')

    torch.cuda.synchronize()
    start = time.time()

    val, _ = torch.kthvalue(tensor, 2)

    torch.cuda.synchronize()
    end = time.time()
    elapsed += end - start

print(elapsed/runs)  # 0.68 sec
```

I would expect much better running time of the `kthvalue`, especially when `k` is as small as `2`.

### Versions

Collecting environment information...
PyTorch version: 1.11.0
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux 10 (buster) (x86_64)
GCC version: (Debian 8.3.0-6) 8.3.0
Clang version: Could not collect
CMake version: version 3.13.4
Libc version: glibc-2.28

Python version: 3.8.12 (default, Oct 12 2021, 13:49:34)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-4.19.0-19-amd64-x86_64-with-glibc2.17
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration:
GPU 0: GeForce RTX 3090

Nvidia driver version: 460.73.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               11.3.1               h2bc3f7f_2
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.4.0           h06a4308_640
[conda] mkl-service               2.4.0            py38h7f8727e_0
[conda] mkl_fft                   1.3.1            py38hd3c417c_0
[conda] mkl_random                1.2.2            py38h51133e4_0
[conda] mypy-extensions           0.4.3                    pypi_0    pypi
[conda] numpy                     1.22.3                   pypi_0    pypi
[conda] numpy-base                1.21.2           py38h79a1101_0
[conda] pytorch                   1.11.0          py3.8_cuda11.3_cudnn8.2.0_0    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torchvision               0.12.0               py38_cu113    pytorch

cc @VitalyFedyunin @ngimel
Add ZeroTensor support for `mm`
Labels: module: performance, module: autograd, triaged, actionable, ZeroTensor

As per the title.

example PR: https://github.com/pytorch/pytorch/pull/71129

cc @VitalyFedyunin @ngimel @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
Add `balance` flag to `random_split`
Labels: triaged, enhancement, module: data

### 🚀 The feature, motivation and pitch

It will be much easier to add a balance flag tp `random_split` for balancing between classes and not needing to go to `Sampler` or `train_test_split` from `sklearn`.

### Alternatives

`Sampler` or `train_test_split` from `sklearn`

### Additional context

Just make it for me, please ^^

cc @VitalyFedyunin @ejguan @NivekT