gopikrsmscs/torch-issues huggingface dataset:
topk returns different results with the same input in cuda and cpu
Labels: module: cuda, triaged, module: sorting and selection

### 🐛 Describe the bug

torch.topk() has different results for input located at cuda and cpu. To reproduce my error, i'm putting my python script and data at https://github.com/Tengxu-Sun/Pytorch-Topk-Bug.

### Versions

pytorch: 1.10.2
cudakit: 11.3
GPU: 2080 Ti
CPU: Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GHz

According to my test, pytorch 1.9, 1.8, 1.7 and 1.6 also have this error.

### Expected behavior
CUDA and CPU return the same result and diff should be 0.

cc @ngimel
Segmentation fault in native_batch_norm
Labels: module: crash, module: error checking, triaged, module: norms and normalization

### 🐛 Describe the bug

Segmentation fault in `native_batch_norm`.

### Example to reproduce

```python
import torch

input = torch.full((5, 5,), 1, dtype=torch.float64, requires_grad=False)
weight = torch.full((14, 9, 12, 0, 6, 0, 15, 0, 0, 0,), -1.5e+300, dtype=torch.float64, requires_grad=False)
bias = torch.full((5,), 1, dtype=torch.float64, requires_grad=False)
running_mean = torch.full((0,), 1, dtype=torch.float64, requires_grad=False)
running_var = torch.full((0,), 1, dtype=torch.float64, requires_grad=False)
training = True
momentum = 0
eps = 0
torch.native_batch_norm(input, weight, bias, running_mean, running_var, training, momentum, eps)
```

### Result
```segmentation fault```

### Expected behavior
Graceful termination or a RuntimeError to be thrown.

### Note
This bug was discovered using fuzz testing.

### Versions

PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux 11 (bullseye) (x86_64)
GCC version: (Debian 10.2.1-6) 10.2.1 20210110
Clang version: 11.0.1-2
CMake version: version 3.22.1
Libc version: glibc-2.31

Python version: 3.9.12 (main, Jun  1 2022, 11:38:51)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.10.0-16-amd64-x86_64-with-glibc2.31
Is CUDA available: N/A
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] torch==1.11.0a0+gitbc2c6ed
[conda] mkl                       2022.0.1           h06a4308_117
[conda] mkl-include               2022.0.1           h06a4308_117
[conda] numpy                     1.22.3           py39h7a5d4dd_0
[conda] numpy-base                1.22.3           py39hb8be1f0_0
[conda] torch                     1.11.0a0+gitbc2c6ed          pypi_0    pypi
Floating point exception in gather gradient computation.
Labels: module: crash, module: error checking, triaged, module: edge cases

### 🐛 Describe the bug

Floating point exception in `gather` gradient computation.

### Example to reproduce

```python
import torch

grad = torch.full((2, 0, 0, 6, 6,), 0, dtype=torch.float64, requires_grad=True)
self = torch.full((2, 0, 0, 6, 6,), 0, dtype=torch.float64, requires_grad=True)
dim = 0
index = torch.full((2, 0, 0, 6, 6,), 0, dtype=torch.float64, requires_grad=True)
sparse_grad = True
res = torch.gather(self, dim, index, sparse_grad=sparse_grad)
grad_out = torch.zeros_like(res)
torch.autograd.backward(res, grad_tensors=grad_out)
```

### Result
```floating point exception```

### Expected behavior
Graceful termination or a RuntimeError to be thrown.

### Note
This bug was discovered using fuzz testing.

### Versions

PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux 11 (bullseye) (x86_64)
GCC version: (Debian 10.2.1-6) 10.2.1 20210110
Clang version: 11.0.1-2
CMake version: version 3.22.1
Libc version: glibc-2.31

Python version: 3.9.12 (main, Jun  1 2022, 11:38:51)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.10.0-16-amd64-x86_64-with-glibc2.31
Is CUDA available: N/A
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] torch==1.11.0a0+gitbc2c6ed
[conda] mkl                       2022.0.1           h06a4308_117
[conda] mkl-include               2022.0.1           h06a4308_117
[conda] numpy                     1.22.3           py39h7a5d4dd_0
[conda] numpy-base                1.22.3           py39hb8be1f0_0
[conda] torch                     1.11.0a0+gitbc2c6ed          pypi_0    pypi
Segmentation fault in mkldnn_reorder_conv2d_weight and mkldnn_reorder_conv3d_weight
Labels: module: crash, triaged, module: mkldnn, module: edge cases

### 🐛 Describe the bug

Segmentation fault in `mkldnn_reorder_conv2d_weight` and  `mkldnn_reorder_conv3d_weight`.

### Example to reproduce

```python
import torch

self = torch.full((3, 3, 1, 1,), 1, dtype=torch.float32, requires_grad=False).to_mkldnn()
padding = []
stride = [65534, 65534]
dilation = [65534, 65534]
groups = 0
torch._C._nn.mkldnn_reorder_conv2d_weight(self, padding, stride, dilation, groups)


import torch

self = torch.full((32, 3, 3, 3, 3,), 1, dtype=torch.float32, requires_grad=False).to_mkldnn()
padding = []
stride = [1250999896764, 1250999896764, 1250999896764]
dilation = [1250999896764, 1250999896764, 1250999896764]
groups = 0
torch._C._nn.mkldnn_reorder_conv3d_weight(self, padding, stride, dilation, groups)
```

### Result
```segmentation fault```

### Expected behavior
Graceful termination or a RuntimeError to be thrown.

### Note
This bug was discovered using fuzz testing.

### Versions

PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux 11 (bullseye) (x86_64)
GCC version: (Debian 10.2.1-6) 10.2.1 20210110
Clang version: 11.0.1-2
CMake version: version 3.22.1
Libc version: glibc-2.31

Python version: 3.9.12 (main, Jun  1 2022, 11:38:51)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.10.0-16-amd64-x86_64-with-glibc2.31
Is CUDA available: N/A
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] torch==1.11.0a0+gitbc2c6ed
[conda] mkl                       2022.0.1           h06a4308_117
[conda] mkl-include               2022.0.1           h06a4308_117
[conda] numpy                     1.22.3           py39h7a5d4dd_0
[conda] numpy-base                1.22.3           py39hb8be1f0_0
[conda] torch                     1.11.0a0+gitbc2c6ed          pypi_0    pypi

cc @ezyang @gchanan @zou3519 @gujinghui @PenghuiCheng @XiaobingSuper @jianyuh @VitalyFedyunin
OSError libstdc++.so.6 at import
Labels: needs reproduction, oncall: binaries

### 🐛 Describe the bug

I have a fresh virtual environment with pytorch installed from `pip install -r requirements.txt` which contain 
```
tensorly
numpy
torch
torchvision
torchtyping
scipy
opt_einsum
matplotlib
tqdm
prettytable
antares
numba
-e .
```
I then run a simple `import torch` which gives the following output:
```
python3
Python 3.10.6 (main, Aug  1 2022, 20:38:21) [GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sebwood/workspace/MemSE/.env/lib/python3.10/site-packages/torch/__init__.py", line 201, in <module>
    _load_global_deps()
  File "/home/sebwood/workspace/MemSE/.env/lib/python3.10/site-packages/torch/__init__.py", line 154, in _load_global_deps
    ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
  File "/home/linuxbrew/.linuxbrew/opt/python@3.10/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libstdc++.so.6: cannot open shared object file: No such file or directory
```

### Versions

```
python collect_env.py 
Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.5 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.10.6 (main, Aug  1 2022, 20:38:21) [GCC 5.4.0 20160609] (64-bit runtime)
Python platform: Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Is CUDA available: N/A
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: NVIDIA GeForce GTX 1650 SUPER
Nvidia driver version: 516.94
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.12.1
[pip3] torchtyping==0.1.4
[pip3] torchvision==0.13.1
[conda] Could not collect
```

cc @ezyang @seemethere @malfet
performance between manually created graph and CUDAGraph.replay
Labels: triaged, module: cuda graphs

### 🐛 Describe the bug

I manually designed a simple neural network and made some experiments as follows:
![image](https://user-images.githubusercontent.com/73052387/190850338-7f8ff363-4870-4e0a-a9cf-9aa7fa3ac3a1.png)

class **_ManuallyCreatedGraph_** is my manually designed in source code , which inputs a captured graph and calls cuda runtime API **_cudaGraphAddChildGraphNode_** to construct a new cuda graph and launches it

I found the duration by **_torch.cuda.CUDAGraph.replay_** is obviously shorter than that of **_ManuallyCreatedGraph_**. I was confused .
Could you give some suggestions？ Thanks in advance!

### Versions

pytorch version : 1.12.0a0+git818477c
cuda version : 11.7

cc @mcarilli @ezyang
pytorch core test failure: RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead.
Labels: triaged, oncall: pt2, module: aotdispatch

Repro:
cherry-pick https://github.com/pytorch/pytorch/pull/85183
```
PYTORCH_TEST_WITH_INDUCTOR=1 pytest test/distributions/test_distributions.py  -k  test_pareto
```
Error:
```
Traceback (most recent call last):
  File "/fsx/users/binbao/pytorch/test/distributions/test_distributions.py", line 2556, in test_pareto
    self._check_log_prob(Pareto(scale, alpha), ref_log_prob)
  File "/fsx/users/binbao/pytorch/test/distributions/test_distributions.py", line 837, in _check_log_prob
    asset_fn(i, val.squeeze(), log_prob)
  File "/fsx/users/binbao/pytorch/test/distributions/test_distributions.py", line 2550, in ref_log_prob
    def ref_log_prob(idx, x, log_prob):
  File "/fsx/users/binbao/conda/envs/release/lib/python3.9/site-packages/scipy/stats/_distn_infrastructure.py", line 1884, in logpdf
    def logpdf(self, x, *args, **kwds):
  File "/fsx/users/binbao/conda/envs/release/lib/python3.9/site-packages/scipy/stats/_distn_infrastructure.py", line 1907, in <graph break in logpdf>
    args, loc, scale = self._parse_args(*args, **kwds)
  File "/fsx/users/binbao/conda/envs/release/lib/python3.9/site-packages/scipy/stats/_distn_infrastructure.py", line 1908, in <graph break in logpdf>
    x, loc, scale = map(asarray, (x, loc, scale))
  File "/fsx/users/binbao/pytorch/torch/_tensor.py", line 945, in __array__
    return self.numpy()
RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead.
----------------------------------------------------------------------------------------------------- Captured log call -----------------------------------------------------------------------------------------------------
ERROR    torchdynamo.convert_frame:convert_frame.py:357 WON'T CONVERT test_pareto /fsx/users/binbao/pytorch/test/distributions/test_distributions.py line 2535 
due to: 
Traceback (most recent call last):
  File "/fsx/users/binbao/torchdynamo-tip/torchdynamo/variables/tensor.py", line 266, in create
    assert (
AssertionError: torch.* op returned non-Tensor dtype call_function <built-in function get_default_dtype>

from user code:
   File "/fsx/users/binbao/pytorch/test/distributions/test_distributions.py", line 2541, in test_pareto
    self.assertEqual(Pareto(scale_1d, 0.5).mean, inf)

Set torchdynamo.config.verbose=True for more information
```

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
NestedTensor 2.0 issue tracking
Labels: triaged, module: nestedtensor

# Summary 
This is used as a top level doc to track all 1.13 issues for nestedtensors

# Issues:
- [x] Complete #83775 (move`torch.nested_tensor`)
- [x] Improve documentation in torch.nested, document op coverage https://github.com/pytorch/pytorch/pull/85186
- [x] Improve perf of NT matmul #85064 https://github.com/pytorch/pytorch/pull/85311
- [x] Improve perf of NT bmm ~https://github.com/pytorch/pytorch/pull/85441~ https://github.com/pytorch/pytorch/pull/85894
- [x] Add fastpath for matmul using bmm https://github.com/pytorch/pytorch/pull/85857
- [x] ~Improve perf of NT softmax https://github.com/pytorch/pytorch/pull/85441~ approach seemed to regress for bmm
- [x] https://github.com/pytorch/pytorch/issues/85376 https://github.com/pytorch/pytorch/pull/85593
- [x] Only allow one "-1" in reshape https://github.com/pytorch/pytorch/pull/85691
- [x] ~[Tutorial](https://pytorch.org/tutorials/prototype/nestedtensor) display issues~
- [x]  The [NestedTensor Tutorial](https://github.com/pytorch/tutorials/blob/main/prototype_source/nestedtensor.py) has been updated to have more reasonable baseline to compare against and utilizes _sdpa under the hood through nn.mha.
- [ ] #91508



# Non Blocking issues:
- [ ] #91471
# Push
- [ ] Do we want to expose `from_padded_tensor` or potentially `torch._nested_from_padded_and_nested_example`?


cc @cpuhrsch @jbschlosser @bhosmer @mikaylagawarecki
torch::quantile performance?
Labels: module: performance, triaged

### 🐛 Describe the bug

Computing quantiles is ~10X slower than expected

```python
import time
import torch
import numpy as np

t = torch.rand(1_000_000)
a = t.numpy()

start = time.time()
t.median()
end = time.time()
print("torch median", end - start)

start = time.time()
t.quantile(torch.tensor([0.25, 0.75]))
end = time.time()
print("torch quant ", end - start)

start = time.time()
np.median(a)
end = time.time()
print("numpy median", end - start)

start = time.time()
np.quantile(a, [0.25, 0.75])
end = time.time()
print("numpy quant ", end - start)
```

```
torch median 0.013309478759765625
torch quant  0.10049819946289062
numpy median 0.012769222259521484
numpy quant  0.014006376266479492
```

### Versions

```
Collecting environment information...
PyTorch version: 1.12.1+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.3 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: version 3.22.4
Libc version: glibc-2.31

Python version: 3.8.10 (default, Nov 26 2021, 20:14:08)  [GCC 9.3.0] (64-bit runtime)
Python platform: Linux-5.13.0-27-generic-x86_64-with-glibc2.29
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: NVIDIA A100-PCIE-40GB
GPU 1: NVIDIA A100-PCIE-40GB
GPU 2: NVIDIA A100-PCIE-40GB
GPU 3: NVIDIA A100-PCIE-40GB

Nvidia driver version: 495.46
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.3.2
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.3.2
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.3.2
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.3.2
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.3.2
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.3.2
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.3.2
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.5
[pip3] torch==1.12.1+cu113
[conda] Could not collect
```

cc @VitalyFedyunin @ngimel
PT Dispatcher confusing error message "There were no tensor arguments to this function"
Labels: triaged, module: dispatch

### 🐛 Describe the bug

I am utilizing the dispatcher to call an operator that I implemented

```c++
c10::intrusive_ptr<Work> allgather(
    const c10::intrusive_ptr<ProcessGroup>& process_group,
    const std::vector<std::vector<at::Tensor>>& output_tensors,
    const std::vector<at::Tensor>& input_tensors,
    const AllgatherOptions& opts) {
  static auto op = c10::Dispatcher::singleton()
                       .findSchemaOrThrow("c10d::allgather_", "")
                       .typed<std::tuple<
                           std::vector<std::vector<at::Tensor>>,
                           c10::intrusive_ptr<Work>>(
                           const std::vector<std::vector<at::Tensor>>&,
                           const std::vector<at::Tensor>&,
                           const c10::intrusive_ptr<::c10d::ProcessGroup>&,
                           int64_t)>();
  return std::get<1>(op.call(
      output_tensors, input_tensors, process_group, opts.timeout.count()));
}
```

and hitting an error at `op.call()`

```
NotImplementedError: There were no tensor arguments to this function (e.g., you passed an empty list of Tensors), but no fallback function is registered for schema c10d::allgather_.  This usually means that this function requires a non-empty list of Tensors, or that you (the operator writer) forgot to register a fallback function.  Available functions are [CPU, CUDA, BackendSelect, Python, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradMPS, AutogradXPU, AutogradHPU, AutogradLazy, Tracer, AutocastCPU, AutocastCUDA, Batched, VmapMode, PythonTLSSnapshot].
```

I'm not sure how to reproduce this issue on current operators. Here is a small PR that you can check out and run to reproduce the error: https://github.com/pytorch/pytorch/pull/85115

### Versions

trunk
Need operator fallback stats
Labels: triaged, oncall: pt2

In our internal benchmark enablement work, we had to set `.debug=True` and eyeball the logs to know if inductor or nvfuser is running. If we have the stats for which ops fall back to eager vs. which ops are run with inductor (or nvfuser) it will be much easier to tell if the desired backend is being used, which will be super helpful.

Format-wise it could be (just one idea):
op_name | fallback to eager
| --- | ----------- |
aten::mm | no
aten::cat | yes

Best if we can show the source line of the op (where it is in the original module file), but without it is fine too.

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
Need easier way to tell which step of the optimized path fails (dynamo + inductor)
Labels: triaged, oncall: pt2, module: dynamo

During internal benchmark enablement, there are multiple times where Dynamo silently starts to not generate any graph and not calling Inductor, but the workflow doesn't throw any error because Dynamo is (rightfully) "optional" with graceful eager fallback.

I learned from Michael Lazos that he plans to add operator stats to show which ops fall back to eager vs. using Inductor. But I believe more importantly we need easier way to tell which step of the compilation workflow has issue that causes the fallback, so that on the performance engineer side we can go in and figure out why the fallback happens.

Specifically it would be great to have:
1. A mode where we always error on fallback (maybe some ops only have fallback? in that case can have whitelist for those ops)
2. We want a way to know which step failed. The logs is currently a bit long and I also don’t know exactly what they mean in terms of what fails. If we can have “Step 1: XYZ, Step 2: XYZ, Step 3: XYZ, failed!” printed in the log, it’s easier to know that it's Step 3 that failed.

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire
Minifier improvements/consistency
Labels: triaged, oncall: pt2, module: minifier

- [x] Add tests
- [x] TORCHDYNAMO_REPRO_AFTER="dynamo" should also write to `minifier_launcher.py` similar to "aot" counterpart.
- [x] Extending accuracy testing for REPRO_AFTER="aot"
- [x] Consistent usage of REPRO_LEVEL between "dynamo" and "aot"
- [x] Better code sharing wherever possible
- [x] Enable seg fault repro as well for "Dynamo". It already exists for "aot"

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
install libtorch cxx11 ABI as default in PyTorch pip installation
Labels: oncall: binaries, triaged

### 🚀 The feature, motivation and pitch

Existing ["pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cpu"](https://github.com/awslabs/multi-model-server/blob/c58e29b29faf8e45abcb73088b011f86b40845ef/docs/batch_inference_with_mms.md#L1) installs libtorch Pre-cxx11 ABI on linux. 

TorchServe cpp uses folly whose build requires CXX11_ABI on linux. So TorchSever is not able to use libtorch Pre-cxx11 ABI from pip installation on linux. It has to[ install libtorch again](https://github.com/pytorch/serve/blob/cpp_backend_lxning/cpp/build.sh#L160) on linux; otherwise, the build gets undefined reference/unresolved symbol errors.

However, there is no problem on Mac. For example, TorchSever [uses libtorch](https://github.com/pytorch/serve/blob/cpp_backend_lxning/cpp/build.sh#L238) from "pip3 install torch torchvision torchaudio".

Is it possible to install libtorch cxx11 ABI as default in PyTorch pip installation on linux?





### Alternatives

_No response_

### Additional context

_No response_

cc @seemethere @malfet @osalpekar @atalman
Update `use_deterministic_algorithms` documentation and tests to include `nn.functional` counterparts for all `nn` modules
Labels: module: docs, module: tests, triaged, module: determinism

### 📚 The doc issue

The `nn.functional` counterparts for all of the `nn` modules mentioned in the [`use_deterministic_algorithms` documentation](https://pytorch.org/docs/1.12/generated/torch.use_deterministic_algorithms.html?highlight=use_deterministic#torch.use_deterministic_algorithms) share the same determinism behavior, because the modules just call the functional variant. The documentation doesn't make that clear.

Also, the nondeterministic alert tests only cover the modules, not the functional variant.

### Suggest a potential alternative/fix

The documentation should be updated to include the functional counterparts, and the tests for these modules should be updated to include the functional variant as well.

cc @svekars @holly1238 @mruberry
Memoizing AOT Autograd Input Conversion Breaks Training with Tied Parameters
Labels: triaged, module: functorch

### 🐛 Describe the bug

AOT Autograd converts inputs to a new set of TensorImpls, either by converting them to fake tensor or by detaching the tensors and then adding back in the gradient `x.detach().requires_grad_(x.requires_grad)`.

When that conversion is memoized, so that two tied parameters are converted to the same FakeTensor/Detached Tensor, models with tied parameters fail. You can replicate this failure:

```
diff --git a/functorch/_src/aot_autograd.py b/functorch/_src/aot_autograd.py
index 6bb4401784..74c2577c73 100644
--- a/functorch/_src/aot_autograd.py
+++ b/functorch/_src/aot_autograd.py
@@ -435,8 +435,14 @@ def create_aot_dispatcher_function(
                 # (resnet50_quantized_qat and mobilenet_v2_quantized_qat)
                 # because they use a "running-mean" style op that requires
                 # resizing the running counter buffers stored on the module.
+                seen = {}
+
                 def make_input(x):
-                    return x.detach().requires_grad_(x.requires_grad)
+                    if id(x) in seen:
+                        return seen[id(x)]
+                    out = x.detach().requires_grad_(x.requires_grad)
+                    seen[id(x)] = out
+                    return out
```

and by running in torchdynamo 
```python benchmarks/huggingface.py --ci -d cuda --float32 --backend=aot_nop --training --only=RobertaForCausalLM```

We would like to memoize the the conversion and reflect shared identities of Parameters so that inplace metadata mutations are reflected on tied parameters. We need to fix the tied parameter gradient issue to do so. 

### Versions

master

cc @zou3519 @Chillee @samdow @soumith
reentrant torch.utils.checkpoint does not work with NamedTuple outputs
Labels: high priority, triage review, oncall: distributed, module: checkpoint

### 🐛 Describe the bug

The reentrant (default) version of torch.utils.checkpoint does not work with NamedTuple outputs, it instead returns back a regular tuple so the outputs cannot be accessed via their original names. For a repro, see:

```
import torch
from torch.utils.checkpoint import checkpoint

from collections import namedtuple

Tup = namedtuple("Tup", "a b c")

tup = Tup(torch.ones(10, requires_grad=True), torch.ones(10, requires_grad=True), torch.ones(10, requires_grad=True))

def foo(tup):
	return Tup(tup.a + tup.b, tup.b, tup.a + tup.b + tup.c)

import pdb ; pdb.set_trace()
out = checkpoint(foo, tup, use_reentrant=False)
```

### Versions

main

cc @ezyang @gchanan @zou3519 @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501
primTorch/nvfuser: have a way to check that refs are added to __all__
Labels: feature, triaged, module: nvfuser, module: primTorch

### 🐛 Describe the bug

In primTorch, with regular PyTorch refs, it's easy to check them based on the `_refs` prefix and look them up in `__all__`. For nvfuser-specific ones, it's not yet clear what to do. There's no `__all__` and the prefix is different: `ops.nvprims`.

See https://github.com/pytorch/pytorch/pull/84792 for more context.

### Versions

master

cc @ezyang @mruberry @ngimel @Lezcano @fdrocha
libtorch create a tensor is very slow, who can tell me why
Labels: module: performance, module: cpp, triaged

I use libtorch create a tensor,but the time consume too long,
why create a tensor will consume such long time

#include <iostream>
#include <torch/torch.h>

int main() {
	auto start = std::chrono::system_clock::now();

	auto ccc = torch::zeros({ 100, 100, 1000 });

	auto end = std::chrono::system_clock::now();
	auto duration = std::chrono::duration_cast<std::chrono::milliseconds>(end - start);
	std::cout << "using time: " << duration.count() << " ms" << std::endl;

	return 0;
}

using time: 23 ms



Thanks

cc @VitalyFedyunin @ngimel @jbschlosser
Segmentation fault in `torch.jit.wait`
Labels: oncall: jit

### 🐛 Describe the bug

Passing `None` to `torch.jit.wait` can cause a Segmentation fault.

## code

```python
import torch

torch.jit.wait(None)
```
## output
```bash
Segmentation fault (core dumped)
```

### Versions

PyTorch version: 1.12.1+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.31

Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.4.0-117-generic-x86_64-with-glibc2.29
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.18.5
[pip3] torch==1.12.1
[conda] Could not collect
Selectively sync internal Meta discussions / posts to dev-discuss.pytorch.org
Labels: triaged

### 🚀 The feature, motivation and pitch

We have an internal developer discussion group at Meta and it would be beneficial to open collaboration to have those posts in PyTorch Dev Discussion (Not Q&A) selectively shared by a bot to dev-discuss.pytorch.org. As a stretch, having dev-discuss.pytorch.org sync internally with a bot post would also be valuable.

### Alternatives

1. Keep conversations and discussion separate as it is right now
2. Continue to actively push Meta internal folks to leverage dev-discuss.pytorch.org, but keep it manual
3. Use Workplace or other systems to match Meta internals

### Additional context

_No response_
Custom autograd functions are not inlined when export mode is ONNX_ATEN_FALLBACK
Labels: module: onnx, triaged

### 🐛 Describe the bug

I am working with a partner, who are using operator_export_type=OperatorExportTypes.ONNX_ATEN_FALLBACK to export a greater number of models without error with the trade off of running inference with the PyTorch aten library included.

They are trying to add support for models that contain custom autograd functions, and have been testing with PyTorch nightly (since this PR went in: https://github.com/pytorch/pytorch/pull/74765).

What we would like to see is:

When operator_export_type=OperatorExportTypes.ONNX_ATEN_FALLBACK and

1. All functionality is supported by ONNX
    then any custom autograd functions are inlined and the graph is exported with all ONNX ops
    
2. When there are aten ops in the custom autograd function   
    then any custom autograd functions are inlined and the graph is exported with the aten ops

What we observe currently:

1. operator_export_type=OperatorExportTypes.ONNX

    If the functionality in the custom autograd function is supported by ONNX ops then the graph is exported to ONNX correctly.

    Export fails if the custom autograd has an aten op with the error message “Could not export operator aten:XXX”

2. operator_export_type=OperatorExportTypes.ONNX_FALLTHROUGH

    The custom autograd functions is exported as a PythonOp node whether or not the functionality of the custom autograd functions is supported by ONNX.

3. operator_export_type=OperatorExportTypes.ONNX_ATEN_FALLBACK

     The export fails with the error “Couldn’t export Python operator <CustomAutogradFunctionName>”

Small sample here demonstrating the above findings: https://github.com/natke/custom_autograd/blob/main/model.py.

### Versions

```
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.1
[pip3] torch==1.13.0.dev20220908
[pip3] torchaudio==0.13.0.dev20220908
[pip3] torchvision==0.14.0.dev20220908
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.4.0           hecd8cb5_637  
[conda] mkl-service               2.4.0            py39h9ed2024_0  
[conda] mkl_fft                   1.3.1            py39h4ab4a9b_0  
[conda] mkl_random                1.2.2            py39hb2f4e1b_0  
[conda] numpy                     1.23.1           py39h2e5f0a9_0  
[conda] numpy-base                1.23.1           py39h3b1a694_0  
[conda] pytorch                   1.13.0.dev20220908         py3.9_0    pytorch-nightly
[conda] torchaudio                0.13.0.dev20220908        py39_cpu    pytorch-nightly
[conda] torchvision               0.14.0.dev20220908        py39_cpu    pytorch-nightly
```
[CheckpointWrapper] Revamp API design
Labels: oncall: distributed, better-engineering, module: fsdp

### 🚀 The feature, motivation and pitch

CheckpointWrapper today is still in a private / prototype stage but nonetheless has gotten quite hacky with several different arguments, some of which don't work with each other, see this comment for details: https://github.com/pytorch/pytorch/pull/84908#discussion_r970933952.

In particular the following is confusing:

1. checkpoint_impl ignored if checkpoint_fn is specified
2. Both of the above are ignored if offload_to_cpu is specified
3. checkpoint_fn_args, checkpoint_fn_kwargs only used when checkpoint_fn is specified,

and adding additional options will likely only exacerbate the issue. We should revamp the design here to make it more user-friendly.

### Alternatives

_No response_

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501
Cuda tensor is zero when passed through multiprocessing queue
Labels: module: multiprocessing, triaged

### 🐛 Describe the bug

I encounter an issue when passing cuda tensors through the multiprocessing queue. The first tensor sent is always 0 when using the following code snippet.

```python
import time
import torch
import torch.multiprocessing as mp


def process(q, N):
    for _ in range(N):
        q.put(torch.tensor(1).to("cuda"))
    while True:
        time.sleep(1)


if __name__ == '__main__':
    q = mp.Queue()
    N = 5
    p = mp.Process(target=process, args=(q, N))
    p.start()
    for _ in range(N):
        print(q.get())

````
When running the above code snippet I get: 
```
tensor(0, device='cuda:0')
tensor(1, device='cuda:0')
tensor(1, device='cuda:0')
tensor(1, device='cuda:0')
tensor(1, device='cuda:0')
```
, which is incorrect. The correct result should be to receive only `tensor(1, device='cuda:0')`.
I am able to reproduce this error consistently on multiple versions of pytorch and cuda.
Sometimes the second tensor sent is also corrupted.

Strangely enough, the error does not occur if you create the tensor directly on the cuda device with `torch.ones(1, device="cuda")`.

### Versions

PyTorch version: 1.13.0.dev20220913+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 Enterprise
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.20.21032501-MSVC_2
Libc version: N/A

Python version: 3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19044-SP0
Is CUDA available: True
CUDA runtime version: 11.7.64
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090
Nvidia driver version: 516.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.13.0.dev20220913+cu117
[pip3] torchaudio==0.13.0.dev20220913+cu117
[pip3] torchvision==0.14.0.dev20220913+cu117
[pip3] torchviz==0.0.2
[conda] Could not collect

cc @VitalyFedyunin
Segmentation fault in `torch.futures.collect_all`
Labels: module: crash, module: error checking, triaged

### 🐛 Describe the bug

In `torch.futures.collect_all` when `futures` got invalid input, it causes a Segmentation fault.

## code

```python
import torch

input = (None,)

torch.futures.collect_all(futures=input)
```

## output
```bash
Segmentation fault (core dumped)
```

### Versions

PyTorch version: 1.12.1+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.31

Python version: 3.8.10 (default, Jun 22 2022, 20:18:18)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.4.0-117-generic-x86_64-with-glibc2.29
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.18.5
[pip3] torch==1.12.1
[conda] Could not collect
Add unit tests for test decorators
Labels: feature, module: tests, triaged

If a test decorator is not working properly, then the test it wraps can become invalid. For instance, in the case of #84874, tests wrapped with `expectedFailureMeta` were never getting run, even though the printout from running these tests falsely indicates that the tests were run. There was nothing obvious to indicate that it wasn't working properly until I accidentally stumbled upon it.

It seems like it would be a good idea to write unit tests for some of the decorators, if they don't already have a test, to make sure they're working as expected and to give us more assurance that our tests are valid. Who knows, maybe there are other decorators with similar problems right now.

cc @mruberry
test_warp_softmax_64bit_indexing_cuda_float16 takes ~147GB of CPU memory and is very slow
Labels: module: memory usage, triaged, module: testing

### 🐛 Describe the bug

test_warp_softmax_64bit_indexing_cuda_float16 takes ~147GB of CPU memory and is very slow

Reproduce:
```
/usr/bin/time -v python test/test_nn.py -v -k test_warp_softmax_64bit_indexing_cuda_float16
```

Output:
```
test_warp_softmax_64bit_indexing_cuda_float16 (__main__.TestNNDeviceTypeCUDA) ... ok

----------------------------------------------------------------------
Ran 1 test in 105.592s

OK
        Command being timed: "python test/test_nn.py -v -k test_warp_softmax_64bit_indexing_cuda_float16"
        User time (seconds): 776.69
        System time (seconds): 2031.00
        Percent of CPU this job got: 2576%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 1:48.96
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 147604400
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 184895836
        Voluntary context switches: 14118
        Involuntary context switches: 2191694
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
```

Considering the test was taking ~84 GB of ram before (https://github.com/pytorch/pytorch/pull/67922#issuecomment-963899593), but it takes 140 GB now, something could have gone wrong. Also, the test name was changed in https://github.com/pytorch/pytorch/pull/84182.

### Versions

torch_commit: https://github.com/pytorch/pytorch/commit/7a9ab5c232f54430704456d18a22f99838489817

V100 32 GB

cc @ptrblck @ngimel @malfet @eqy
DISABLED test_random_seed (__main__.TestDataLoaderUtils)
Labels: module: dataloader, triaged, skipped

Platforms: asan

This test was disabled because it is failing on master ([recent examples](http://torch-ci.com/failure/test_random_seed%2CTestDataLoaderUtils))

Examples: (look for "The action has timed out"):
* https://hud.pytorch.org/pytorch/pytorch/commit/dbd38f63f5731f8403edfdf9d5956ca872453dd3
* https://hud.pytorch.org/pytorch/pytorch/commit/d6733327829fa02295c239ad96a26bef8afa6da4
* https://hud.pytorch.org/pytorch/pytorch/commit/31cc03cc132020244f6985aefdcd1c05babc2e17

cc @SsnL @VitalyFedyunin @ejguan @NivekT
[FSDP] Test optimizer state dict with CPU offload
Labels: triaged, module: fsdp

The current [FSDP optimizer state dict tests](https://github.com/pytorch/pytorch/blob/master/test/distributed/fsdp/test_fsdp_optim_state.py) do not test with CPU offload enabled. This misses bugs like the one fixed in https://github.com/pytorch/pytorch/pull/84708. We should add some tests with CPU offload enabled to exercise the case where optimizer states are on CPU instead of GPU.

cc @zhaojuanmao @mrshenli @rohan-varma
Separate doc and binaries build
Labels: triaged, module: doc infra

### 🚀 The feature, motivation and pitch

At the moment, if we would want to update a stable version of the docs after the release, we'd have to build new binaries and add a new tag. We need to be able to update just the stable versions of  the docs w/o building everything else. 

### Alternatives

_No response_

### Additional context

_No response_

cc @ezyang @zou3519 @holly1238 @malfet
`is_pinned()` support in PrimTorch and FakeTensor.
Labels: triaged, module: primTorch, module: fakeTensor

### 🚀 The feature, motivation and pitch

Trying to add `Tensor.to` in PrimTorch in #84802. ATen implementation supports pin_memory, which is unfortunately not supported by FakeTensor nor PrimTorch.

See comment here: https://github.com/pytorch/pytorch/pull/84802#issuecomment-1244155840

Without `is_pinned()` supported in FakeTensor or PrimTorch, we can't safely implement/fallback `Tensor.to` in Prim. Looking at this comment here (https://github.com/pytorch/pytorch/blob/7a9ab5c232f54430704456d18a22f99838489817/torch/_subclasses/fake_tensor.py#L849-L850) seems like it's something planned for FakeTensor already. Would be nice if we can prioritize it.

### Alternatives

_No response_

### Additional context

_No response_

cc @ezyang @mruberry @ngimel @Lezcano @fdrocha
Functorch functionalization causes increased memory usage
Labels: triaged, module: functionalization, module: functorch

### 🐛 Describe the bug

Functorch functionalization causes more memory usages. It was turned on in PR #84435

See script below. It's a TIMM resnet50 model and was working fine with batch size 128 on a 16 GB V100, with or without functorch fusion.

However, after that PR, if we do `memory_efficient_fusion` on that model, it gets OOM.

```python
import torch
import timm

from functorch.compile import memory_efficient_fusion

x = torch.randn(128, 3, 224, 224, dtype=torch.float, device='cuda')
model = timm.create_model('resnet50')
model = model.cuda()

model = model.train()

# model = memory_efficient_fusion(model)

out = model(x)
out.sum().backward()

torch.cuda.synchronize()
```

### Versions

pytorch https://github.com/pytorch/pytorch/commit/e68df8e4a14ce1fbedf6b20e132b11ec7b151f8a
timm https://github.com/rwightman/pytorch-image-models/commit/fa8c84eede55b36861460cc8ee6ac201c068df4d
torchvision https://github.com/pytorch/vision/commit/84dcf695d64c15f8a0be845ac65901bdde845429
torchdynamo https://github.com/pytorch/torchdynamo/commit/fe3173f7e6c804e6330ac187ea8e4101f45ff9a2

V100 16 GB

cc @bdhirsh @ezyang @soumith @zou3519 @Chillee @samdow @ngimel @ptrblck @csarofeen @kevinstephano
Re-Running PR Sanity Check after Adding `skip-pr-sanity-checks` Label Still Fails
Labels: module: ci, triaged

The PR https://github.com/pytorch/pytorch/pull/83665 exceeds the 2k LOC limit for the PR sanity check. After adding the label `skip-pr-sanity-checks` and re-running the failing check, the check continues to fail when the expected behavior is for it to now pass.

(I have since rebased the PR since one of the preceding PRs had some flaky build failures.)

cc @seemethere @malfet @pytorch/pytorch-dev-infra
torch.utils.checkpoint (with use_reentrant=False) doesn't work with all PyTorch features that set TLS
Labels: module: checkpoint, triaged, module: __torch_dispatch__, tensor subclass

## Issue description

torch.utils.checkpoint (with use_reentrant=False) returns silently incorrect results with some PyTorch features that set TLS. This includes: functorch (see https://github.com/pytorch/functorch/issues/993), torch_dispatch Mode, and possibly also more.

The root of the issue is that:
- checkpoint needs to re-run the function passed to it to compute intermediates
- This function must be re-run with the same global state as the original function was run
- checkpoint hard-codes some the state it needs:(https://github.com/pytorch/pytorch/blob/2698f99dc7a2efe6d60fffa43beb545901a57c9b/torch/utils/checkpoint.py#L398-L408)

## Code example

Repro for torch_dispatch Mode:
```py
import torch
from torch.utils._python_dispatch import TorchDispatchMode
import torch.utils.checkpoint

class NoRandomnessMode(TorchDispatchMode):
    def __torch_dispatch__(self, func, types, args=(), kwargs=None):
        if kwargs is None:
            kwargs = {}
        import pdb; pdb.set_trace()
        if func == torch.ops.aten.rand.default:
            return torch.ones(*args, **kwargs) / 2
        rs = func(*args, **kwargs)
        return rs

def f(x):
  y = torch.rand(x.shape)
  return x.clone() * y

def g(x):
  return torch.utils.checkpoint.checkpoint(f, x, use_reentrant=False)

x = torch.ones([], requires_grad=True)

with NoRandomnessMode():
  y = g(x)

y.backward()

# assertion failure
assert torch.allclose(x.grad, y)
```

## Resolution

The implication for torch_dispatch mode is that checkpointing needs to somehow save the mode stack at the time of the forward pass and then restore it when it re-runs the forward pass.

Robustness: this problem sounds similar to [Aten/ThreadLocalState.h](https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/ThreadLocalState.h) -- in ATen, there is one place that encapsulates all TLS. If we had something like that for Python, then if anyone comes along to add some TLS in the future, checkpoint would automatically work with it.


cc @Chillee @ezyang @albanD @samdow
View consistency for PrimTorch+nvFuser tests
Labels: triaged, module: nvfuser, module: primTorch

### 🚀 The feature, motivation and pitch

`test/test_ops.py:_ref_test_helper` checks whether the output of a reference implementation returns the same `._is_view()` as eager.

The nvFuser execution function always returns a fresh copy currently so `tensor._is_view()` is False. These tests are currently skipped.
How important is view consistency? It's not yet clear whether it's feasible to implement for nvFuser execution and how difficult it is.

### Alternatives

_No response_

### Additional context

Latest master.

cc @ezyang @mruberry @ngimel @Lezcano @fdrocha
Feature Request: deterministic adaptive_avg_pool2d_backward_cuda
Labels: feature, module: cuda, triaged, module: determinism

### 🐛 Describe the bug

def __init__(self, inp, oup, reduction=32):
        super(CoordAtt, self).__init__()
        self.pool_h = nn.AdaptiveAvgPool2d((None, 1))
        self.pool_w = nn.AdaptiveAvgPool2d((1, None))

        mip = max(8, inp // reduction)


### Versions

PyTorch version: 1.12.1+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
CMake version: version 3.22.6
Libc version: glibc-2.26

Python version: 3.7.13 (default, Apr 24 2022, 01:04:09)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic
Is CUDA available: False
CUDA runtime version: 11.1.105
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.0.5
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.6
[pip3] torch==1.12.1+cu113
[pip3] torchaudio==0.12.1+cu113
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.13.1
[pip3] torchvision==0.13.1+cu113
[conda] Could not collect

cc @ngimel @mruberry @kurtamohler