gopikrsmscs/torch-issues huggingface dataset:
JIT return a tensor with different datatype from the tensor w/o gradient and normal function
Labels: oncall: jit

### 🐛 Describe the bug

JIT return a tensor with different datatype from the tensor w/o gradient and normal function

```py
import torch

class M(torch.nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, input):
        res = torch.abs(input, )
        return torch.div(res, torch.tensor(7, dtype=torch.float32, device='cpu'))

fn = M().to('cpu')
input = torch.tensor(11, dtype=torch.bfloat16)

v = fn(input.clone().requires_grad_())
print(v.dtype)

jit_fn = torch.jit.script(fn)
jv1 = jit_fn(input.clone())
print(jv1.dtype)

jv2 = jit_fn(input.clone().requires_grad_())
print(jv2.dtype)
```

```
torch.float32
torch.float32
torch.bfloat16
```

### Versions

```
PyTorch version: 1.13.0.dev20221001
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.9.13 (main, Aug 25 2022, 23:26:10)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.19.5-051905-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.99
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3060
Nvidia driver version: 515.65.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.13.0.dev20221001
[pip3] torchaudio==0.13.0.dev20221001
[pip3] torchvision==0.14.0.dev20221001
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py39h7f8727e_0  
[conda] mkl_fft                   1.3.1            py39hd3c417c_0  
[conda] mkl_random                1.2.2            py39h51133e4_0  
[conda] numpy                     1.23.3                   pypi_0    pypi
[conda] numpy-base                1.23.1           py39ha15fc14_0  
[conda] pytorch                   1.13.0.dev20221001 py3.9_cuda11.7_cudnn8.5.0_0    pytorch-nightly
[conda] pytorch-cuda              11.7                 h67b0de4_0    pytorch-nightly
[conda] pytorch-mutex             1.0                        cuda    pytorch-nightly
[conda] torch                     1.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchaudio                0.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchvision               0.14.0.dev20220923+cu116          pypi_0    pypi
```
`F.affine_grid` crashes on MPS
Labels: triaged, module: mps

### 🐛 Describe the bug

`F.affine_grid` crashes on MPS. Code to reproduce:

```python
import torch
import torch.nn.functional as F

dev_cpu = torch.device('cpu')
dev_mps = torch.device('mps')

device=dev_cpu

laf = torch.tensor([[2, 0, 4.], [0, 2., 5.]], device=device)
grid = F.affine_grid(laf.view(1,2,3), [1, 3, 3, 3], align_corners=False)

print ("cpu grid:", grid)

laf=laf.to(dev_mps)
grid = F.affine_grid(laf.view(1,2,3), [1, 3, 3, 3], align_corners=False)
print ("mps grid:", grid)
```

Output:

```
cpu grid: tensor([[[[2.6667, 3.6667],
          [4.0000, 3.6667],
          [5.3333, 3.6667]],

         [[2.6667, 5.0000],
          [4.0000, 5.0000],
          [5.3333, 5.0000]],

         [[2.6667, 6.3333],
          [4.0000, 6.3333],
          [5.3333, 6.3333]]]])
-:27:11: error: invalid input tensor shapes, indices shape and updates shape must be equal
-:27:11: note: see current operation: %25 = "mps.scatter_along_axis"(%23, %arg1, %24, %1) {mode = 6 : i32} : (tensor<27xf32>, tensor<3xf32>, tensor<9xi32>, tensor<i32>) -> tensor<27xf32>
/AppleInternal/Library/BuildRoots/a0876c02-1788-11ed-b9c4-96898e02b808/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphExecutable.mm:1267: failed assertion `Error: MLIR pass manager failed'
zsh: abort      python fail_affine_grid.py
```

Expected output: don't crash.

### Versions

```
Collecting environment information...
PyTorch version: 1.13.0.dev20221002
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.6 (arm64)
GCC version: Could not collect
Clang version: 14.0.6
CMake version: version 3.22.3
Libc version: N/A

Python version: 3.9.4 | packaged by conda-forge | (default, May 10 2021, 22:10:52)  [Clang 11.1.0 ] (64-bit runtime)
Python platform: macOS-12.6-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.910
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.2
[pip3] pytest-mypy==0.9.0
[pip3] torch==1.13.0.dev20221002
[pip3] torch-dimcheck==0.0.1
[pip3] torchaudio==0.11.0
[pip3] torchvision==0.10.0
[conda] numpy                     1.21.2                   pypi_0    pypi
[conda] torch                     1.13.0.dev20221002          pypi_0    pypi
[conda] torch-dimcheck            0.0.1                    pypi_0    pypi
[conda] torchaudio                0.11.0                   pypi_0    pypi
[conda] torchvision               0.10.0                   pypi_0    pypi
```

cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
[Activation Checkpointing] Investigate pin_memory for CPU offload
Labels: oncall: distributed, module: checkpoint, triaged

### 🚀 The feature, motivation and pitch

@awgu had a good point here: https://github.com/pytorch/pytorch/pull/85459#discussion_r979263528 that we shouldn't assume we have unlimited space in the pinned memory region, right now `save_on_cpu` does pin_memory=True in a hardcoded way, we should investigate performance implications of this and improve our intuition.

### Alternatives

_No response_

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501
Figure out the future of Metal backend given the existence of MPS
Labels: module: build, triaged, module: arm, module: mps

### 🐛 Describe the bug

We have simultaneously a Metal backend and an MPS backend in PyTorch. It's not clear to me we need both of them. In particular, if we want to deploy to iOS, if you can be iOS 9+ MPS should be usable. Well, I don't know if anyone has actually tried this in reality. And there might be deployment considerations (as binary size matters for the mobile target.) We are also still using Metal in prod here at Meta.

Any comments from folks about if we should get rid of Metal for MPS? Or if we don't get rid of it, should there be better interop between the two? E.g., not having a separate Tensor type for MPS versus Metal. The Metal support in PT is very limited, and a few years ago @xta0 was complaining about how views are not done correctly; MPS *does* do views correctly.

cc @malfet @seemethere @kulinseth @albanD @DenisVieriu97 @razarmehr @abhudev  for perspectives on MPS

cc @dreiss @xta0 @SS-JIA  for perspectives on Metal

### Versions

master
torch.remainder and torch.fmod produce wrong results
Labels: module: numerical-stability, triaged, module: correctness (silent)

### 🐛 Describe the bug

For some inputs, torch.remainder and torch.fmod produce wrong results, especially for integer datatype. When converting the int32 input to float32, they can produce correct results.

I suspect this might be caused by type promotion.

Reproduce code:
```
import torch
import numpy as np

input = torch.tensor(1693446850, dtype=torch.int32)
other = torch.tensor([7285], dtype=torch.int16)

# test the apis with int input (wrong results)
r = torch.remainder(input, other)
print(r)
r = torch.fmod(input, other)
print(r)
r = np.fmod(input.numpy(), other.numpy())
print(r)

input = input.to(torch.float32)

# test the apis with float input (correct results)
r = torch.remainder(input, other)
print(r)
r = torch.fmod(input, other)
print(r)
r = np.fmod(input.numpy(), other.numpy())
print(r)
```
Output:
```
tensor([3895], dtype=torch.int16)
tensor([-3390], dtype=torch.int16)
[4890]
tensor([4952.])
tensor([4952.])
[4952.]
```

### Versions

```
Collecting environment information...
PyTorch version: 1.13.0.dev20220923+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.17

Python version: 3.7.13 (default, Mar 29 2022, 02:18:16)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-4.15.0-176-generic-x86_64-with-debian-buster-sid
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.6
[pip3] torch==1.13.0.dev20220923+cpu
[pip3] torchaudio==0.13.0.dev20220923+cpu
[pip3] torchvision==0.14.0.dev20220923+cpu
[conda] numpy                     1.21.6                   pypi_0    pypi
[conda] torch                     1.13.0.dev20220923+cpu          pypi_0    pypi
[conda] torchaudio                0.13.0.dev20220923+cpu          pypi_0    pypi
[conda] torchvision               0.14.0.dev20220923+cpu          pypi_0    pypi
```
partial view/reshaping
Labels: triaged, enhancement, module: python frontend

### 🚀 The feature, motivation and pitch

The current reshaping procedure of tensors like [A x B*C  x D] to [A x B x C x D] requires the passing of sizes of new dimensions.

`.view(A, B, C, D)`
`.view(-1, B, C, D)`

The user needs to find the exact values of ABCD or at least 3 of them.

Is it possible to add dimensions keeping like dimension auto-calculation with -1?

proposed key `0` - like `-1` but for keeping dimension size.

For the example above:

`.view(0, B, -1, 0)`

keep A and D as is. add dim B and calculate dim C from tensor size and given B.


### Alternatives

_No response_

### Additional context

_No response_
Significantly worse MPS performance between torch 1.13.0.dev20220922 and torch 1.13.0.dev20220930
Labels: module: performance, triaged, module: mps

### 🐛 Describe the bug

This is running Stable Diffusion code. A basic Stable Diffusion example used to have somewhere around 1.6it/s on an Apple M1 Max machine with torch 1.13.0.dev20220922. But once I upgrade to the latest torch 1.13.0.dev20220930 version, the same code (with everything else the same) takes around 8s/it (notice that it takes 8s for each iteration as opposed to 1.6 iterations per second).

I have two separate conda environments with the two torch nightly versions and the timing difference is consistent across multiple runs.

### Versions

PyTorch version: 1.13.0.dev20220930
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.6 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.102)
CMake version: version 3.24.1
Libc version: N/A

Python version: 3.9.13 | packaged by conda-forge | (main, May 27 2022, 17:00:33)  [Clang 13.0.1 ] (64-bit runtime)
Python platform: macOS-12.6-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.13.0.dev20220930
[pip3] torchaudio==0.13.0.dev20220930
[pip3] torchvision==0.14.0.dev20220930
[conda] numpy                     1.23.3           py39hcb4b507_0    conda-forge
[conda] pytorch                   1.13.0.dev20220930         py3.9_0    pytorch-nightly
[conda] torchaudio                0.13.0.dev20220930        py39_cpu    pytorch-nightly
[conda] torchvision               0.14.0.dev20220930        py39_cpu    pytorch-nightly

cc @VitalyFedyunin @ngimel @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
Functorch memory_efficient_fusion gives wrong output batch size
Labels: module: docs, triaged, module: functorch

### 🐛 Describe the bug

Run the following script:

```python
import torch

from functorch.compile import memory_efficient_fusion

import timm

x = torch.randn(5, 3, 224, 224).cuda()
y = torch.randn(6, 3, 224, 224).cuda()

net = timm.create_model('resnet50').cuda()
net = memory_efficient_fusion(net)
net.eval()

with torch.jit.fuser('fuser2'):
    xout = net(x)
    yout = net(y)

print(xout.shape, yout.shape)
```

Got:
```
torch.Size([5, 1000]) torch.Size([5, 1000])
```

Expecting:
```
torch.Size([5, 1000]) torch.Size([6, 1000])
```

Note that `yout` has the wrong batch size. Commenting out the memory_efficient_fusion (using eager mode) gives the expected result.


### Versions

Was working fine on 9/28

torch_commit: https://github.com/pytorch/pytorch/commit/614d6f19e3d30cac0d77059e738d1f25d75eb408
torchdynamo_commit: https://github.com/pytorch/torchdynamo/commit/6ead5cae0d1234aa64db06fe230ef56e12ec76fe
torchvision_commit: https://github.com/pytorch/vision/commit/a35be97a6e6725c83032315a8f5e5f6911c9ef41

First saw error on 9/29

torch_commit: https://github.com/pytorch/pytorch/commit/3cfc61b84659cea435411a546eca6a891584247f
torchdynamo_commit: https://github.com/pytorch/torchdynamo/commit/6ead5cae0d1234aa64db06fe230ef56e12ec76fe
torchvision_commit: https://github.com/pytorch/vision/commit/a35be97a6e6725c83032315a8f5e5f6911c9ef41

cc @svekars @carljparker @zou3519 @Chillee @samdow @soumith @ngimel @ptrblck
Quantized version of Sigmoid doesn't have _get_name method
Labels: oncall: quantization, triaged

### 🐛 Describe the bug

Hello, i'm interested in fx quantization of pytorch model. In my project i need to register model by its class name, for example:
I define module class from GraphModule by calling node.target. For quantized version of ConvBnReLU output would be QuantizedConvBnReLU.

Despite quantized Sigmoid my method works for any quantized module because of “Quantized” prefix in module class name.

I checked realization of quantized activations and Sigmoid doesn’t contain method _get_name like any other realization of quantized activation.

### Versions

wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel
Discrepancy in output shape for batch_norm inference mode between CUDA and CPU
Labels: module: nn, module: cuda, triaged, actionable

### 🐛 Describe the bug

For CPU inputs `torch.native_batch_norm` returns 0-sized outputs, but for CUDA inputs it returns a copy of `running_mean` argument and inverse of standard deviation.
What's the reason for this discrepancy and what's the correct output? Can we align the behavior of two backends?

CPU code path:
https://github.com/pytorch/pytorch/blob/8f4edf1e1dc9419f0bab66a67c8f149d7b53fc25/aten/src/ATen/native/Normalization.cpp#L684-L687

CUDA code path:
https://github.com/pytorch/pytorch/blob/fe87ae692f813934d1a74d000fd1e3b546c27ae2/aten/src/ATen/native/cuda/Normalization.cu#L440-L443

### Versions

Latest master.

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are @ngimel
CUDA OOM issue when running tests in CI
Labels: high priority, module: ci, triaged

### 🐛 Describe the bug

There have been reports of OOM issue when running CUDA tests in CI. For example,

* https://github.com/pytorch/pytorch/actions/runs/3146997338/jobs/5116296083

As the first step, we need to gather more examples to see if the same test fails always or if it's a more widespread issue because Linux CUDA test_ops are now run in parallel (https://github.com/pytorch/pytorch/pull/85528, in 3 processes)

### Versions

CUDA 11.6 (and may be 10.2 too)

cc @ezyang @gchanan @zou3519 @seemethere @malfet @pytorch/pytorch-dev-infra @clee2000
Setup ssh sometimes fail
Labels: module: ci, triaged

Example failure: https://github.com/pytorch/pytorch/actions/runs/3154700893/jobs/5132596332

```
Prepare all required actions
Getting action download info
Download action repository 'seemethere/add-github-ssh-key@v1' (SHA:105f7619adc4054f5f1be5f79ebd354d8[2](https://github.com/pytorch/pytorch/actions/runs/3154700893/jobs/5132596332#step:4:2)[3](https://github.com/pytorch/pytorch/actions/runs/3154700893/jobs/5132596332#step:4:3)8[4](https://github.com/pytorch/pytorch/actions/runs/3154700893/jobs/5132596332#step:4:4)638)
Run ./.github/actions/setup-ssh
Run seemethere/add-github-ssh-key@v1
Grabbing public ssh keys from https://github.com/albanD.keys
Error: Server Error

```

cc @seemethere @malfet @pytorch/pytorch-dev-infra
Steam Deck Core Dump
Labels: high priority, module: rocm, triaged

### 🐛 Describe the bug

The following happened on my new Steam Deck.

OK, thre's obviously no driver but I think a core dump is a bit excessive (I can fall back to CPU)

```
Python 3.10.2 (main, Jan 15 2022, 19:56:27) [GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> if torch.cuda.is_available() and torch.version.hip:
...     print('We have ROCm')
... else:
...     print('CPU Only')
...
"hipErrorNoBinaryForGpu: Unable to find code object for all current devices!"
Aborted (core dumped)
(134)(deck@steamdeck ~)$

```

### Versions

pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/rocm5.1.1

cc @ezyang @gchanan @zou3519 @jeffdaily @sunway513 @jithunnair-amd @ROCmSupport @KyleCZH
DISABLED test_aot_autograd_exhaustive_as_strided_scatter_cpu_float32 (__main__.TestEagerFusionOpInfoCPU)
Labels: triaged, module: flaky-tests, skipped, module: functorch

### 🐛 Describe the bug

Platforms: mac, macos

It looks like another side effect of https://github.com/pytorch/pytorch/pull/85794 and https://github.com/pytorch/pytorch/pull/85820 is that test_aot_autograd_exhaustive_as_strided_scatter_cpu_float32 (main.TestEagerFusionOpInfoCPU) is now flaky.  Here are some example:

* https://hud.pytorch.org/pytorch/pytorch/commit/0b93afb112d48bb6d89a1e183a90b403560e84e4
* https://hud.pytorch.org/pytorch/pytorch/commit/29c78266c046c7f83e7d84fc764af47e62ae9542 
* https://hud.pytorch.org/pytorch/pytorch/commit/5deeb09d4e3001adfd3d04139b4a330915069ea7

There is also an occurrence in Windows, but I won't disable the test there yet untill i see more of them there:

* https://hud.pytorch.org/pytorch/pytorch/commit/29c78266c046c7f83e7d84fc764af47e62ae9542 

### Versions

Functorch on Mac

cc @zou3519 @Chillee @samdow @soumith
JIT model could return 'NaN' gradient after the first execution
Labels: oncall: jit

### 🐛 Describe the bug

JIT model could return 'NaN' gradient after the first execution

```py
import torch

class M(torch.nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, input):
        input = torch.nn.functional.relu(input)
        input = torch.cos(input)
        return torch.acos(input, )
input_tensor = torch.tensor([-1., -2.], dtype=torch.float32)

device = 'cuda'

m = M().to(device)
input = input_tensor.to(device)
from torch.autograd.functional import jacobian

input1 = input.clone().requires_grad_()
jac1 = jacobian(m, input1)
print(jac1)

jit_m = torch.jit.script(m)
# jit_m = torch.jit.trace(m, input.clone())
input2 = input.clone().requires_grad_()
input_temp = input.clone().requires_grad_()

jit_m(input_temp) # without this, it will output the same gradient

jac2 = jacobian(jit_m, input2)
print(jac2)
```

```
tensor([[0., 0.],
        [0., 0.]], device='cuda:0')
tensor([[nan, nan],
        [nan, nan]], device='cuda:0')
```

### Versions

```
Collecting environment information...
PyTorch version: 1.13.0.dev20220923+cu116
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.9.13 (main, Aug 25 2022, 23:26:10)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.19.5-051905-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3060
Nvidia driver version: 515.65.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.13.0.dev20220923+cu116
[pip3] torchaudio==0.13.0.dev20220923+cu116
[pip3] torchvision==0.14.0.dev20220923+cu116
[conda] numpy                     1.23.3                   pypi_0    pypi
[conda] torch                     1.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchaudio                0.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchvision               0.14.0.dev20220923+cu116          pypi_0    pypi

```
Print a warning when user specifies a qconfig for some node and the qconfig is not supported by BackendConfig
Labels: oncall: quantization, triaged

### 🐛 Describe the bug

today when user provide a qconfig that doesn't match the BackendConfig, we just ignore the qconfig, it would be better if we can provide a warning message so that user know the qconfig is ignored for the operator

### Versions

master

cc @jianyuh @raghuramank100 @jamesr66a @vkuzo
[primTorch] Need to update data-dependent check policy
Labels: triaged, module: primTorch

### 🐛 Describe the bug

Excerpt from #81128 that checks if the values are within bounds for `nll_loss` reference.
```python
    num_classes = input.shape[1] if input.ndim > 1 else input.shape[0]
    valid_classes_mask = torch.logical_and(
        (flat_target >= 0), (flat_target < num_classes)
    )
    class_check = torch.all(torch.logical_or(ignore_classes_mask, valid_classes_mask))

    # TODO: This check does not work with FakeTensor inputs
    # Explicit cast for class_check to bool; See Issue #78071
    utils.check(
        isinstance(target, FakeTensor) or bool(class_check.item()),
        lambda: "A target class is out-of-bounds and not the ignore index.",
    )
```

### Versions

Pytorch upstream commit: a4bd89b267e81dc2a23ed767e1efc30fcfb7c665

cc @ezyang @mruberry @ngimel @Lezcano @fdrocha
string interning for dispatcher operator names
Labels: triaged, module: dispatch

## The Problem

Sometimes in a boxed fallback we need to branch on whether an OperatorHandle is a specific operator. In https://github.com/pytorch/pytorch/pull/85374, functorch's grad transform needs to handle lift_fresh and alias differently. The current thing we do in that PR is we check the operator handle for the name of the operator and then do a string comparison, but string comparisons are slow and affect our microbenchmarks.

The traditional way to differentiate between operators in the dispatcher is: your subsystem gets a dispatch key and you register an override for the (operator, dispatch key) and each override behaves differently. However, this doesn't apply cleanly to functorch: functorch's grad transform is not a dispatch key, it works through a combination of two functorch-specific dispatch keys (DynamicLayer{Front, Back}Mode) and also re-uses the existing autograd keys (Autograd{Backend}, ADInplaceAndView).

## Proposal 1: Hack this together inside of functorch

- functorch DynamicLayerFront can get a separate registration per kernel (instead of using a boxed fallback)
- the registration per kernel passes an enum around, where the enum gets a different value per ATen operator
- functorch's logic then can branch on the value of the enum

## Proposal 2: Intern operator names

- Give FunctionSchema `interned_name` and `interned_overload_name` fields.
- These probably need to be optional and would be a part of some enum somewhere that we put together at compile time.

## Discussion

I don't really like either solutions. Proposal 1 sounds like a hack, Proposal 2 shouldn't be necessary in the traditional sense of the dispatcher.

Thoughts, @ezyang @samdow @bdhirsh ?
`vector_norm` will trigger "Tracing failed sanity checks" for JIT when ord is boolean tensor
Labels: oncall: jit

### 🐛 Describe the bug

`vector_norm` will trigger "Tracing failed sanity checks" for JIT when ord is boolean tensor

```py
import torch

def f(A):
    ord = torch.tensor(True, dtype=torch.bool)
    return torch.linalg.vector_norm(A, ord=ord, )

A = torch.tensor(1.0, dtype=torch.float64)

torch.jit.trace(f, A)
```

```
torch.jit._trace.TracingCheckError: Tracing failed sanity checks!
encountered an exception while running the trace with test inputs.
```

### Versions

```
Collecting environment information...
PyTorch version: 1.13.0.dev20220923+cu116
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.9.13 (main, Aug 25 2022, 23:26:10)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.19.5-051905-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3060
Nvidia driver version: 515.65.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.13.0.dev20220923+cu116
[pip3] torchaudio==0.13.0.dev20220923+cu116
[pip3] torchvision==0.14.0.dev20220923+cu116
[conda] numpy                     1.23.3                   pypi_0    pypi
[conda] torch                     1.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchaudio                0.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchvision               0.14.0.dev20220923+cu116          pypi_0    pypi
```
JIT fails to trace `sparse.mm` with a strange error
Labels: oncall: jit

### 🐛 Describe the bug

JIT fails to trace `sparse.mm` with a strange error. It says that "RuntimeError: Unsupported value kind: Tensor"

```py
import torch

class M(torch.nn.Module):
    def __init__(self):
        super().__init__()
        mat1_tensor = torch.randint(0, 1, [20, 1], dtype=torch.int32)
        mat1 = mat1_tensor.to_sparse()
        self.mat1 = mat1

    def forward(self, mat2):
        return torch.sparse.mm(self.mat1, mat2, )

mat2_tensor = torch.rand([1, 20], dtype=torch.float32)

f = M()
torch.jit.trace(f, mat2_tensor)
```

```
RuntimeError: Unsupported value kind: Tensor
```

### Versions

```
Collecting environment information...
PyTorch version: 1.13.0.dev20220923+cu116
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.9.13 (main, Aug 25 2022, 23:26:10)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.19.5-051905-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3060
Nvidia driver version: 515.65.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.13.0.dev20220923+cu116
[pip3] torchaudio==0.13.0.dev20220923+cu116
[pip3] torchvision==0.14.0.dev20220923+cu116
[conda] numpy                     1.23.3                   pypi_0    pypi
[conda] torch                     1.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchaudio                0.13.0.dev20220923+cu116          pypi_0    pypi
[conda] torchvision               0.14.0.dev20220923+cu116          pypi_0    pypi
```
nn.CrossEntropyLoss overflow with FP16 and minibatch
Labels: module: nn, module: loss, triaged, module: edge cases

### 🐛 Describe the bug

Using nn.CrossEntropyLoss with FP16 and long sequence is stable. However, introducing minibatch dimension would led to overflow and `CrossEntropyLoss` would output `inf`. 

To reproduce:

```Python
import torch

ce = torch.nn.CrossEntropyLoss().cuda().half()

inp = torch.rand((20, 14749, 1025))
inp = inp.cuda().half()
t = torch.randint(low=0, high=14749, size=[20, 1025]).cuda()

loss = ce(inp, t)
print(loss)

ce = torch.nn.CrossEntropyLoss().cuda()
inp = torch.rand((20, 14749, 1025))
inp = inp.cuda()

loss = ce(inp, t)
print(loss)

ce.half()
inp = inp.cuda().half()
inp = inp.transpose(1,2)
inp = inp.flatten(start_dim=0, end_dim=1)
t = t.flatten(start_dim=0, end_dim=1)

loss = ce(inp, t)
print(loss)
```

The first loss would be `inf`. Both the second and the third would be correct.

### Versions

I tested on 1.8.2 and 1.12.1, both are the same.

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
Replace same with TestCase assertEqual
Labels: triaged, oncall: pt2

In TorchDynamo tests, we use `same` function to check for accuracy. But for many cases, we can rely on TestCase assertEqual which has pre-defined atol/rtol for different dtypes. We should migrate the tests to `assertEqual`  as much as possible. Thanks to @SherlockNoMad for pointing this out.

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @williamwen42 if you are interested.
[ONNX] Conversion failed when using dict as input to a scripted module
Labels: module: onnx, triaged, onnx-triaged

### 🐛 Describe the bug

```python
import torch
from typing import Dict

class Module(torch.nn.Module):
    def forward(
        self, x: Dict[str, torch.Tensor]
    ) -> Dict[str, torch.Tensor]:
        return x

model = torch.jit.script(Module())
x = {"input": torch.zeros(1)}
torch.onnx.export(model, (x, {}), "out", opset_version=9)  # ERR

```
```
Traceback (most recent call last):
  File "test.py", line 14, in <module>
    torch.onnx.export(model, (x, {}), "out", opset_version=9)  # ERR
  File "/opt/homebrew/lib/python3.9/site-packages/torch/onnx/__init__.py", line 350, in export
    return utils.export(
  File "/opt/homebrew/lib/python3.9/site-packages/torch/onnx/utils.py", line 163, in export
    _export(
  File "/opt/homebrew/lib/python3.9/site-packages/torch/onnx/utils.py", line 1074, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/opt/homebrew/lib/python3.9/site-packages/torch/onnx/utils.py", line 747, in _model_to_graph
    example_outputs = _get_example_outputs(model, args)
  File "/opt/homebrew/lib/python3.9/site-packages/torch/onnx/utils.py", line 630, in _get_example_outputs
    example_outputs = model(*input_args, **input_kwargs)
  File "/opt/homebrew/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
RuntimeError: forward() is missing value for argument 'x'. Declaration: forward(__torch__.Module self, Dict(str, Tensor) x) -> (Dict(str, Tensor))
```
Using dict as input raises runtime error when exporting scripted module to ONNX.

It seems the inputs are not passed to the model as they should, since in
> File .../torch/onnx/utils.py, line 630, in _get_example_outputs
    example_outputs = model(*input_args, **input_kwargs)

the variables ```input_args``` and ```input_kwargs``` are
```
input_args=()
input_kwargs={'input': tensor([0.])}
```

### Versions

PyTorch version: 1.12.0
Is debug build: False
OS: macOS 12.2.1 (arm64)
Python version: 3.9.13 (main, May 24 2022, 21:13:51)  [Clang 13.1.6 (clang-1316.0.21.2)] (64-bit runtime)
Minifier dumps checkpoints which don't actually reproduce the error
Labels: triaged, bug, oncall: pt2, module: minifier

Steps to reproduce:
1. Checkout 07d794094bd4eaf1ad946429044f1c742f278d6b of pytorch (at time of writing, this is tip of symbolic-shapes branch)
2. Checkout e50d6695990fda7a0dfde460c6c6a2f93e1c8d1d of torchdynamo (at time of writing, this is tip of accuracy-minifier branch)
3. Intentionally introduce a failure in pytorch

```
diff --git a/torch/_decomp/__init__.py b/torch/_decomp/__init__.py
index 9512b0f38a..7f3d29290e 100644
--- a/torch/_decomp/__init__.py
+++ b/torch/_decomp/__init__.py
@@ -100,7 +100,6 @@ def register_decomposition(aten_op, registry=None, *, disable_meta: bool = False
                 if op_overload in registry:
                     raise RuntimeError(f"duplicate registrations for {op_overload}")
                 registry[op_overload] = fn
-                op_overload.py_impl(torch._C.DispatchKey.Meta)(fn)
                 # TODO: factor this logic into OpOverload or Library API
                 name = op_overload._schema.name
                 if op_overload._schema.overload_name:
```

4. Trigger the error and try to minify it `(cd ../torchdynamo && TORCHDYNAMO_REPRO_AFTER=dynamo TORCH_SHOW_CPP_STACKTRACES=1 AOT_FX_GRAPHS_JOINT=1 TORCHDYNAMO_DYNAMIC_SHAPES=1 AOT_DYNAMIC_SHAPES=1 time python benchmarks/torchbench.py --only BERT_pytorch --accuracy --backend aot_eager --training)`

This produces this checkpoint /tmp/minifier_ezyang/checkpoints/2.py

```
import torch
import torchdynamo
from torch import tensor, device
import torch.fx as fx
from torchdynamo.testing import rand_strided
from math import inf
from torchdynamo.debug_utils import run_fwd_maybe_bwd


from torch.nn import *
class Repro(torch.nn.Module):
    def __init__(self):
        super().__init__()

    
    
    def forward(self, embedding):
        return (embedding,)
        

mod = Repro().cuda()
opt_mod = torchdynamo.optimize("aot_eager")(mod)


args = [((2, 128, 768), (98304, 768, 1), torch.float32, 'cuda', True)]
args = [rand_strided(sh, st, dt, dev).requires_grad_(rg) for (sh, st, dt, dev, rg) in args]


with torch.cuda.amp.autocast(enabled=False):
    ref = run_fwd_maybe_bwd(mod, args)
    res = run_fwd_maybe_bwd(opt_mod, args)
```

However this script runs without reproducing the error. It looks like the final repro.py DOES reproduce the error, so I am confused why there's a checkpoint that doesn't do it.

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh @anijain2305
[Quant] Remove or clarify the meaning of Nones in QConfig/BackendConfig
Labels: oncall: quantization, low priority, triaged

There are many instances of potential `None`s throughout QConfig/BackendConfig, e.g.

```
dtype_with_constraints.dtype
dtype_config.is_dynamic
qconfig.activation
qconfig.weight
get_target_activation_dtype_for_node(...)  # in fx/prepare.py
```

We can probably remove the `Optional` around some of these, such as `dtype_config.is_dynamic`. For others, we should clearly document what `None` means.

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel
[ONNX] torch/onnx is using rank to differentiate between ScalarType and TensorType
Labels: module: onnx, triaged, onnx-triaged

### 🐛 Describe the bug

ONNX is using condition of following:
https://github.com/pytorch/pytorch/blob/6a04df3ac85714b33dcb2af20d0eeea96d131d75/torch/csrc/jit/passes/onnx/scalar_type_analysis.cpp#L209
to differentiate between ScalarType and TensorType, which is revealed as bug by the following case:

```python
import torch
from io import BytesIO
import onnx


class MulLeakyReLU(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.m = torch.nn.LeakyReLU(negative_slope=0.1)

    def forward(self, x):
        return x * self.m(x)


f = BytesIO()
torch.onnx.export(
    MulLeakyReLU(),
    torch.randn((), dtype=torch.float64),
    f,
    verbose=True,
    opset_version=14,
)
onnx_model = onnx.load_from_string(f.getvalue())
onnx.checker.check_model(onnx_model, full_check=True)
assert MulLeakyReLU()(torch.randn((), dtype=torch.float64)).dtype == torch.float64
onnx_output_type = onnx_model.graph.output[0].type.tensor_type.elem_type
assert (
    onnx_output_type == onnx.TensorProto.DataType.DOUBLE
), f"Expected output to be double but the converted ONNX model outputs is {onnx.TensorProto.DataType.Name(onnx_output_type)}"

"""
Exported graph: graph(%input : Double(requires_grad=0, device=cpu)):
  %/m/LeakyRelu_output_0 : Double(requires_grad=0, device=cpu) = onnx::LeakyRelu[alpha=0.10000000000000001, onnx_name="/m/LeakyRelu"](%input), scope: __main__.MulLeakyReLU::/torch.nn.modules.activation.LeakyReLU::m
  %/Cast_output_0 : Float(requires_grad=0, device=cpu) = onnx::Cast[to=1, onnx_name="/Cast"](%input), scope: __main__.MulLeakyReLU:: # test.py:14:0
  %/Cast_1_output_0 : Float(requires_grad=0, device=cpu) = onnx::Cast[to=1, onnx_name="/Cast_1"](%/m/LeakyRelu_output_0), scope: __main__.MulLeakyReLU:: # test.py:14:0
  %5 : Float(requires_grad=0, device=cpu) = onnx::Mul[onnx_name="/Mul"](%/Cast_output_0, %/Cast_1_output_0), scope: __main__.MulLeakyReLU:: # test.py:14:0
  return (%5)

Traceback (most recent call last):
  File "test.py", line 29, in <module>
    assert (
AssertionError: Expected output to be double but the converted ONNX model outputs is FLOAT
"""
```

NOTE: the tensor with no rank is recognized as scalar.

Actionable iitems:
1. Explore in tracing to see if we can get scalartype/tensortype information from upstream.
2. SymFloat might be a great help.

### Versions

Nightly
[functorch] CUDA Graph failure with AOTAutograd
Labels: triaged, module: cuda graphs, module: functorch

### 🐛 Describe the bug

We are having issues with one of the latest commits on functorch. Namely [PR 85301](https://github.com/pytorch/pytorch/pull/85301) is breaking our TIMM benchmark with CUDA Graphs:

Before PyTorch commit `5f623f5c4c759cada9f0dc3866b63c906178dbc1` it was working fine 

```
Benchmarking in float32 precision. NCHW layout. torchscript disabled
Model adv_inception_v3 created, param count: 23834568
Running train benchmark on adv_inception_v3 for 40 steps w/ input size (3, 224, 224) and batch size 32.
/home/aaitzhan/git/pytorch-master/torch/jit/_check.py:181: UserWarning: The TorchScript type system doesn't support instance-level annotations on empty non-base types in `__init__`. Instead, either 1) use a type annotation in the class body, or 2) wrap the type in `torch.jit.Attribute`.
  warnings.warn("The TorchScript type system doesn't support "
Train [8/40]. 199.36 samples/sec. 160.512 ms/step.
Train [16/40]. 196.29 samples/sec. 163.025 ms/step.
Train [24/40]. 196.33 samples/sec. 162.992 ms/step.
Train [32/40]. 196.01 samples/sec. 163.257 ms/step.
Train [40/40]. 196.39 samples/sec. 162.943 ms/step.
Train benchmark of adv_inception_v3 done. 196.38 samples/sec, 162.94 ms/sample
--result
{
    "model": "adv_inception_v3",
    "train_samples_per_sec": 196.38,
    "train_step_time": 162.943,
    "train_batch_size": 32,
    "train_img_size": 224,
    "param_count": 23.83
}
```

However on PyTorch commit `5f623f5c4c759cada9f0dc3866b63c906178dbc1` and newer it is failing:
```
Benchmarking in float32 precision. NCHW layout. torchscript disabled
Model adv_inception_v3 created, param count: 23834568
Running train benchmark on adv_inception_v3 for 40 steps w/ input size (3, 224, 224) and batch size 32.
/home/aaitzhan/git/pytorch-master/torch/jit/_check.py:181: UserWarning: The TorchScript type system doesn't support instance-level annotations on empty non-base types in `__init__`. Instead, either 1) use a type annotation in the class body, or 2) wrap the type in `torch.jit.Attribute`.
  warnings.warn("The TorchScript type system doesn't support "
ERROR: "CUDA error: operation failed due to a previous error during capture
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1." while running benchmark.
```

Steps to reproduce:
```
git clone --recursive https://github.com/xwang233/pytorch-image-models.git
cd pytorch-image-models/
git checkout 094755c9ed1f92575ed74bc29d09fba11d96a3bc
TIMM_BENCHMARK_ENABLE_CUDAGRAPH=1 python -u ./benchmark.py --bench train --model adv_inception_v3 --img-size 224 -b 32 --fuser nvfuser --aot-autograd
```

CC @ezyang


### Versions

PyTorch upstream.

cc @mcarilli @ezyang @zou3519 @Chillee @samdow @soumith
[functorch] conv.{1, 2, 3}d should raise errors
Labels: triaged, module: functorch

### 🐛 Describe the bug

We discovered it in #85202.

`conv.{1, 2, 3}d` should raise *RuntimeError* when the bias shape does not equal the number of output channels. Instead, in functorch, it throws the *ValueError*.

#### Current behavior
```python
In [3]: input = torch.rand((1, 1, 4, 4))

In [4]: weight = torch.rand((1, 1, 3, 2))

In [5]: bias = torch.rand((2,))

In [6]: vmap(torch.nn.functional.conv2d)(input, weight, bias)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [6], in <cell line: 1>()
----> 1 vmap(torch.nn.functional.conv2d)(input, weight, bias)

File ~/Documents/pytorch/functorch/_src/vmap.py:348, in vmap.<locals>.wrapped(*args, **kwargs)
    345 @functools.wraps(func)
    346 def wrapped(*args, **kwargs):
    347     _check_out_dims_is_int_or_int_pytree(out_dims, func)
--> 348     batch_size, flat_in_dims, flat_args, args_spec = _process_batched_inputs(in_dims, args, func)
    349     return _flat_vmap(
    350         func, batch_size, flat_in_dims, flat_args, args_spec, out_dims, randomness, **kwargs
    351     )

File ~/Documents/pytorch/functorch/_src/vmap.py:102, in _process_batched_inputs(in_dims, args, func)
     99     if in_dim is not None and in_dim < 0:
    100         flat_in_dims[i] = in_dim % arg.dim()
--> 102 return _validate_and_get_batch_size(flat_in_dims, flat_args), flat_in_dims, flat_args, args_spec

File ~/Documents/pytorch/functorch/_src/vmap.py:35, in _validate_and_get_batch_size(flat_in_dims, flat_args)
     33     raise ValueError('vmap: Expected at least one Tensor to vmap over')
     34 if batch_sizes and any(size != batch_sizes[0] for size in batch_sizes):
---> 35     raise ValueError(
     36         f'vmap: Expected all tensors to have the same size in the mapped '
     37         f'dimension, got sizes {batch_sizes} for the mapped dimension')
     38 return batch_sizes[0]

ValueError: vmap: Expected all tensors to have the same size in the mapped dimension, got sizes [1, 1, 2] for the mapped dimension
```

#### Expected
```python
In [7]: torch.nn.functional.conv2d(input, weight, bias)
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Input In [7], in <cell line: 1>()
----> 1 torch.nn.functional.conv2d(input, weight, bias)

RuntimeError: Given weight of size [1, 1, 3, 2], expected bias to be 1-dimensional with 1 elements, but got bias of size [2] instead
```

The CI test raises *AssertionError* stating `Exception not raised`. 

Link: https://github.com/pytorch/pytorch/actions/runs/3093401271/jobs/5005991218. Currently, these test cases are skipped in the `test_vmap` file.

### Versions

`1.13.0a0+git83c73f7`

cc @zou3519 @Chillee @samdow @soumith
CUDA allocator feature requests
Labels: module: cuda, triaged, module: CUDACachingAllocator

### 🚀 The feature, motivation and pitch

I'm working on torch for R, a project that exposes similar functionality as PyTorch for the R language. 

Tensors are exposed to R as very thin wrappers around LibTorch tensors (basically just pointers to the actual LibTorch tensor),   thus R is not aware at all of memory used by torch Tensors. 

However, R is a garbage collected language and its GC is very lazy. It's only called when R really needs another chunk of memory from the OS. This causes tensors to keep dangling on the session using the valuable GPU memory.  To solve this problem we've been using the `FreeMemoryCallbacks` functionality that will call the R GC whenever getting a free block fails: 

https://github.com/pytorch/pytorch/blob/89896b8778b76685c4fc40d1f8ce337d36105c02/c10/cuda/CUDACachingAllocator.cpp#L607-L611

In theory this solves the problem, but it adds a lot of overhead (calling a full garbage collection is very slow) and the callbacks are called almost for every CUDA allocation if the session still didn't 'reserve' a good chunk of memory. 

Like Python's garbage collector, R's is generational garbage collector, so you can choose to do faster collections. But with the `FreeMemoryCallbacks` API we don't have enough information to decide whether we should do a full collection or a faster one (we end up needing to do full collections to support peak memory usage) because

1. The callbacks don't know how much memory needs to be allocated, nor in which device.
2. Either all callbacks are called or no callback is called - as you can see the `trigger_free_memory_callbacks` implementation.
https://github.com/pytorch/pytorch/blob/89896b8778b76685c4fc40d1f8ce337d36105c02/c10/cuda/CUDACachingAllocator.cpp#L1285-L1292



### Alternatives

I'm proposing two alternatives, but open to other suggestions.

1. Passing `device` and `size`  (or possibly `AllocParams`) to the callback `Execute` method defined in:
https://github.com/pytorch/pytorch/blob/89896b8778b76685c4fc40d1f8ce337d36105c02/c10/cuda/CUDACachingAllocator.h#L16-L20 This way we could query for how much memory the faster collection released and if necessary run a full collection.

2. Introduce some notion of priority to the callbacks and call them in order until it was possible to release enough memory. For example, modifying `trigger_free_memory_callback` to something like:
      ```c++
      bool trigger_free_memory_callbacks(AllocParams& p) {
        bool freed_memory = false;
        for (const auto& name : FreeCudaMemoryCallbacksRegistry()->Keys()) {
          if (FreeCudaMemoryCallbacksRegistry()->Create(name)->Execute() && get_free_block(p)) {
            return true;
          }
        }
        return false;
      }
      ``` 

Thank you very much for taking a look at this! 

### Additional context

_No response_

cc @ngimel
How to install pytorch with cuda 11.7 in anaconda envirment?
Labels: triaged

### 📚 The doc issue
![image](https://user-images.githubusercontent.com/32769358/192286577-2ed4b09a-b062-4f21-a6ae-eadab21b5a3e.png)

![image](https://user-images.githubusercontent.com/32769358/192285880-7ffcbaa7-ad35-418c-a9ca-c7c7e9b25fc4.png)
could not find the version of cuda 11.7 when use conda or pip 

### Suggest a potential alternative/fix

add cuda 11.7 in conda