gopikrsmscs/torch-issues huggingface dataset:
backward not available for index and mask 
Labels: needs reproduction, module: autograd, triaged, module: sorting and selection

### 🐛 Describe the bug

When use index or mask for tensor indices, the forward can run normally, but the backward is not available. After change it to _index_select_ or _masked_select_, the backward can run normally. If there any bug for index graph?

```
## pred（float）: B x M x D, mask(bool): B x M, xs_layer(float): B x M

# not work:
pred_layer_reshape = pred.reshape(-1,D)[mask.reshape(-1)]
gt_box3d = gt.reshape(-1, D)[mask.reshape(-1)]
# work:
pred_layer_reshape = torch.index_select(pred.reshape(-1, D), 0, mask.reshape(-1).nonzero().squeeze())
gt_box3d = torch.index_select(gt.reshape(-1, D), 0, mask.reshape(-1).nonzero().squeeze())

# not work:
xs_layer_reshape = xs_layer.reshape(-1)[mask.reshape(-1)].unsqueeze(-1)
ys_layer_reshape = ys_layer.reshape(-1)[mask.reshape(-1)].unsqueeze(-1)
# work:
xs_layer_reshape = torch.masked_select(xs_layer.reshape(-1), mask.reshape(-1)).unsqueeze(-1)
ys_layer_reshape = torch.masked_select(ys_layer.reshape(-1), mask.reshape(-1)).unsqueeze(-1)

```
RuntimeError: merge_sort: failed to synchronize: cudaErrorIllegalAddress: an illegal memory access was encountered

```

### Versions

torch                     1.8.2+cu111

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
iOS TestApp from mobile performance recipes tutorial doesn't build on macOS
Labels: oncall: mobile, module: ios

### 📚 The doc issue

I'm trying the following tutorial: https://pytorch.org/tutorials/recipes/mobile_perf.html#android-benchmarking-setup

The model preparation part works fine. When I'm trying to build TestApp in Xcode, I get 11 `Target Integrity` errors, yielding i.e.
`/Users/derrkater/repo/pytorch/ios/TestApp/TestApp.xcodeproj Building for iOS Simulator, but the linked library 'libcpuinfo.a' was built for macOS.`

My environment is:
Macbook 12", MacOS Monerey 12.4
Python 3.9.12
Xcode 13.4.1
iPhone 13 Pro simulator
commit: ea6fa8dc95

Possibly importantly, I have modified `build_ios.sh` script with a following line: `CMAKE_ARGS+=("-DCMAKE_IOS_SDK_ROOT=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS10.3.sdk")`

How can I proceed to build the test app correctly?

### Suggest a potential alternative/fix

_No response_
RuntimeError: "reflection_pad2d" not implemented for 'Half' in autocast enabled region
Labels: triaged, module: amp (automated mixed precision)

### 🐛 Describe the bug

I'm trying to convert a model to jit fp16  as described here https://pytorch.org/docs/stable/amp.html#torch.autocast

```
    image = torch.rand(1, 3, 120, 120, dtype=torch.float16)
    mask = torch.rand(1, 1, 120, 120, dtype=torch.float16)

    torch._C._jit_set_autocast_mode(False)

    with torch.cpu.amp.autocast(cache_enabled=False):
        traced_model = torch.jit.trace(jit_model_wrapper, (image, mask), strict=False).to(device)
```

and i get this error 
  File "/home/venv/lib/python3.8/site-packages/torch/nn/modules/padding.py", line 174, in forward
    return F.pad(input, self.padding, 'reflect')
RuntimeError: "reflection_pad2d" not implemented for 'Half'

### Versions

torch==1.12.0

cc @mcarilli @ptrblck
model.to(device) takes time forever on A40-8Q, NVIDIA. cuda11.1, torch1.9.1.
Labels: module: cuda, triaged

### 🐛 Describe the bug

model.to(device) takes time forever on A40-8Q, NVIDIA. cuda11.1, torch1.9.1.

### Versions

py3.6.9,
CUDA available:True,
GPU0: nvidia a40-8q,
nvcc: Build cuda_11.1.TC455_06.29069683_0,
gcc: ubuntu7.5.0,
Pytorch: 1.9.1+cu111,



cc @ngimel
Provide error handling for ops that don't yet support Dynamic Shape
Labels: triaged, lazy

### 🚀 The feature

PyTorch to integrate a default error message in the LTC codegen to inform the user when an op misses dynamic shape support. Perhaps this goes hand-in-hand with a whitelist of ops that do support Dynamic Shape in LTC?

@wconstab @Krovatkin I am filing this ticket per our offline conversation the other week.

CC @JackCaoG @wonjoolee95
`torch.sum` promotes integral tensors to `int64`.
Labels: module: docs, triaged, module: type promotion, actionable, module: reductions

### 🐛 Describe the bug

Repro:

```python
In [1]: import torch
   ...: dtypes = [torch.bool, torch.int8, torch.int32, torch.bfloat16, torch.float32, torch.float64]
   ...: for dtype in dtypes:
   ...:     a = torch.tensor([], dtype=dtype)
   ...:     a_sum = a.sum()
   ...:     if a.dtype != a_sum.dtype:
   ...:         print(f"t.dtype != t.sum().dtype, got {a.dtype} != {a_sum.dtype}")
   ...: 
t.dtype != t.sum().dtype, got torch.bool != torch.int64
t.dtype != t.sum().dtype, got torch.int8 != torch.int64
t.dtype != t.sum().dtype, got torch.int32 != torch.int64
```

This is the cause of https://github.com/pytorch/pytorch/issues/82150.

### Versions

Current master.

cc @svekars @carljparker @nairbv @mruberry @holly1238
[Checkpoint] Support multiple unpack in saved tensor hooks
Labels: module: checkpoint, triaged

### 🐛 Describe the bug

Currently, if the `storage` of a particular tensor is accessed twice without recomputation being triggered in between, we raise the following error: https://github.com/pytorch/pytorch/blob/master/torch/utils/checkpoint.py#L391. However, there is a use case which requires this support.

Listing out a few things that we've currently tried:

1. Using python WeakValueDictionary for `storage` instead of a regular dict, to avoid keeping references alive to activation tensors. This turns out to not work because in some cases, the reference in the dict that needs to be returned is the only reference keeping the activation tensor alive, which is quite confusing. 
2. Clearing storage and re-running the forward pass when we're about to access a tensor which has already been returned by `unpack()`. However, this is a hacky fix and results in a performance slowdown due to re-running the forward pass potentially several times.

### Versions

main
set_grad_enabled not respected when running on a web server
Labels: module: dependency bug, module: autograd, triaged, actionable

### 🐛 Describe the bug

The `torch.set_grad_enabled` flag is not respected when torch is being run as part of a web server (flask/fastapi).

Repro

```python
import torch
from fastapi import FastAPI
import uvicorn
import requests

torch.set_grad_enabled(False)


def grad_enabled():
    x = torch.tensor([1.], requires_grad=True)
    y = x * 2
    return y.requires_grad


app = FastAPI()

@app.get("/")
def predict():
    return grad_enabled()



if __name__ == "__main__":
    assert not grad_enabled()
    uvicorn.run(app, host="0.0.0.0", port=8000)
```

Run `python <filename>`. And go to localhost:8000, you should see `true` even though `set_grad_enabled` is set to `False`. `set_grad_enabled` does work outside the context of a web server though, as shown by the assert statement.


Is this a bug or user error? 

Seeing it in such a condensed code block, I can try to convince myself that this is not how `set_grad_enabled` is supposed to work but I'm not exactly sure why that would be the case.

Certainly this can bite someone else in the butt like it did me and lead to very mysterious CUDA out of memory errors so I think there will be value in helping users by either preventing this from happening or educating users that this can happen.

### Versions

```
Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: macOS 12.2.1 (arm64)
GCC version: Could not collect
Clang version: 13.0.0 (clang-1300.0.27.3)
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.12 (main, Apr  5 2022, 01:52:34)  [Clang 12.0.0 ] (64-bit runtime)
Python platform: macOS-12.2.1-arm64-arm-64bit
Is CUDA available: N/A
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] numpy==1.22.4
[conda] numpy                     1.22.4                   pypi_0    pypi
```

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
External libraries cannot have a requirements.txt that needs to install a cpp_extension
Labels: module: cpp-extensions, triaged

### 🐛 Describe the bug

If an external library (e.g. pytorch/tutorials) has a requirements.txt with the following:
```
torch
git+https://github.com/pytorch/functorch.git@v0.2.0
```
then `pip install -r requirements.txt` errors out with "module torch not found". Here's why:
1. pip reads requirements.txt
1. pip tries to determine each package's dependencies before installing anything
1. pip tries to read functorch's dependencies by running its setup.py
1. functorch's setup.py imports torch to get the cpp_extension
1. torch does not exist, so python throws an error

This was discovered in https://github.com/pytorch/tutorials/pull/1968

## Pitch

People might need to pin libraries like this (I'm surprised this is the first time we've run into it). So it would be cool if we could offer a way for pip to read the torch dependency in a c++ extension library (e.g. functorch) without actually importing torch.

I am not sure if this is actually possible, since we need to pass setup.py a BuildExtension object somehow

### Versions

main

cc @malfet @zou3519
Move functorch tests to under test/
Labels: module: tests, triaged

## Issue description

The functorch tests are currently under `functorch/test/*`. PyTorch's test infra assumes all tests exist under `test/*`. Here are a couple of problems we ran into:

- The test names in test-reports are awkward (see attached image)
- Smart test sharding may not work (because the test names are awkward)
- flaky test reporting may not work (if they depend on the test-reports file)
- Linters for tests don't work

![image](https://user-images.githubusercontent.com/5652049/180800211-124671b6-28c2-4169-b452-4e87e6f99941.png)

## The fix

We should just move functorch tests to under `test/*`. We can do this once functorch fully commits to being a part of PyTorch and we turn off the sync to https://github.com/pytorch/functorch. This should happen in 1-2 months.

cc @mruberry @janeyx99
[feature request] Discover actually loaded shared libraries at runtime
Labels: module: build, triaged, enhancement

### 🚀 The feature, motivation and pitch

Sometimes it's useful for debugging linking / library loading (e.g. for  related: https://github.com/pytorch/pytorch/issues/78489) to be able to find actually loaded shared dependency library full paths (libcudnn.so, libcublas.so and others) at runtime. In Unix systems, it often can be done by scanning via `/proc/$PID/maps`. But maybe some mechanism also exists in Mac/Windows.

So, having a simple API method would be nice for this (and then it could be reused in collect_env and in error messages complaining about bad library versions being loaded)

Also, seems one can retrieve some library paths via  `torch.utils.cpp_extension(fromlist=[None]).library_paths()`

### Alternatives

_No response_

### Additional context

_No response_

cc @malfet @seemethere
torch.concat type hints fail for keyword argument
Labels: module: typing, triaged

### 🐛 Describe the bug

Using a keyword version of passing the axis to `torch.concat` causes type checkers (mypy, pyright) to reject working code:

```python 
import torch

a = (torch.randn([42]), torch.randn([42]))
torch.concat(a, 0)          # works fine!
torch.concat(a, dim=0)      # works fine!
torch.concat(a, axis=0)     # error: No overload variant matches argument types "Tuple[Tensor, Tensor]", "int" 
```

### Versions

PyTorch version: 1.12.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 11.6.6 (x86_64)
GCC version: Could not collect
Clang version: 13.0.0 (clang-1300.0.29.30)
CMake version: Could not collect
Libc version: N/A

Python version: 3.8.5 (default, Sep  4 2020, 02:22:02)  [Clang 10.0.0 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.961
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.22.4
[pip3] torch==1.12.0
[conda] blas                      1.0                         mkl    defaults
[conda] cpuonly                   1.0                           0    pytorch
[conda] mkl                       2019.4                      233    defaults
[conda] mkl-service               2.3.0            py38hfbe908c_0    defaults
[conda] mkl_fft                   1.2.0            py38hc64f4ea_0    defaults
[conda] mkl_random                1.1.1            py38h959d312_0    defaults
[conda] numpy                     1.22.4                   pypi_0    pypi
[conda] torch                     1.12.0                   pypi_0    pypi


cc @ezyang @malfet @rgommers @xuzhao9 @gramster
linear.matrix_power is not composite compliant
Labels: triaged, module: linear algebra

### 🐛 Describe the bug

You have to patch in https://github.com/pytorch/pytorch/pull/81984 to see the error. It's kind of annoying to guess and check what the actual composite compliance error is.

Repro command

```
python test/test_ops.py -k test_operator_linalg_matrix_power
```

cc @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano @zou3519 

### Versions

master
Untangle TorchScript prim ops in aten namespace
Labels: oncall: jit

### 🐛 Describe the bug

Did you know aten::size.default is an op? It's not defined anywhere in native_functions.yaml. In fact, it gets indirectly registered as part of TorchScript prim registrations (despite not actually living in the prim namespace, which is where most of these are supposed to go). These ended up being relied upon by non-TS users for customizable sizes/strides. Let's properly put these in native_functions.yaml so they are handled the same way as other ops.

### Versions

master
Could be clearer that Cross Entropy takes logits as input
Labels: module: docs, module: nn, module: loss, triaged, actionable

### 📚 The doc issue

Hi, I felt that it could be made a bit more clear that the cross entropy function is taking logits in as input. Right now, the documentation is correct but I felt that it could be a bit more clear. It says that the input is required to take "raw, unnormalized scores for each class", but for me when reading that, I was a bit uncertain about what that meant. Then I scrolled down a bit more and saw that (target) would take in probabilities as input and incorrectly started to believe that cross entropy takes probabilities as input. Especially since that is the normal mathematical definition of cross entropy, I think it would be nice to be more clear; a lot of people will assume that it takes probabilities and get confused; I noticed that this question has a lot of upvotes 
https://stackoverflow.com/questions/49390842/cross-entropy-in-pytorch  on stackoverflow

### Suggest a potential alternative/fix

Instead of 
`The input is expected to contain raw, unnormalized scores for each class.`
it could say
`The input is expected to contain the logits for each class (which do not need to be positive or sum to 1, in general). This class will compute a softmax followed by cross entropy to compute the final loss.`

Of course, if one reads the documentation carefully enough (e.g. the formulas) they will eventually figure it out, but if you come in with bad assumptions like me and don't read carefully then you might spend a really long time trying to debug your model! Since it's a popular page I think it's worth it to be super clear

cc @svekars @holly1238 @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
Autocast documentation examples would break
Labels: module: docs, triaged, module: amp (automated mixed precision)

### 📚 The doc issue

In the `torch.autocast` [API reference](https://pytorch.org/docs/stable/amp.html#autocasting) we have several examples of autocasting that use simply `with autocast()`, e.g.

https://github.com/pytorch/pytorch/blob/594652f0e494caadac1076ab58e66af6823c3a83/torch/amp/autocast_mode.py#L44

However, since `torch.autocast` requires an argument `device_type`, this would fail with 
```
__init__() missing 1 required positional argument: 'device_type'
```

Another point of confusion that I have is why is `torch.autocast` defined in the `torch.amp.autocast_mode` file?

### Suggest a potential alternative/fix

Shall we update these docs to: `with torch.autocast("cuda")`?


cc @svekars @holly1238 @mcarilli @ptrblck
CUDACachingAllocator should be cuda memory merge/compact friendly
Labels: module: cuda, triaged, module: CUDACachingAllocator

### 🚀 The feature, motivation and pitch

I found iteration latency for  VIT huge increased greatly in big batch size(64), used nsight to profile it: 
![image](https://user-images.githubusercontent.com/1720972/180645163-4724b66f-a2ce-4ba8-a714-4d5c857722e0.png). we can see several cudaFrees and one cuda malloc.

When this happens, cuda doesn't have free memory available, so allocator should free blocks to let cuda merge adjacent blocks and make current malloc success. Currently CUDACachingAllocator will release all non split blocks(release_cached_blocks) when can't find one oversize blocks(release_available_cached_blocks). This behavior causes huge impact. 

So I propose:
1. Reduce cudaFrees by  letting allocator release appropriate blocks that are concisely enough to let cuda compact these blocks and make malloc success.
2. For further optimization, we can avoid cudaMalloc by managing cuda memory in caching allocator? We assume pytorch is using GPU exclusively in   training scenario

### Alternatives

_No response_

### Additional context

_No response_

cc @ngimel
upsample_bilinear2d() received an invalid combination of arguments
Labels: module: onnx, module: nn, triaged, module: regression

### 🐛 Describe the bug

In the past it was possible to use a Tensor within the `scale_factor` argument, but on newer versions this code fails. 

Works on every version:
```python
import torch
import torch.nn.functional as F
test_input = torch.rand(1,3,256,256)

x = F.interpolate(
    input=test_input, scale_factor=0.5, mode="bilinear", align_corners=False
)
print(x.shape)
```

Does not work on 1.11 and nightly anymore, but works on 1.10.1:
```python
import torch
import torch.nn.functional as F
test_input = torch.rand(1,3,256,256)

scale = torch.Tensor([0.5])
# same with
# scale = torch.tensor(0.5)

x = F.interpolate(
    input=test_input, scale_factor=scale, mode="bilinear", align_corners=False
)
print(x.shape)
```
```
    x = F.interpolate(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/functional.py", line 3919, in interpolate
    return torch._C._nn.upsample_bilinear2d(input, output_size, align_corners, scale_factors)
TypeError: upsample_bilinear2d() received an invalid combination of arguments - got (Tensor, NoneType, bool, list), but expected one of:
 * (Tensor input, tuple of ints output_size, bool align_corners, tuple of floats scale_factors)
      didn't match because some of the arguments have invalid types: (Tensor, NoneType, bool, list)
 * (Tensor input, tuple of ints output_size, bool align_corners, float scales_h, float scales_w, *, Tensor out)
```

It is possible to get it working with the current nightly with a workaround:
```python
import torch
import torch.nn.functional as F
test_input = torch.rand(1,3,256,256)

scale = torch.Tensor([0.5])

x = F.interpolate(
    input=test_input, scale_factor=scale.numpy()[0], mode="bilinear", align_corners=False
)
print(x.shape)
```
I would prefer it if I don't need to append this numpy suffix everywhere in my code and it did work in the past. That does not look very good. If there is a reason for this, then I would like to hear it, but I am assuming something went wrong when pytorch updated.

To make my code usable again, I need to do this:
```python
import torch
import torch.nn.functional as F
import numpy as np
test_input = torch.rand(1,3,256,256)

scale = torch.tensor(0.5)
scale = torch.Tensor([(1 / scale)]).numpy().astype(np.float64)[0]

x = F.interpolate(
    input=test_input, scale_factor=scale, mode="bilinear", align_corners=False
)
print(x.shape)
``` 
instead of
```
scale_factor = 1.0 / scale
```
Which does not look very good. :( 

### Versions

Mentioned above.

cc @ezyang @gchanan @zou3519 @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
optimize_for_mobile vulkan_prepack::conv2d_clamp_prepack
Labels: oncall: mobile, module: vulkan

### 🐛 Describe the bug

trying to convert a simple resnet18 model to mobile vulkan backend
i am being asked to report the bug `please report a bug to PyTorch. We don't have an op for vulkan_prepack::conv2d_clamp_prepack but it isn't a special case`

```python
import torch
from torch.utils.mobile_optimizer import optimize_for_mobile
print(torch.__version__)
m = torch.jit.load("ptl-models/resnet18_wd4_2_3_scripted.pt")
vm = optimize_for_mobile(m, backend='vulkan')
```

scripted.pt can be downloaded here https://drive.google.com/file/d/1crd0f-jq9GBAE0a7kKXAthgWcsHTNgYH/view?usp=sharing

```
1.12.0+cu102
Traceback (most recent call last):
  File "./pt_vulkan.py", line 5, in <module>
    vm = optimize_for_mobile(m, backend='vulkan')
  File "./venv/lib/python3.8/site-packages/torch/utils/mobile_optimizer.py", line 67, in optimize_for_mobile
    optimized_cpp_module = torch._C._jit_pass_vulkan_optimize_for_mobile(script_module._c, preserved_methods_str)
RuntimeError: 0 INTERNAL ASSERT FAILED at "../torch/csrc/jit/ir/alias_analysis.cpp":614, please report a bug to PyTorch. We don't have an op for vulkan_prepack::conv2d_clamp_prepack but it isn't a special case.  Argument types: Tensor, Tensor?, int[], int[], int[], int, NoneType, NoneType, 
```

### Versions

```
PyTorch version: 1.12.0+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 8.4.0-3ubuntu2) 8.4.0
Clang version: 10.0.0-4ubuntu1 
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.8.10 (default, Jun 22 2022, 20:18:18)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.15.0-41-generic-x86_64-with-glibc2.29
Is CUDA available: True
CUDA runtime version: 10.1.243
GPU models and configuration: GPU 0: Quadro RTX 5000 with Max-Q Design
Nvidia driver version: 515.48.07
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.4.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.0
[pip3] pytorchcv==0.0.67
[pip3] torch==1.12.0
[conda] Could not collect
```
Documentation for torch.cuda.Event(blocking=True) is wrong
Labels: module: docs, module: cuda, triaged

### 📚 The doc issue

https://pytorch.org/docs/stable/generated/torch.cuda.Event.html#torch.cuda.Event
doc states:
```
blocking (bool, optional): if ``True``, :meth:`wait` will be blocking (default: ``False``)
```

But cross referencing:
https://github.com/pytorch/pytorch/blob/30fb2c4abaaaa966999eab11674f25b18460e609/torch/csrc/cuda/Event.cpp#L48
and:
https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__EVENT.html#group__CUDART__EVENT_1g7b317e07ff385d85aa656204b971a042


[cudaEventBlockingSync](https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html#group__CUDART__TYPES_1g26509a522be9d449aa7c8c279612452d): Specifies that event should use blocking synchronization. A host thread that uses [cudaEventSynchronize()](https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__EVENT.html#group__CUDART__EVENT_1g949aa42b30ae9e622f6ba0787129ff22) to wait on an event created with this flag will block until the event actually completes.


### Suggest a potential alternative/fix

```
blocking (bool, optional): if ``True``, :meth:`synchronize` will be blocking. if ``False``,  :meth:`synchronize` will cause the CPU to busy-wait until the event has completed (default: ``False``).
```

cc @ezyang @gchanan @zou3519 @svekars @holly1238 @ngimel
Inconsistent implementation of quant_utils:: ChooseQuantizationParams compared with fbgemm:: ChooseQuantizationParams
Labels: oncall: quantization, triaged

### 🐛 Describe the bug

The documentation says it should use sum

https://github.com/pytorch/pytorch/blob/d576a7dc97d222ca8db5f8b876ce391866018a9c/aten/src/ATen/native/quantized/cpu/QuantUtils.h#L135-L137

> sum of absolute values of terms

However, the implementation uses `difference`:
https://github.com/pytorch/pytorch/blob/d576a7dc97d222ca8db5f8b876ce391866018a9c/aten/src/ATen/native/quantized/cpu/QuantUtils.h#L140-L143

---

Also, the comment says
https://github.com/pytorch/pytorch/blob/d576a7dc97d222ca8db5f8b876ce391866018a9c/aten/src/ATen/native/quantized/cpu/QuantUtils.h#L58

But `fbgemm` is using `sum`, not difference. See
https://github.com/pytorch/FBGEMM/blob/49061a284744423bad17162d45a9bdc56adb4739/src/QuantUtils.cc#L100-L103
```cpp
  double zero_point_from_min_error =
      std::abs(qmin) + std::abs(min / static_cast<double>(scale));
  double zero_point_from_max_error =
      std::abs(qmax) + std::abs(max / static_cast<double>(scale));
```

### Versions

The latest master.

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel
[Misleading] The doc started using Tensorflow terminology in the document to explain how to use the Pytorch code.
Labels: module: docs, triaged

### 📚 The doc issue

![image](https://user-images.githubusercontent.com/21982975/180585585-6077a456-c2d9-4e15-a1f6-014b93feb13b.png)
 the model must be executed in inference mode and operate on input tensors that do not collect gradient tape information (e.g., running with torch.no_grad).

### Suggest a potential alternative/fix

 the model must be executed in inference mode and operate on input tensors that do not collect gradient tape information (e.g., running with torch.no_grad).
Change it to be:
the model must be executed in inference mode and operate on input tensors that does not accumulate gradient. (e.g, setting the model with torch.no_grad).

cc @svekars @holly1238
[PyTorch/XLA] Improve the XLA PR landing process
Labels: module: ci, triaged, module: xla

### 🚀 The feature, motivation and pitch

# Current situation
Currently to merge a XLA breaking pr, we follow these procedures
1. PyTorch/XLA merge the fix PR, which will break the PyTorch/XLA head CI
2. PyTorch PR rebased and updated the [xla_hash](https://github.com/pytorch/pytorch/blob/master/.github/ci_commit_pins/xla.txt)
3. PyTorch merge the xla break pr
4. PyTorch/XLA rerun the head CI and it should be green again

# User expericncen flaw
This process has help a lot for pytorch CI stability. However it creates quite a lot of burden on the PyTorch/XLA maintainer side because
1. PyTorch/XLA CI will be broken when fix pr merged and won't be fix until corresponding PyTorch PR merged. 
2. Merging a pytorch pr is a lot more effort than merging a pytorch/xla PR. In step 2 pytorch pr needs to be rebased hence will rerun all test. Waiting for all tests to finish will take hours and might fail for various reason. In this whole time PyTorch/XLA head will be broken
3. using `merge -f` will help this merging delay but people are generally concern about this command because it is somewhat dangerous for such a complcaited codebase like pytorch.
4. PyTorch/XLA maintainer needs to closely monitor the upstream pr until it is merged.

# Proposal 
I am proposing a new merge process hopefully can address above issue
1. PyTorch/XLA rebased the "fix pr"
2. PyTorch pr rebased and pin to the PyTorch/XLA pr's branch in [xla.txt](https://github.com/pytorch/pytorch/blob/master/.github/ci_commit_pins/xla.txt)
3. PyTorch pr started a merge job and can wait until all test green(xla test should pass too)
4. PyTorch/XLA merge the fix pr (need to turn off auto-delete branch after merge)
5. PyTorch bot will update the [xla.txt](https://github.com/pytorch/pytorch/blob/master/.github/ci_commit_pins/xla.txt) to the new PyTorch/XLA master hash which will also have the fix

The advantage of this approach ares
1. PyTorch and PyTorch/XLA pr will be both green all the time
2. PyTorch/XLA maintainer only need to merge the fix pr the same day(before pytorch bot started). This requires much less coordination work between two sides.

# Work that needs to be done
PyTorch CI need to understand  [xla.txt](https://github.com/pytorch/pytorch/blob/master/.github/ci_commit_pins/xla.txt) can contain both branch name and commit.

FYI @suo @bdhirsh @wconstab 

### Alternatives

_No response_

### Additional context

_No response_

cc @seemethere @malfet @pytorch/pytorch-dev-infra @bdhirsh
Issues with custom types defining `__new__`
Labels: triaged, bug, oncall: pt2

Meta-internal example code at D38093407 with output P517669217.

The error is:
```
  File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/scripts/bwen/dynamo_example/test#link-tree/torchdynamo/symbolic_convert.py", line 690, in LOAD_ATTR
    result = BuiltinVariable(getattr).call_function(
  File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/scripts/bwen/dynamo_example/test#link-tree/torchdynamo/variables/builtin.py", line 229, in call_function
    constant_args = check_constant_args(args, kwargs)
  File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/scripts/bwen/dynamo_example/test#link-tree/torchdynamo/utils.py", line 414, in check_constant_args
    return all(x.is_python_constant() for x in itertools.chain(args, kwargs.values()))
  File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/scripts/bwen/dynamo_example/test#link-tree/torchdynamo/utils.py", line 414, in <genexpr>
    return all(x.is_python_constant() for x in itertools.chain(args, kwargs.values()))
  File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/scripts/bwen/dynamo_example/test#link-tree/torchdynamo/variables/base.py", line 148, in is_python_constant
    self.as_python_constant()
  File "/data/sandcastle/boxes/fbsource/fbcode/buck-out/dev/gen/scripts/bwen/dynamo_example/test#link-tree/torchdynamo/variables/lists.py", line 37, in as_python_constant
    return self.python_type()([x.as_python_constant() for x in self.items])
TypeError: __new__() missing 2 required positional arguments: 'lambda1' and 'lambda2'
```

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
linspace cpu and sometimes cuda is wrong on integral types
Labels: triaged, module: correctness (silent), module: python frontend, module: edge cases

Repro:
```python
import torch

print("cpu")
for dtype in (torch.int8, torch.int16, torch.int32, torch.int64):
    print(torch.linspace(0, 4, 50, dtype=dtype, device="cpu").bincount())

print("cuda")
for dtype in (torch.int8, torch.int16, torch.int32, torch.int64):
    print(torch.linspace(0, 4, 50, dtype=dtype, device="cuda").bincount())
```
prints:
```
cpu
tensor([13, 12, 12, 12,  1])
tensor([13, 16,  8, 12,  1])
tensor([16, 16,  8,  9,  1])
tensor([16, 12, 12,  9,  1])
cuda
tensor([13, 12, 12, 12,  1], device='cuda:0')
tensor([13, 12, 12, 12,  1], device='cuda:0')
tensor([13, 12, 12, 12,  1], device='cuda:0')
tensor([13, 12, 12, 12,  1], device='cuda:0')
```
int8 is correct for some reason
Unify c10::Event and at::cuda::CUDAEvent
Labels: module: cuda, triaged, better-engineering

### 🐛 Describe the bug

They are two implementations of essentially the same thing. CUDAEvent has a slightly more expansive API, and it also seems to support IPC events which the basic c10::Event doesn't support. It still makes sense to have a c10::cuda::CUDAEvent, but the shared functionality should live only in one place.

cc @ngimel @lw 

### Versions

master
nn.InstanceNorm and nn.GroupNorm are affected by padding, so they need to masking
Labels: high priority, triaged, module: nestedtensor, module: norms and normalization, module: padding, module: masked operators, oncall: pt2

### 🐛 Describe the bug

For batch_size > 1, variable-length inputs (e.g. speech, text) are padded in order to construct one batch tensor.

**When this tensor goes through nn.InstanceNorm series (nn.InstanceNorm1d, nn.InstanceNorm2d, ...) or nn.GroupNorm,
the calculations of mean and variance take the padded dummies altogether.**

This causes harmful batch inconsistency problem.

**So, nn.InstanceNorm and nn.GroupNorm must be modified to take valid input lengths and to mask out the padded dummies during calculating mean and variance.**

Sorry for not attaching example code.

### Versions

PyTorch version: 1.12.0

cc @ezyang @gchanan @zou3519 @cpuhrsch
backwards compatibility ALLOWLIST is misused
Labels: module: ci, triaged

### 🐛 Describe the bug

We now check against backwards incompatibility from the base commit (vs the most recent nightly), but the ALLOWLIST has never worked the way it was intended to. My understanding of the ALLOWLIST is that for every function + date we set, we allow the function to be "backwards incompatible" until the date.

What actually happens in CI is that the function is incompatible for at most a day, and we allow it to be incompatible for eternity until someone comes and adds back the lost schema. 

To illustrate:
commit a -> function exists
commit b -> function doesn't exist
.
.
.
commit c -> function doesn't exist

The check will complain at commit b that the function being lost is backwards incompatible as we're comparing the schema of b and a. After some time, the check will compare commits to a newer commit, so commit c will no longer complain about the function being missing (whether the new commit is a new nightly or just a newer commit)! 

## What we can do
1. If we continue letting the bc test in CI currently check for immediate regressions only, we should build some mechanism to create an issue or more permanently warn us when something has introduced incompatibility. Before we implement this, the OSS CI oncall would have to be hyper aware when these errors arise + notify/create an issue. In this case, we can treat the modify the current ALLOWLIST to be a list of schemas we know have regressed + also ensure we track which commit since it has regressed. This way, we can have a system of verifying when something leaves the regression list. 

2. We can also make the bc test on trunk compare against a longer term previous schema/something that represents what we promise to users. Unfortunately, this wouldn't be as easy as pinning it to an old commit, since that commit wouldn't have any new schemas added since then. 

Do people like 1 or 2 better? 

Realization happened in PR description of https://github.com/pytorch/pytorch/pull/81980.

### Versions

CI

cc @seemethere @malfet @pytorch/pytorch-dev-infra
test_sparse_matmul_cpu_complex128 fails on my local copy
Labels: module: sparse, triaged, module: complex

### 🐛 Describe the bug

```
======================================================================
FAIL: test_sparse_matmul_cpu_complex128 (__main__.TestSparseCPU)
This function test `torch.sparse.mm` when both the mat1 and mat2 are sparse tensors.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/raid/ezyang/pytorch-scratch2/torch/testing/_internal/common_device_type.py", line 377, in instantiated_test
    result = test(self, **param_kwargs)
  File "/raid/ezyang/pytorch-scratch2/torch/testing/_internal/common_utils.py", line 978, in wrapper
    fn(*args, **kwargs)
  File "/raid/ezyang/pytorch-scratch2/torch/testing/_internal/common_utils.py", line 3226, in wrapped
    f(self, *args, **kwargs, coalesced=True)
  File "test/test_sparse.py", line 3484, in test_sparse_matmul
    test_sparse_matmul(2, 10, [n, m], [m, p])
  File "test/test_sparse.py", line 3458, in test_sparse_matmul
    grad_with_custom_sparsity_pattern_test_helper(sparse_dims, nnz, shape_a, shape_b)
  File "test/test_sparse.py", line 3428, in grad_with_custom_sparsity_pattern_test_helper
    self.assertEqual(a.grad, a_grad)
  File "/raid/ezyang/pytorch-scratch2/torch/testing/_internal/common_utils.py", line 2364, in assertEqual
    assert_equal(
  File "/raid/ezyang/pytorch-scratch2/torch/testing/_comparison.py", line 1093, in assert_equal
    raise error_metas[0].to_error(msg)
AssertionError: Sparse COO values are not close!

Mismatched elements: 4 / 4 (100.0%)
Greatest absolute difference: 3.3639772140974413 at index (2,) (up to 1e-07 allowed)
Greatest relative difference: 2.0993343823035606 at index (0,) (up to 1e-07 allowed)

----------------------------------------------------------------------
```

This is on a CUDA 11.6 build

### Versions

master

cc @nikitaved @pearu @cpuhrsch @amjames @ezyang @anjali411 @dylanbespalko @mruberry @Lezcano
test_sparse_spdiags_cpu_bool fails on my local working copy
Labels: module: sparse, triaged

### 🐛 Describe the bug

Error looks like

``` ======================================================================                                              
ERROR: test_sparse_spdiags_cpu_bool (__main__.TestSparseCPU)                                                        
----------------------------------------------------------------------                                              
Traceback (most recent call last):                                                                                  
  File "/data/users/ezyang/pytorch-tmp/torch/testing/_internal/common_device_type.py", line 377, in instantiated_tes
t                                                                                                                   
    result = test(self, **param_kwargs)                                                                             
  File "/data/users/ezyang/pytorch-tmp/torch/testing/_internal/common_device_type.py", line 970, in only_fn         
    return fn(slf, *args, **kwargs)                                                                                 
  File "/data/users/ezyang/pytorch-tmp/test/test_sparse.py", line 3632, in test_sparse_spdiags                      
    check_valid(*case)                                                                                              
  File "/data/users/ezyang/pytorch-tmp/test/test_sparse.py", line 3590, in check_valid                              
    ref_out = reference(diags, offsets, shape)                                                                      
  File "/data/users/ezyang/pytorch-tmp/test/test_sparse.py", line 3583, in reference                                
    data = data[off:]                                                                                               
IndexError: slice() cannot be applied to a 0-dim tensor.                                                            
                                                                                                                    
----------------------------------------------------------------------      
```

Not sure why it's only my local copy.

### Versions

master

cc @nikitaved @pearu @cpuhrsch @amjames
Tensor.backward type hints clarification
Labels: module: docs, module: autograd, module: typing, triaged, actionable

### 📚 The doc issue

The signature of `Tensor.backward` looks like this:
https://github.com/pytorch/pytorch/blob/c52ee6dc0a00dd46f9424424b59dd8714496e9a1/torch/_tensor.py#L425-L427

And in the docstring it defines the args like this:
https://github.com/pytorch/pytorch/blob/c52ee6dc0a00dd46f9424424b59dd8714496e9a1/torch/_tensor.py#L454-L472

I'd like to clarify a few things about the types here. I have numbered them to allow for easier discussion.
* gradient (Tensor or None): 
1. It seems that the type here is clearly `Optional[torch.Tensor]`
2. I find the sentence "If a None value would be acceptable then this argument is optional." a bit odd - should we remove it? 

* retain_graph (bool, optional):
3. Why does this default to `None` rather than `False`?
       
* inputs (sequence of Tensor): 
4. It seems that the type is `Optional[Sequence[torch.Tensor]]` - should we update the docstring to say it is optional like the others?


### Suggest a potential alternative/fix

Should we add type hints?

cc @svekars @holly1238 @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7 @malfet @rgommers @xuzhao9 @gramster
Overloading multiple signatures for a single ref
Labels: triaged, module: primTorch

Consider an op like `arange` which is able to support the following two signatures using python arg parser

https://github.com/pytorch/pytorch/blob/8d0cbce0696e76c9e4c929c24c0331df25125b40/torch/csrc/autograd/python_torch_functions_manual.cpp#L87-L90

It seems tricky to add an op like this since Python does not support overloads and arguments with defaults must come after arguments that don't have defaults.  

but one solution could be to accept a more general signature and parse it yourself (similar to what python arg parser does):
  - in the below example, we have `arange(arg1: optional[Number], arg2: optional[Number], step=1, *, ...)`  add in the function body, have checks to route the call to the correct prim, which itself does not take any optional arguments
  - registering stubs with `@overload` decorator from the `typing` module seems to help with static checking tools 

https://github.com/pytorch/pytorch/blob/d1a9372e611b6e833f83fab57816819b7732e1cc/torch/_refs/__init__.py#L2989-L3055

Curious if others think this is a good general solution

cc @ezyang @mruberry @ngimel
Unknown builtin op: aten::broadcast_shapes
Labels: oncall: jit

### 🚀 The feature, motivation and pitch

`torch.broadcast_shapes` doesn't work with torch.jit.script：
```
@torch.jit.script
def broadcast_common_batch_shape(x, x_data, y_data):
    common_shape = torch.broadcast_shapes(x.size()[:-1], x_data.size()[:-1], y_data.size()[:-1])
    return (torch.broadcast_to(x, common_shape + x.size()[-1:]).contiguous(),
                torch.broadcast_to(x_data, common_shape + x_data.size()[-1:]).contiguous(),
                torch.broadcast_to(y_data, common_shape + y_data.size()[-1:]).contiguous())
````
It would be pleased to have this feature.

### Alternatives

_No response_

### Additional context

_No response_
Dependency header directory is not properly expanded in the utils.cpp_extention in ninja mode
Labels: module: cpp-extensions, triaged

### 🐛 Describe the bug

Build pytorch_sparse use torch.utils cpp_extension throws error of no such file or directory with `use_ninja=True`
```
fatal error: parallel_hashmap/phmap.h: No such file or directory
 | #include "parallel_hashmap/phmap.h"
```

In the pytorch_sparse, setup.py file, a third_party dependency is set with relative path. It is not expanded to a absolute paths when ninja build was used. 

Proposed solution: 
Adding `convert_to_absolute_paths_inplace(include_dirs)` under this line 
https://github.com/pytorch/pytorch/blob/94090150942cea8625b9d6068e41cf2c2a88429a/torch/utils/cpp_extension.py#L601.

### Versions

PyTorch version: 1.13.0a0+08820cb
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.23.2
Libc version: glibc-2.31

Python version: 3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:04:10)  [GCC 10.3.0] (64-bit runtime)
Python platform: Linux-4.15.0-36-generic-x86_64-with-glibc2.10
Is CUDA available: True
CUDA runtime version: 11.7.99

cc @malfet @zou3519
RuntimeError: CUDA error: no kernel image is available for execution on the device
Labels: module: cuda, triaged

### 🐛 Describe the bug

I am deploying a model using torchserve. It was easily deployed on Tesla K80 GPU. But now when i shifted it to newer GPU Nvidia A30. I am getting this error



```
torch_version: '1.12.0+cu113'
cudnn_version: 8302
```

![new](https://user-images.githubusercontent.com/89393062/180234274-d891796c-428c-412c-8ad3-e41676fa1c7b.png)


cc @ngimel
grid_sample and mode='bilinear' induces errors at discrete pixel locations
Labels: module: nn, triaged

### 🐛 Describe the bug

For an tensor `I`, if location `x` and `y` are integers (before normalisation), the result of bilinear interpolation of `I` at `(x,y)` should be strictly equal to `I(x,y)`. This is not currently the case when using `grid_sample` with `mode='bilinear'`.

### Reproduction steps

```python
import torch
print(f'Running PyTorch version: {torch.__version__}')

torchdevices = [torch.device('cpu')]
if torch.cuda.is_available():
  torchdevices.append(torch.device('cuda'))
  print('Default GPU is ' + torch.cuda.get_device_name(torch.device('cuda')))

for torchdevice in torchdevices:
  print('Running on ' + str(torchdevice))

  img = torch.rand(1, 1, 40, 40, device=torchdevice)
  theta = torch.zeros(1,2,3, device=torchdevice)
  theta[0,0,0] = 1
  theta[0,1,1] = 1

  align_opts = [True, False]
  for align_opt in align_opts:
    grid = torch.nn.functional.affine_grid(theta, img.shape, align_corners=align_opt)
    out = torch.nn.functional.grid_sample(img, grid, mode='bilinear', align_corners=align_opt)
    print(f"Residual with align_corners={align_opt}: {torch.linalg.vector_norm(out-img, ord=torch.inf).cpu().numpy()}")
```

### Observed behavior

```
Running PyTorch version: 1.12.0+cu113
Default GPU is Tesla T4
Running on cpu
Residual with align_corners=True: 2.768472768366337e-06
Residual with align_corners=False: 1.914799213409424e-06
Running on cuda
Residual with align_corners=True: 2.562999725341797e-06
Residual with align_corners=False: 1.0468065738677979e-06
```

### Expected behavior

`out` should be equal to `img` for an identity transformation.

### Additional context

The errors are seen in both CPU and GPU mode. This issue was first observed as part of using kornia:
https://github.com/kornia/kornia/issues/1799

### Versions

- PyTorch Version (e.g., 1.0): 1.12.0+cu113
- OS (e.g., Linux): Google colab
- How you installed PyTorch (`conda`, `pip`, source): Google colab default version
- Build command you used (if compiling from source): N/A
- Python version: 3.7.13 (default, Apr 24 2022, 01:04:09)
- CUDA/cuDNN version: cu113
- GPU models and configuration: Tesla T4
- Any other relevant information: N/A

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
Compatibility with newest MKL
Labels: module: build, triaged, module: mkl

### 🚀 The feature, motivation and pitch

libtorch 1.12.0 is currently released with version 2020.0
please make torch compatible with latest version of MKL as it is much faster. I note that the `cblas_?gemm` functions are about twice as fast on my machine(AMD) for ~1000x1000 matrices

I tried building libtorch with 2022.0.1 version of MKL.
a call to at::matmul yeilds error
`Intel MKL ERROR: Parameter 9 was incorrect on entry to cblas_sgemm_batch.`


### Alternatives

_No response_

### Additional context

_No response_

cc @malfet @seemethere