gopikrsmscs/torch-issues huggingface dataset:
[RFC] Module specific workflows
Labels: module: rocm, triaged

@jeffdaily asked for a ciflow label to trigger ROCm jobs because `ciflow/trunk` runs a lot of non-rocm things. This is totally reasonable and likely to be a common request from module maintainers, so it's worth designing a process that can scale well.

The problem definition is: module maintainers want to be able to run a set of jobs specific to their module on PRs. Sometimes those jobs are a subset of the `trunk` or `periodic` jobs that we just want PR-time signal on, sometimes they wholly custom to a module (experimental tests, optional tests, etc.).

Here is our proposal. I phrase it specific to ROCm, but it can be read as generic across any module (e.g. onnx, functorch).
- The Infra team will continue to maintain the _core workflows_ (`pull`, `trunk`), etc. as they are today, so no change there.
  - This includes ever-stricter standards on time-to-signal and reliability, since the core workflows are run for all PT devs.
  - It also includes guarantees that the core workflows will be green; i.e. we will revert if one is broken.
- We define a ROCm-specific GitHub Actions workflow.
- The workflow is scoped to only affect ROCm maintainers (or other interested parties).
  - Initially, this means that it will be triggered by `ciflow/rocm` label only (not pull requests or master).
  - In the future, one could imagine tracking a mirror of the master branch with custom validation jobs.
- The workflow is maintained by the ROCm maintainers.
  - There are no restrictions on what stuff can be put in there. The only caveat is that we need to make sure usage of shared global resources (e.g. the AMD runners) does not affect core workflows.
  - The PyTorch Infra team will not maintain the workflow; that is, breakages to the workflow will not be triaged by the infra team and we will not auto-revert PRs based on failures in the workflow.
  - Notably, keeping the workflow in sync with ROCm jobs in core workflows is the responsibility of the ROCm maintainers. We will develop linting/syncing infra to help there though.

The initial version of the ROCm workflow will be a strict subset of what we run on `trunk`, but one could imagine putting arbitrary ROCm-specific testing there.


cc @jeffdaily @sunway513 @jithunnair-amd @ROCmSupport @KyleCZH
Orthogonal Polynomials
Labels: feature, triaged, module: special

# Orthogonal Polynomials

A brief proposal for providing a complete suite of orthogonal polynomials as PyTorch operators. Enjoy!

One of a five-part series of special functions issues:

- Gamma and Related Functions (#78065)
- Bessel and Related Functions (#76324)
- Orthogonal Polynomials (#80152)
- Elliptic Functions and Integrals (#80157)
-

## API

### Chebyshev Polynomials

- [x] `chebyshev_polynomial_t(input: Tensor, n, *, out=None) → Tensor`

Chebyshev polynomial of the first kind $T_{n}\left(\text{input}\right)$.

- [x] `chebyshev_polynomial_u(input: Tensor, n, *, out=None) → Tensor`

Chebyshev polynomial of the first kind $U_{n}\left(\text{input}\right)$.

- [x] `chebyshev_polynomial_v(input: Tensor, n, *, out=None) → Tensor`

Chebyshev polynomial of the first kind $V_{n}\left(\text{input}\right)$.

- [x] `chebyshev_polynomial_w(input: Tensor, n, *, out=None) → Tensor`

Chebyshev polynomial of the first kind $W_{n}\left(\text{input}\right)$.

### Shifted Chebyshev Polynomials

- [x] `shifted_chebyshev_polynomial_t(input: Tensor, n, *, out=None) → Tensor`

Shifted Chebyshev polynomial of the first kind $T_{n}^{\ast}\left(\text{input}\right)$.

- [x] `shifted_chebyshev_polynomial_u(input: Tensor, n, *, out=None) → Tensor`

Shifted Chebyshev polynomial of the first kind $U_{n}^{\ast}\left(\text{input}\right)$.

- [x] `shifted_chebyshev_polynomial_v(input: Tensor, n, *, out=None) → Tensor`

Shifted Chebyshev polynomial of the first kind $V_{n}^{\ast}\left(\text{input}\right)$.

- [x] `shifted_chebyshev_polynomial_w(input: Tensor, n, *, out=None) → Tensor`

Shifted Chebyshev polynomial of the first kind $W_{n}^{\ast}\left(\text{input}\right)$.

### Hermite Polynomials

- [x] `hermite_polynomial_h(input: Tensor, n, *, out=None) → Tensor`

Physicist’s Hermite polynomial $H_{n}\left(\text{input}\right)$.

- [x] `hermite_polynomial_he(input: Tensor, n, *, out=None) → Tensor`

Probabilist’s Hermite polynomial $He_{n}\left(\text{input}\right)$.

### Charlier Polynomials

- [ ] `charlier_polynomial_c(input: Tensor, a, n, *, out=None) → Tensor`

### Gegenbauer Polynomials

- [ ] `gegenbauer_polynomial_c(input: Tensor, m, n, *, out=None) → Tensor`

Gegenbauer polynomial $C_{n}\left(\text{input}\right)$.

### Jacobi Polynomials

- [ ] `jacobi_polynomial_p(input: Tensor, a, b, n, *, out=None) → Tensor`

Jacobi polynomial $P_{n}\left(\text{input}\right)$.

### Laguerre Polynomials

- [ ] `generalized_laguerre_polynomial_l(input: Tensor, a, n, *, out=None) → Tensor`

- [x] `laguerre_polynomial_l(input: Tensor, n, *, out=None) → Tensor`

### Legendre Polynomials

- [x] `legendre_polynomial_p(input: Tensor, n, *, out=None) → Tensor`

Legendre polynomial $P_{n}\left(\text{input}\right)$.

### Shifted Legendre Polynomials

- [ ] `shifted_legendre_polynomial_p(input: Tensor, n, *, out=None) → Tensor`

Shifted Legendre polynomial $P_{n}^{\ast}\left(\text{input}\right)$.
activation checkpointing with non_reentrant implementation memory leaks
Labels: high priority, triage review, oncall: distributed, triaged

### 🐛 Describe the bug

This is coming from an internal report where using `CheckpointWrapper` with the `CheckpointImpl.NO_REENTRANT` gradually leaks memory, leading to CUDA OOM. 

### Versions

main

cc @ezyang @gchanan @zou3519 @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501
CPUProfilingAllocator greedy allocation plan generation failed
Labels: oncall: mobile

### 🐛 Describe the bug

Segmentation fault during greedy allocation plan generation using CPUProfilingAllocator. After carefully looking into the code, I found the root cause is the `free_size_to_offset` doesn't consider tensors with the same size. 

I fixed this issue by let `free_size_to_offset` map a size to a list of offset. When there are a new tensor allocation with existing size, it will add to the existing item's value list.


### Versions

Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.10.5 (main, Jun 16 2022, 19:48:34) [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.4.56.bsk.10-amd64-x86_64-with-glibc2.31
Is CUDA available: N/A
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] torch==1.13.0a0+git226a5e8
[conda] Could not collect
[feature request] Add support for a custom DatasetFetcher in DataLoader 
Labels: module: dataloader, triaged, enhancement, module: data

### 🚀 The feature, motivation and pitch

I'm working on a time-series dataset, and I stumbled upon a limitation withing pytorch's DataLoader that I'd like to address here.

I have a custom map-style dataset that implements a `__getitem__` and `__len__` methods.

`__getitem__`  has a following signature and returns by default features of shape `(seq_length, n_features)` and an integer label.
```python
def __getitem__(self, i: int, seq_length: int = 100):
    # ....
    return features, label
```
Sequence length is fixed by default, but can be changed by passing a `seq_length` argument.

I'm using an LSTM + Transformer model that is able to process an input of variable sequence length.\
BUT I'm not leveraging this feature of my model because pytorch's DataLoader expects all the samples returned by the dataset to be of the same shape.
A traditional approach to solving this problem is **sequence padding**. This doesn't really solve the problem in my opinion (the model still gets constant sequence length as an input) and leads to a range of further questions (which values to use for padding, how to interpolate ect.).

**What I would like to do instead** is to use a custom DatasetFetcher to create batches that have constant sequence length within a batch, but **varying** sequence length over the dataset.\
So let's say or batch has a shape `batch_size, seq_length, n_features`.  (Let batch_size=32, n_features=10)\
Our DataLoader will sample indices for the first batch and then fetch samples with a constant `seq_length=100`, which will result in a batch of shape `32, 100, 10`.\
On the next batch, it will randomly (or by any other rule) pick a new sequence length (let's say 80) 
and create a batch of shape `32, 80, 10`.

```python
# Default behaviour
data = [self.dataset[idx] for idx in possibly_batched_index]
# New behaviour
data = [self.dataset.__getitem__(idx, seq_length) for idx in possibly_batched_index]
```

I'm pretty sure it will have a regularization effect.





### Alternatives

_No response_

### Additional context

_No response_

cc @SsnL @VitalyFedyunin @ejguan @NivekT
Expose more MAGMA backends for solve_triangular
Labels: triaged, module: linear algebra, module: magma

### 🚀 The feature, motivation and pitch

MAGMA has quite a few (at least 3) backends for triangular solvers, and their efficiency varies wildly. We should expose as many backends as we can and do a proper benchmark of all of them to have faster triangular solvers, as triangular solvers are the basis for almost all other linalg algorithms. 

This was discovered while running the benchmarks for https://github.com/pytorch/pytorch/pull/80074

cc @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano
Allow a user provided "test name - test time" mapping file work with pytorch's test sharding mechanism
Labels: module: ci, triaged

### 🚀 The feature, motivation and pitch

We're running pytorch unit tests with sharding, but we can only do plain test sharding because "test time-based sharding" is only available from pytorch's past CI data, for example, `get_shard_based_on_S3` needs `test_times_file` from S3.

A plain sharding doesn't always work well because some shards finish in 20 minutes while other shards take longer than 3 hour to finish. This happens more frequently as we're having more primTorch tests and they all occurred in one test file (suite).

### Alternatives

It's ok that pytorch CI collects their own time data. Can we add some mechanism that allows users to collect and export their own test time data? Can I then save this mapping file somewhere and later reuse this data file to get a better sharding?

We currently run our tests with the below command like the pytorch CI does:
```
CI=1 python test/run_test.py -v --save-xml
```
This outputs CI stats to xml files `test/test-reports/python-unittest/` which contain test name - test time mapping. It's nice if we can rework the stats parser `upload_test_stats.py` to dump a mapping file and let the "test sharder" `get_shard_based_on_S3` be able to reuse this file.

(impl details: If I run test in 4 shards and there are 4 mapping files dumped, do we take an extra step to merge them or do we allow the "test sharder" to take 4 sharded mapping files?)

### Additional context

N/A

cc @seemethere @malfet @pytorch/pytorch-dev-infra @mruberry @janeyx99 @suo @ptrblck
Provide error message when thread pool is exhausted in RPC
Labels: high priority, oncall: distributed, triaged, module: rpc

### 🐛 Describe the bug

See issue raised in https://github.com/pytorch/pytorch/issues/80017. We should provide a better error message when we can no longer send requests due to exhaustion of threads in thread pool. Docs: https://pytorch.org/docs/master/rpc.html#torch.distributed.rpc.TensorPipeRpcBackendOptions

### Versions

current

cc @ezyang @gchanan @zou3519 @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @kwen2501 @jjlilley @mrzzd @H-Huang
Complex support in DDP
Labels: oncall: distributed, triaged, module: ddp

### 🐛 Describe the bug

There are a few issues to support complex tensors in DDP:

1. Currently, passing in complex tensors to DDP does not work because ProcessGroupNCCL doesn't know how to handle the type. @kumpera has a WIP fix in https://github.com/pytorch/pytorch/pull/74039. 
2. Conjugated tensors won't work out of the box as they need to be materialized before calling `view_as_real` for correctness.

As an e2e test, we should verify that training local model with complex inputs and DDP with complex inputs for conjugated /unconjugated complex tensors is equivalent. 

### Versions

main

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501
FakeTensor: Support torch.tensor([FakeTensor, 0])
Labels: triaged, module: meta tensors

### 🚀 The feature, motivation and pitch

Repro:
```
with enable_torch_dispatch_mode(FakeTensorMode(inner=None)):
    torch.tensor([torch.ones([], dtype=torch.int64), 0],)
```
> NotImplementedError: Could not run 'aten::_local_scalar_dense' with arguments from the 'Meta' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process

`torch.tensor` undergoes a special path that doesn't go through a lot of the typical `__torch_dispatch__`. 



### Alternatives

_No response_

### Additional context

This showed up in the `maml` benchmark with torch dynamo. 

cc @Chillee @ezyang @zou3519 @albanD @samdow
pow CUDA tensor raised to CPU scalar tensor result can't backward properly
Labels: module: autograd, triaged, actionable

### 🐛 Describe the bug

Here is a short snippet to demonstrate how difficult this could be to traceback.
```
>>> a = torch.tensor(1.0, device='cuda')
>>> b = torch.tensor(2.0, device='cuda')
>>> c = torch.tensor(2.0, device='cpu')
>>> a.requires_grad = True
>>> (a**c).backward()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/amai/.conda/envs/th/lib/python3.9/site-packages/torch/_tensor.py", line 363, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/amai/.conda/envs/th/lib/python3.9/site-packages/torch/autograd/__init__.py", line 173, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: Expected condition, x and y to be on the same device, but condition is on cpu and x and y are on cuda:0 and cuda:0 respectively
>>> (a**b).backward()
```
When running with anomaly mode, it just points to the PowBackward1 as the issue, but there is no explanation of why.

### Versions

1.11.0, 1.13.0. The other issue is present across different environments and comptuers.

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7 @gchanan
Support `antialias` option on `torch.interpolate` for ONNX export
Labels: module: onnx, triaged, onnx-triaged

### 🚀 The feature, motivation and pitch

Hi, I'm working on a model that uses `torch.interpolate` for upsampling/downsampling with `antialias` function, i.e., `torch>=1.11.0`.  Antialiasing is necessary before downsampling to remove upper half frequency components.

I would like to be able to export this model with ONNX. But, currently it does not yet support the exportation with `antialias` option. 

Is it possible to request this addition on ONNX export?

Many thanks before.

### Alternatives

It is also possible to create a custom convolution for low-pass filter to perform similar thing as the `antialias` option. In this case, ONNX export should be OK. However, it would also be nice to have it supported to sync with the latest functionality of PyTorch.

### Additional context

ONNX version 1.12.0
`torch.special.gammainc` backward pass with respect to the first argument
Labels: module: distributions, module: autograd, triaged

### 🚀 The feature, motivation and pitch

The current implementation of [`torch.special.gammainc`](https://pytorch.org/docs/stable/special.html#torch.special.gammainc) only supports backward pass with respect to the second argument. The documentation notes that I should open a GitHub issue in case I'm interested in the gradient w.r.t. the first argument.

The `gammainc` function (and the related [`gammaincc`](https://pytorch.org/docs/stable/special.html#torch.special.gammaincc)) is used when computing the CDF and the survival function of the [Gamma](https://en.wikipedia.org/wiki/Gamma_distribution) and [Poisson](https://en.wikipedia.org/wiki/Poisson_distribution) distributions. These are some of the most commonly used distributions for modeling non-negative & count data in statistics.

One use case where we need to backpropagate w.r.t. both arguments of `gammainc` is when modeling time until an event (e.g., predicting time until death in survival analysis or time until an earthquake in temporal point processes). If we model time until the event with a Gamma distribution, computing the likelihood of a partially observed / censored sequence will involve computing the survival function of the Gamma distribution, which relies on `gammainc`. Since we cannot backpropagate, we cannot learn the first parameter of the Gamma distribution. 

Here is a minimal example that reproduces the problem.
```python
import torch

a = torch.tensor(2.0, requires_grad=True)
b = torch.tensor(0.5, requires_grad=True)
t = torch.tensor(1.0)

# Probability that the event hasn't occurred until time t
likelihood = torch.special.gammaincc(a, b * t)
likelihood.backward()  # raises NotImplementedError
```
This code would be equivalent to `likelihood = 1.0 - torch.distributions.Gamma(a, b).cdf(t)` if `torch.distributions.Gamma` implemented the `cdf` method.

### Alternatives

_No response_

### Additional context

This question has already been raised in #67763 but the issue was closed since the OP was actually interested in the derivative w.r.t. the second argument, which was already implemented.

cc @fritzo @neerajprad @alicanb @nikitaved @ezyang @albanD @zou3519 @gqchen @pearu @soulitzer @Lezcano @Varal7
static builds are broken by MKL_DNN
Labels: module: build, triaged, module: regression, module: third_party

### 🐛 Describe the bug

Building with BUILD_SHARED_LIBRARIES=OFF  will fail if MKL / MKL DNN is enabled. 

```shell
USE_CUDA=OFF BUILD_SHARED_LIBS=OFF USE_TENSORPIPE=OFF python setup.py install
```
Tensor pipe has another issue with static builds here: https://github.com/pytorch/tensorpipe/issues/449

Errors out like
```
CMake Error:
  Running

   '/usr/bin/ninja' '-C' '/home/anush/github/pytorch/build' '-t' 'recompact'

  failed with:

   ninja: error: build.ninja:63037: bad $-escape (literal $ must be written as $$)

```

This is because when MKL is enabled and it is a static build it generates wrong targets in the Ninja file:

```
 $<TARGET_FILE:dnnl_graph>  
```


### Versions

Top of main

cc @malfet @seemethere
when forward use **kwargs，how to construct the example_ Inputs parameter in jit.trace?
Labels: oncall: jit

### 🐛 Describe the bug

import torch

class Model(nn.Module):
    def forward(self, **kwargs):
         # kwargs contains more than dozens of tensors
         pass

model = Model()
trace_model = torch.jit.trace(model, example_inputs=??)

### Versions

PyTorch version: 1.6.0+cu101
Is debug build: False
CUDA used to build PyTorch: 10.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.3 LTS (x86_64)
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
Clang version: Could not collect
CMake version: version 3.10.2
Libc version: glibc-2.26

Python version: 3.7.5 (default, Nov  7 2019, 10:50:52)  [GCC 8.3.0] (64-bit runtime)
Python platform: Linux-3.10.0-1.3.2.el7.x86_64-x86_64-with-Ubuntu-18.04-bionic
Is CUDA available: True
CUDA runtime version: 10.1.243
GPU models and configuration: 
GPU 0: GeForce RTX 2080 Ti
GPU 1: GeForce RTX 2080 Ti

Nvidia driver version: 440.44
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.18.5
[pip3] torch==1.6.0+cu101
[pip3] torchvision==0.7.0+cu101
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.2.0           h06a4308_296  
[conda] mkl-service               2.3.0            py38h27cfd23_1  
[conda] mkl_fft                   1.3.0            py38h42c9631_2  
[conda] mkl_random                1.2.1            py38ha9443f7_2  
[conda] numpy                     1.20.1           py38h93e21f0_0  
[conda] numpy-base                1.20.1           py38h7d8b39e_0  
[conda] numpydoc                  1.1.0              pyhd3eb1b0_1
Comprehensive documentation for Tensor indexing?
Labels: module: docs, triaged, module: advanced indexing

### 📚 The doc issue

Is there a comprehensive doc page for Tensor indexing, like there is for [numpy.ndarray](https://numpy.org/doc/stable/user/basics.indexing.html)?

I am facing some inconsistencies, particularly when using combined advanced boolean indexing and basic indexing, where the behaviour is clearly documented in the numpy side of things, but I am getting IndexError s in pytorch.

Would be great to know to what extent Tensor indexing mimics ndarray indexing.

Cheers.

### Suggest a potential alternative/fix

Add a page like https://numpy.org/doc/stable/user/basics.indexing.html#advanced-indexing to the docs.

cc @svekars @holly1238
Deterministic `index_put` on CUDA fails when broadcasting is required
Labels: triaged, module: advanced indexing

### 🐛 Describe the bug

`index_put` with `torch.use_deterministic_algorithms(True)` when broadcasting is implied fails on CUDA while it succeeds on CPU and without `use_deterministic_algorithms`.

Note that while this appears to be related to previous issues, I do not believe this is a duplicate of an existing bug that has been _already_ fixed. See also: #57515, #61032, #61612, #67189, #72053

Report originated from forum post: https://discuss.pytorch.org/t/assigning-tensor-to-multiple-rows-on-gpu/154421
Repro:
```
import torch
torch.use_deterministic_algorithms(True)
x = torch.zeros((5,4), device=torch.device('cuda:0'))
x[[False,True,False,True,True]] = torch.tensor([1.0, 1.0, 1.0, 1.0], device=torch.device('cuda:0'), dtype=torch.float32)
```

I'll open a PR right after the posting of this issue with a band-aid fix, but I'm skeptical of its robustness.

CC @ptrblck @ngimel

### Versions

```
Collecting environment information...
PyTorch version: 1.12.0a0+bd13bc6
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.22.3
Libc version: glibc-2.31

Python version: 3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:04:10)  [GCC 10.3.0] (64-bit runtime)
Python platform: Linux-5.13.0-30-generic-x86_64-with-glibc2.10
Is CUDA available: True
CUDA runtime version: 11.6.124
GPU models and configuration:
GPU 0: NVIDIA RTX A6000
GPU 1: NVIDIA RTX A6000

Nvidia driver version: 470.103.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.4.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.4.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.4.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.4.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.4.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.4.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.4.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] pytorch-quantization==2.1.2
[pip3] torch==1.12.0a0+bd13bc6
[pip3] torch-tensorrt==1.1.0a0
[pip3] torchtext==0.13.0a0
[pip3] torchvision==0.13.0a0
[conda] magma-cuda110             2.5.2                         5    local
[conda] mkl                       2019.5                      281    conda-forge
[conda] mkl-include               2019.5                      281    conda-forge
[conda] numpy                     1.22.3           py38h1d589f8_2    conda-forge
[conda] pytorch-quantization      2.1.2                    pypi_0    pypi
[conda] torch                     1.12.0a0+bd13bc6           dev_0    <develop>
[conda] torch-tensorrt            1.1.0a0                  pypi_0    pypi
[conda] torchtext                 0.13.0a0                 pypi_0    pypi
[conda] torchvision               0.13.0a0                 pypi_0    pypi
```
[CI] Do we run all cpp tests on CI?
Labels: module: ci, triaged

### 🐛 Describe the bug

We should! But recently, @vors pointed out that there have been regressions not caught by CI before. https://github.com/pytorch/pytorch/pull/79926 

Proposal:
We should make sure all cpp tests that are expected to run are run somewhere in CI.

### Versions

CI

cc @seemethere @malfet @pytorch/pytorch-dev-infra
PrimTorch burns in static shapes
Labels: triaged, module: primTorch

With the latest version of PyTorch, I've been migrating TorchInductor to prims (see https://github.com/pytorch/torchdynamo/pull/431) and I've noticed many cases where PrimTorch burns in shapes in cases where the prior _decomps didn't.

The top culprits I see are:
- `prims.broadcast_in_dim` with static integer args
- Tensor constants containing shapes (e.g. numel)

cc @ezyang @mruberry @ngimel @Chillee
[feature request] LazyTensor that provides/loads/computes its contents only upon request to be returned from torch.load
Labels: feature, triaged, module: lazy

### 🚀 The feature, motivation and pitch

Discussed in https://github.com/pytorch/pytorch/issues/29523#issuecomment-1157729352 and originally proposed by @dzhulgakov in https://github.com/pytorch/pytorch/issues/64601#issuecomment-920469049

Scenario: do not load materialize state_dict all at once at torch.load time and provide some sort of lazy tensors that either + maybe some usecases by @stas00 also referenced in the issue above (https://github.com/pytorch/pytorch/issues/29523#issuecomment-1157230391):
* know how to lazy_tensor.copyto(param) -> this would be the most efficient and allow GPUDirect and other smart implementations of reading from disk directly to the GPU-placed param
* also support torch.as_tensor(lazy_tensor) to materialize to a dense tensor

Probably this would also include adding `torch.load(..., lazy = True)` or `map_location = 'lazy'` option and modifying load_state_dict to dispatch copy_ call to instance method copyto or if it doesn't exist, call torch.as_tensor(lazy_tensor) / use `__array__` before copy_ (this would allow working copying from NumPy arrays or h5py Datasets. A problem may be to not call copyto on non torch-known objects or somehow figure out this case. Currently NumPy does not have instance method copyto, but it may be introduced in the future and this might cause problems.

Maybe having access to tensor meta-info (like shape) should be done in non-lazy way (discussable)

As pointed by @albanD, probably this can be prototyped in Python. But this idea of a pattern of optimized loading of state dict maybe deserves some design in non-python case as well.

I don't know if this has any relation to existing LazyTensor XLA-like infra in PyTorch

Implementation probably would be different depending on underlying storage format: zip or pickle

### Alternatives

_No response_

### Additional context

_No response_
Modify update-viable-strict GHA to use internal version of checkout
Labels: module: ci, triaged

I previously encountered an error regarding workflow permissions (linked [here](https://github.com/pytorch/pytorch-canary/runs/6921418059?check_suite_focus=true)) which was temporarily resolved by using `actions/checkout@v2` instead of our internal version of checking out pytorch, which would be more reliable. 

cc @seemethere @malfet @pytorch/pytorch-dev-infra
test_meta_vstack_cuda_int16 (__main__.TestMetaCUDA) Fails with DEBUG=1
Labels: module: autograd, triaged, needs research

### 🐛 Describe the bug

`python test/test_meta.py -k test_meta_vstack_cuda_int16`

> ERROR: test_meta_vstack_cuda_int16 (__main__.TestMetaCUDA) 
Traceback (most recent call last):
  File "/scratch/eellison/pytorch/test/test_meta.py", line 325, in run_meta_crossref
    meta_rs = func(*meta_args, **meta_kwargs)
  File "/private/home/eellison/anaconda3/lib/python3.8/site-packages/torch/overrides.py", line 1768, in wrapped
    return f(self, *args, **kwargs)
  File "/private/home/eellison/anaconda3/lib/python3.8/site-packages/torch/overrides.py", line 1873, in __torch_function__
    return func(*args, **kwargs)
RuntimeError: self__storage_saved.value().is_alias_of(result.storage()) INTERNAL ASSERT FAILED at "/raid/eellison/pytorch/torch/csrc/autograd/generated/VariableType_4.cpp":13537, please report a bug to PyTorch. 

`test_meta_unsqueeze_cuda_int8` also fails


### Versions

master

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
All {view}_scatter variants should support all (or most) dtypes
Labels: triaged, module: functionalization

We've been using `{view}_scatter` to replace some in-place operations (e.g. x`.diag().copy_(y) -> diagonal_scatter(x, y)`) for the purposes of composite compliance. However, as discovered by @kshitij12345, it turns out that the {view}_scatter operations do not have the same coverage as the original operations (they may not support complex dtype, and they may not support complex autodiff even if they support complex dtype).

We should make sure the {view}_scatter operations have the same coverage (dtype, autograd, etc) as their in-place `{view}.copy_(...)` variants.

cc @bdhirsh
Automatically calculate output_shape of sequential model (or any other fCNN)
Labels: triaged, module: meta tensors

### 🚀 The feature, motivation and pitch

Follow up on #79512, if support for meta conv layers is added, hypothetically, it should be possible to do this:

```python
import torch

feature_extractor = torch.nn.Sequential(
    torch.nn.Conv2d(3, 8, kernel_size=3),
    torch.nn.ReLU(),
    torch.nn.MaxPool2d(kernel_size=2),
    torch.nn.Conv2d(8, 16, kernel_size=4),
    torch.nn.ReLU(),
    torch.nn.MaxPool2d(kernel_size=3)
)

b_img = torch.rand(32, 3, 28, 28)
b_img.to('meta')
feature_extractor.to('meta')

print(feature_extractor(b_img).shape)

# mlp stuff here
```

### Alternatives


```python
import torch

feature_extractor = torch.nn.Sequential(
    torch.nn.Conv2d(3, 8, kernel_size=3),
    torch.nn.ReLU(),
    torch.nn.MaxPool2d(kernel_size=2),
    torch.nn.Conv2d(8, 16, kernel_size=4),
    torch.nn.ReLU(),
    torch.nn.MaxPool2d(kernel_size=3)
)

b_img = torch.rand(32, 3, 28, 28)

# this is computationally expensive

print(feature_extractor(b_img).shape)

```

### Additional context

I am currently working on #79834 which should add support for meta conv layers.
Automatically use CUDA
Labels: triaged, needs research, module: python frontend

### 🚀 The feature, motivation and pitch

To instantiate the Torch DSL you must presently use the following code:

```
import torch; torch.set_default_tensor_type(torch.cuda.FloatTensor)
from torch import *
```

The feature request is that the first line happens by default when CUDA is available. 

Additionally, switching modes should be as simple as:
```
cpu() # now everything happens on the CPU
```
```
gpu() # now everything happens on the GPU
```


### Alternatives

_No response_

### Additional context

_No response_
[ONNX] Replace test inheritance for `test/onnx/test_models.py` with parameterizing 
Labels: module: onnx, triaged, onnx-triaged

### 🚀 The feature, motivation and pitch

`test/onnx/test_models_onnxruntime.py` Inherits tests from `test/onnx/test_models.py`, swapping the onnx backend to test with.
Rewrite with parameterizing to clear test cases and test files relation.

### Alternatives

_No response_

### Additional context

_No response_
Parameter.__deepcopy__ doesn't preserve view relationships
Labels: module: nn, triaged, module: correctness (silent)

When deepcopying a module with `Parameter`s that share the same storage, the resultant copy does not maintain the view relationship.

```python
import torch
from copy import deepcopy

class MyModule(torch.nn.Module):
    def __init__(self):
        super().__init__()
        t = torch.randn(3, requires_grad=True)
        self.p1 = torch.nn.Parameter(t)
        self.p2 = torch.nn.Parameter(t)

m = MyModule()

# True; parameters share same storage
print(m.p1.storage().data_ptr() == m.p2.storage().data_ptr())

m_copy = deepcopy(m)

# False; parameters don't share same storage for copy
print(m_copy.p1.storage().data_ptr() == m_copy.p2.storage().data_ptr())
```

This happens because memoization for `Parameter.__deepcopy__()` is done at the `Parameter` level, and so will only kick in if it sees the same `Parameter` object twice:

https://github.com/pytorch/pytorch/blob/bcc4dba439db7a9e11dec9bc98a41069a7adef57/torch/nn/parameter.py#L52-L58

In contrast, `Tensor.__deepcopy__()` does memoization at the `Tensor` level, deepcopying the underlying storage once and reusing if the tensor has been seen before (preserving view relationships). 

Ideally, `Parameter.__deepcopy__()` memoization should happen at this level as well. Proposed fix is to tweak `Parameter.__deepcopy__()` to call `Tensor.__deepcopy__()` instead of `clone()`.

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
Improve clarity by making sharding a static nightly update
Labels: module: ci, triaged

Currently, sharding is done at every CI job, which involves downloading and parsing data from the outside world in the job, which can allow for flakiness or redness that is NOT the fault of the commit. Now that we have GH1, we can write test times to the repo every night and the CI jobs can read from that file in the repo instead of needing to access the outside world.

cc @seemethere @malfet @pytorch/pytorch-dev-infra
android-tests is often flaky
Labels: module: ci, triaged

### 🐛 Describe the bug

Twice every day or so, this test will fail to download important things and flakily fail. This detriments CI reliability 
https://github.com/pytorch/pytorch/runs/6923556353?check_suite_focus=true and causes confusion for the oncall and any HUD viewers.

It looks like we have been trying to download certain things and failing. Could we wrap downloads with retries? cc. @kit1980 

### Versions

CI

cc @seemethere @malfet @pytorch/pytorch-dev-infra
[FSDP] Test that module using mixed precision can be loaded into non-mp module
Labels: triaged, better-engineering, module: fsdp

### 🚀 The feature, motivation and pitch

In FSDP, mixed precision checkpoint is taken with the full parameter precision, but we're missing a test that does this and restores into a non-mixed precision module. Having this test will help provide confidence that use cases which use these such as finetuning / inference work as expected.

### Alternatives

_No response_

### Additional context

_No response_

cc @zhaojuanmao @mrshenli @rohan-varma
Compliance with PEP-0523
Labels: high priority, triage review, oncall: binaries, triaged

### 🚀 The feature, motivation and pitch

Simple PyPI API index which is used to install PyTorch with `pip` does not meet [PEP-0523][1]. Here is an example of installation script.
```shell
pip install \
    --extra-index-url "https://download.pytorch.org/whl/cu102" \
    torch
```
The issue is that simple PyPI API does not redirect to `/`-URL and uses relative paths to wheels URLs. This is what PEP-0523 says.
> All URLs which respond with an HTML5 page MUST end with a / and the repository SHOULD redirect the URLs without a `/` to add a `/` to the end.

Despite the fact PEP-0523 does not requires mandatory redirect, it is expected and usually assumed by users and developers. 

[1]: https://peps.python.org/pep-0503/

### Alternatives

_No response_

### Additional context

_No response_

cc @ezyang @gchanan @zou3519 @seemethere @malfet
quantization: misleading backend config for linear_dynamic_fp16
Labels: oncall: quantization, triaged

### 🐛 Describe the bug

In the current native_backend_config_dict (https://pytorch.org/docs/master/quantization-backend-configuration.html), we have the following entry for linear:

```
{
  'pattern': <class 'torch.nn.modules.linear.Linear'>,
  'dtype_configs': [
    {
      'input_dtype': torch.quint8,
      'weight_dtype': torch.qint8,
      'bias_dtype': torch.float32,
      'output_dtype': torch.quint8,
    },
    {
      'input_dtype': torch.quint8,
      'weight_dtype': torch.qint8,
      'bias_dtype': torch.float32,
      'output_dtype': torch.float32,
      'is_dynamic': True,
    },
    {
      'input_dtype': torch.float16,
      'weight_dtype': torch.float16,
      'bias_dtype': torch.float32,
      'output_dtype': torch.float32,
      'is_dynamic': True,
    },
  ],
  'observation_type': ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT,
  'root_module': <class 'torch.nn.modules.linear.Linear'>,
  'reference_quantized_module_for_root': <class 'torch.nn.quantized._reference.modules.linear.Linear'>,
  'qat_module': <class 'torch.nn.qat.modules.linear.Linear'>,
},
```

This is misleading because the `linear_dynamic_fp16` is only supported for fbgemm and not qnnpack.  Filing this issue to not forget about this.

### Versions

torch version: '1.13.0a0+git3acd9bb'
revision: c6c207f620d01c02f192613c5a231f2581dc3997

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @jgong5 @Xia-Weiwen @leslie-fang-intel @vkuzo
[FX] TypeError when tracing cat taking split's output as input
Labels: triaged, module: fx

### 🐛 Describe the bug

Hi,

Let's say I'd like to feed `cat` with `split`'s output and trace it :

```python
import torch
from torch.fx import symbolic_trace

class M(torch.nn.Module):
    def forward(self, x):
        y = torch.split(x, 4, 2)
        return torch.cat(y, 0)

m = M()
traced = symbolic_trace(m)
print(traced.graph)
```

When running this, I get a `TypeError` because the tracer causes `split` to output a `Proxy` object (as expected, thanks to FX), but `cat` currently needs a sequence of tensors :
```shell
TypeError: cat() received an invalid combination of arguments - got (Proxy, int), but expected one of:
 * (tuple of Tensors tensors, int dim, *, Tensor out)
 * (tuple of Tensors tensors, name dim, *, Tensor out)
```

IMO issue #34294 seems to be closely related to what we see here.

### Versions

PyTorch version: 1.11.0+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: 14.0.0-1ubuntu1
CMake version: version 3.22.1
Libc version: glibc-2.35

Python version: 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.0-37-generic-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.5
[pip3] torch==1.11.0
[pip3] torch-tb-profiler==0.4.0
[pip3] torchdiffeq==0.2.3
[pip3] torchensemble==0.1.7
[pip3] torchtext==0.12.0
[pip3] torchvision==0.12.0
[conda] Could not collect

cc @ezyang @SherlockNoMad
ONEDNN testing is not done properly in quantization codebase
Labels: oncall: quantization, triaged

Update: we have various tests in quantization which assume fbgemm but do not guard for fbgemm properly. We should change

```
@skipIfNoFBGEMM
def test_foo(...):
   ...
```

to 

```
@skipIfNoFBGEMM
def test_foo(...):
    with override_qengines('fbgemm'):
        ...
```


### 🐛 Describe the bug

When I run 

```
python test/test_quantization.py -k Fx
```

on master (revision c6c207f620d01c02f192613c5a231f2581dc3997), there are test failures. There should be no test failures.

Log of failures: https://gist.github.com/vkuzo/d33ce5313b2ac2180064a5a55b9093a2

### Versions

https://gist.github.com/vkuzo/f5683a595a1481aac892e3c300de3acd

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @jgong5 @Xia-Weiwen @leslie-fang-intel @vkuzo
gradgradcheck fails for torch.native_layer_norm
Labels: module: double backwards, module: autograd, triaged

### 🐛 Describe the bug

gradgradcheck fails when `weights=None` and bias is defined for `torch.native_layer_norm`.
```python
import torch
from torch.autograd.gradcheck import gradgradcheck

a = torch.randn(2, requires_grad=True, dtype=torch.float64)
bias = torch.randn(2, requires_grad=True, dtype=torch.float64)

def func(a, bias):
    return torch.native_layer_norm(a, (2,), None, bias, 1e-5)

gradgradcheck(func, [a, bias])
```
```py
GradcheckError: Jacobian mismatch for output 1 with respect to input 2,
numerical:tensor([[1.0000, 0.0000],
        [0.0000, 1.0000]], dtype=torch.float64)
analytical:tensor([[0., 0.],
        [0., 0.]], dtype=torch.float64)
```

### Versions

Fails on Colab (1.11.0+cu113) and master branch.

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
Float and double tensors randomly initialized with the same seed get different values for size >= 16
Labels: triaged, module: random

### 🐛 Describe the bug

Hello, when randomly initializing with the same seed two size-n tensors with dtypes float and double, they get different values for n >= 16. For n <= 15 they are equal as expected.

### Sample code to reproduce the problem
```python
import torch

n = 16
t1 = torch.zeros(n, dtype=torch.float)
t2 = torch.zeros(n, dtype=torch.double)

torch.manual_seed(0)
t1.normal_(0, 1)
torch.manual_seed(0)
t2.normal_(0, 1)

print(t1)
print(t2)
```

### Expected result
`t1` and `t2` should contain the same values.

### Obtained result
`t1` and `t2` have different values for n >= 16. For example with n=16:
```python
tensor([-1.1258, -1.1524, -0.2506, -0.4339,  0.8487,  0.6920, -0.3160, -2.1152,
         0.3223, -1.2633,  0.3500,  0.3081,  0.1198,  1.2377,  1.1168, -0.2473])
tensor([-2.3104, -0.3733, -1.0608,  0.9995, -0.8840, -1.2755, -0.6232, -0.8664,
        -1.2956,  1.5236,  0.3237,  2.0177,  1.1357, -1.2269,  0.0714,  0.3380],
       dtype=torch.float64)
```
`t1` and `t2` have equal values for n <= 15. For example with n=15:
```python
tensor([ 1.5410, -0.2934, -2.1788,  0.5684, -1.0845, -1.3986,  0.4033,  0.8380,
        -0.7193, -0.4033, -0.5966,  0.1820, -0.8567,  1.1006, -1.0712])
tensor([ 1.5410, -0.2934, -2.1788,  0.5684, -1.0845, -1.3986,  0.4033,  0.8380,
        -0.7193, -0.4033, -0.5966,  0.1820, -0.8567,  1.1006, -1.0712],
       dtype=torch.float64)
```

### Versions

```
Collecting environment information...
PyTorch version: 1.11.0.post2
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.3 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: 10.0.0-4ubuntu1 
CMake version: Could not collect
Libc version: glibc-2.31

Python version: 3.10.5 | packaged by conda-forge | (main, Jun 14 2022, 07:06:46) [GCC 10.3.0] (64-bit runtime)
Python platform: Linux-5.14.0-1042-oem-x86_64-with-glibc2.31
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.4
[pip3] torch==1.11.0.post2
[conda] libblas                   3.9.0            15_linux64_mkl    conda-forge
[conda] libcblas                  3.9.0            15_linux64_mkl    conda-forge
[conda] liblapack                 3.9.0            15_linux64_mkl    conda-forge
[conda] mkl                       2022.1.0           h84fe81f_915    conda-forge
[conda] numpy                     1.22.4          py310h4ef5377_0    conda-forge
[conda] pytorch                   1.11.0          cpu_py310h75c9ab6_2    conda-forge
```

cc @pbelevich
Does Torch JIT Support Trace High-level Custom Op?
Labels: oncall: jit

### 🚀 The feature, motivation and pitch

e.g. What I need is something like this:

```py
@jit_trace_atomic_define("my_custom_ops_0")
def my_op(x):
  y = many-complex-custom-func(x)
  return F.relu(y) * 2.5
```

By tracing this operator, I expect to get is
```
%y.5 : Tensor = my_custom_ops_0(%x), scope: __module._xxx # ...
```
instead of **expanding every internal small computation functions** (because serveral of internal functions doesn't support jit tracing)

Is this requirement possible in current pytorch?


### Alternatives

_No response_

### Additional context

_No response_
tensorboard SummaryWriter.add_graph fails when model uses empty tuples
Labels: triaged, module: tensorboard

### 🐛 Describe the bug

I'd like to add the following model to tensorboard, because the model is traceable and can produce a graph that can be printed correctly:
```python
import torch
from torch import nn
from torch.utils.tensorboard import SummaryWriter

@torch.jit.script_if_tracing
def func(x: torch.Tensor):
    return x, ()

class A(nn.Module):
    def forward(self, x):
        return func(x)[0]

model = A()
print(torch.jit.trace(model, torch.rand(3)).graph)
writer = SummaryWriter("output")
writer.add_graph(model, torch.rand(10, 10, 10))
writer.close()
```

However, the `add_graph` method fails to parse the graph due to the empty tuple. It gives the following error:
```
graph(%self : __torch__.A,
      %x : Float(3, strides=[1], requires_grad=0, device=cpu)):
  %4 : Function = prim::Constant[name="func"]()
  %5 : (Tensor, ()) = prim::CallFunction(%4, %x)
  %6 : Tensor, %7 : () = prim::TupleUnpack(%5)
  return (%6)

Traceback (most recent call last):
  File "//a.py", line 20, in <module>
    writer.add_graph(model, torch.rand(10, 10, 10))
  File "/usr/lib/python3.10/site-packages/torch/utils/tensorboard/writer.py", line 736, in add_graph
    self._get_file_writer().add_graph(graph(model, input_to_model, verbose, use_strict_trace))
  File "/usr/lib/python3.10/site-packages/torch/utils/tensorboard/_pytorch_graph.py", line 301, in graph
    list_of_nodes = parse(graph, trace, args)
  File "/usr/lib/python3.10/site-packages/torch/utils/tensorboard/_pytorch_graph.py", line 237, in parse
    nodes_py.append(NodePyOP(node))
  File "/usr/lib/python3.10/site-packages/torch/utils/tensorboard/_pytorch_graph.py", line 93, in __init__
    self.attributes = str({k: node_cpp[k] for k in node_cpp.attributeNames()}).replace("'", ' ')
  File "/usr/lib/python3.10/site-packages/torch/utils/tensorboard/_pytorch_graph.py", line 93, in <dictcomp>
    self.attributes = str({k: node_cpp[k] for k in node_cpp.attributeNames()}).replace("'", ' ')
  File "/usr/lib/python3.10/site-packages/torch/onnx/utils.py", line 1262, in _node_getitem
    return getattr(self, sel)(k)
AttributeError: 'torch._C.Node' object has no attribute 'ival'
```

It looks to me like an issue of tracing - tracing creates a node with missing attributes.

Downstream error report: https://github.com/facebookresearch/detectron2/issues/1607#issuecomment-1093355307 

### Versions

My code was run in `pytorch/pytorch:1.10.0-cuda11.3-cudnn8-runtime` official docker container.