gopikrsmscs/torch-issues huggingface dataset:
[Bug] Circular Import 
Labels: caffe2, triaged

### 🐛 Describe the bug
A Circular Import when importing  caffe2.proto with Anaconda 
```
from matplotlib import pyplot
import numpy as np
import time
# These are the droids you are looking for.
from caffe2.python import core, workspace
from caffe2.proto import caffe2_pb2
# Let's show all plots inline.
%matplotlib inline
```
The Error (In CLI)

```File "D:\Anaconda\lib\site-packages\caffe2\python\__init__.py", line 7, in <module>
    from caffe2.proto import caffe2_pb2
  File "D:\Anaconda\lib\site-packages\caffe2\proto\__init__.py", line 15, in <module>
    from caffe2.proto import caffe2_pb2, metanet_pb2, torch_pb2
ImportError: cannot import name 'metanet_pb2' from partially initialized module 'caffe2.proto' (most likely due to a circular import) (D:\Anaconda\lib\site-packages\caffe2\proto\__init__.py)
````
The error in (Anaconda Jupyter notebook )
```
 try:
---> 15     from caffe2.proto import caffe2_pb2, metanet_pb2, torch_pb2
     16 except ImportError:
     17     warnings.warn('Caffe2 support is not enabled in this PyTorch build. '
     18                   'Please enable Caffe2 by building PyTorch from source with `BUILD_CAFFE2=1` flag.')

ImportError: cannot import name 'metanet_pb2' from partially initialized module 'caffe2.proto' (most likely due to a circular import) (D:\Anaconda\lib\site-packages\caffe2\proto\__init__.py)
```



### Versions

```
PyTorch version: 1.12.1+cpu
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Pro
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.21.3
Libc version: N/A

Python version: 3.9.12 (main, Apr  4 2022, 05:22:27) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.22000-SP0
Is CUDA available: False
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: NVIDIA GeForce GTX 1060
Nvidia driver version: 511.23
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.5
[pip3] numpydoc==1.2
[pip3] torch==1.12.1
[pip3] torchfile==0.1.0
[pip3] torchvision==0.13.1
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.4.0           haa95532_640  
[conda] mkl-service               2.4.0            py39h2bbff1b_0  
[conda] mkl_fft                   1.3.1            py39h277e83a_0  
[conda] mkl_random                1.2.2            py39hf11a4ad_0  
[conda] numpy                     1.21.5           py39h7a0a035_1  
[conda] numpy-base                1.21.5           py39hca35cd5_1  
[conda] numpydoc                  1.2                pyhd3eb1b0_0  
[conda] torch                     1.12.1                   pypi_0    pypi
[conda] torchfile                 0.1.0                    pypi_0    pypi
[conda] torchvision               0.13.1                   pypi_0    pypi

[Done] exited with code=0 in 10.961 seconds
```
Inconsistency between index_select and __get_item__
Labels: triaged, module: advanced indexing

### 🐛 Describe the bug

We should be able to index using `torch.int32` using the bracket notation. It is possible using `index_select`:

```python
>>> idx = torch.tensor([0]).int()
>>> a = torch.rand(3)
>>> a[idx]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: tensors used as indices must be long, byte or bool tensors
>>> a.index_select(0, idx)
tensor([0.6999])
```

When dealing with large sets of indices (e.g. graph edge lists), this means we have to use more memory.

### Versions

master
distributed tests take a long time
Labels: oncall: distributed, module: ci

`pull / linux-bionic-cuda11.6-py3.10-gcc7 / test (distributed, 1, 2, linux.8xlarge.nvidia.gpu)` and `
trunk / linux-bionic-cuda10.2-py3.9-gcc7 / test (distributed, 1, 2, linux.8xlarge.nvidia.gpu)` are our two longest jobs on pull and trunk, respectively, each with a p90 of over 3 hours.  

We are unable to shard these jobs further because `distributed/test_distributed_spawn` is responsible for about half the test time.

Is there anyway we could either break up this file into smaller files that could be put on different shards or make these tests faster?

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501 @seemethere @malfet @pytorch/pytorch-dev-infra
Huggingface Transformers Trainer Test
Labels: triaged, oncall: pt2

Transformers trainer API uses TorchDynamo. As we cleaned up Dynamo, we did not percolate the changes to trainer API, leading to some failures - https://github.com/huggingface/transformers/issues/18127

This is a tracker to improve the situation
1) Better API - Currently, we pass strings and then depending on the strings, we find the backend for Dynamo. Instead, we could just simplify all this, and directly do torchdynamo.optimze(backend_str) - https://github.com/huggingface/transformers/blob/main/src/transformers/trainer.py#L639-L666
2) Remove ctx manager.
3) Decompositions might not be working correctly for relu. I had to switch to cos instead of relu in this PR to see the memory footprint reduction - https://github.com/huggingface/transformers/pull/18685
4) Add a test in Dynamo CI that brings huggingface and tests Dynamo related tests
~~~
pytest tests/trainer/test_trainer.py -k torchdynamo
~~~

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @ydshieh @stas00
quantization: unexpected casting of tensor min and max to int in histogram observer
Labels: oncall: quantization, triaged

### 🐛 Describe the bug

Originally reported in https://discuss.pytorch.org/t/casting-to-int-of-data-min-and-max-in-histogramobserver/159316

There is code in `HistogramObserver` which casts the tensor min and max to integers before calculating the histogram: https://github.com/pytorch/pytorch/blame/a9ba3fe1dbf2cea45c9a7e723010c27c211f7fe3/torch/ao/quantization/observer.py#L1143.  It's unclear on why this is here since we want the histogram bins be as accurate as possible, we should verify if there is a reason for having this and remove it if there isn't a reason.

### Versions

master

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @jgong5 @Xia-Weiwen @leslie-fang-intel @vkuzo
[inductor] Lower aten.cumsum
Labels: triaged, oncall: pt2, module: inductor

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @soumith
[Discussion] Add custom device
Labels: triaged, module: dispatch, module: backend

### 🚀 The feature, motivation and pitch

We are an accelerator vendor and provide Pytorch extensions for our hardware MLU(https://github.com/Cambricon/catch). We would like to add a custom dispatch key and notice that other vendors have submitted their requests like XPU, IPU or NPU(only parts of dispatch keys are accepted), and you recommend to use PrivateUse key instead. We know the limits of adding a new key in Pytorch files but still have some questions.

1. Are all following vendors unable to merge their own custom dispatch key?

2. In some case, PrivateUse key can not meet demands. We think each kind of device may have some special cases to handle like:

https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/TensorShape.cpp#L1305
```
if (!self.is_xla() && !self.is_lazy() && !self.is_ipu()) { 
    return self._reshape_alias(shape, stride.value());
} else {
    return self.view(shape);
}
```

https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/TensorIterator.cpp#L1497
```
if (common_device_.type() == DeviceType::XLA  ||
      common_device_.type() == DeviceType::IPU  ||
      common_device_.type() == DeviceType::Lazy ||
      common_device_.type() == DeviceType::ORT  ||
      common_device_.type() == DeviceType::HPU) return;
```
It is easy to deal with such conditional judgement if our custom key is merged, but how to solve the problem when we do not change Pytorch files and only use PrivateUse key?


### Alternatives

_No response_

### Additional context

_No response_
[feature request] PyTorch vmap for efficient Evolutionary Strategies
Labels: feature, triaged, module: vmap, module: functorch

### 🚀 The feature, motivation and pitch

### Situation
I am currently working on Evolutionary Strategies in PyTorch and I noticed that there is no efficient way to use Modules for defining a Network that I want to train. In [EvoJAX](https://github.com/google/evojax) it is possible to define flax Modules and vmap over their parameters and forward function. In PyTorch it is currently possible to vmap over forward functions, but not over parameters (at least not that I can think of any way).

### Imagine following scenario
You want to train a simple MLP for solving XOR with Evolutionary Strategies in PyTorch.

Writing a custom forward method, this can be done very efficiently using einsums:
c: parameters of children in current population with shape [npop, params]
x: batched input: `torch.tensor([[0., 0.], [0., 1.], [1., 0.], [1., 1.]])`

```python
@torch.jit.script
def forward(c, x):
    # linear1 (in: 2, out: 2)
    x = torch.einsum('bi,nio->bno', x, c[..., :4].view(-1, 2, 2))
    x = x + c[..., 4:6]

    # relu
    x = F.relu(x)

    # linear2 (in: 2, out: 1)
    x = torch.einsum('bni,nio->bno', x, c[..., 6:8].view(-1, 2, 1))
    x = x + c[..., 8:9]

    return x
```

Now I would like to simply define my Network as nn.Module and  vmap over its parameters and forward function like this:

```python
def f(model, c, x):
   torch.nn.utils.vector_to_parameters(c, model.parameters())
   return model(x)

outputs = vmap(f, in_dims=(None, 0, None))(model, child_params, X)
```
This is currently not possible, because parameter assignment is weird in PyTorch.


### Alternatives

_No response_

### Additional context

_No response_

cc @zou3519 @Chillee @samdow
Unhelpful error message from torch.linalg.ldl_factor
Labels: triaged, module: linear algebra, module: edge cases

### 🐛 Describe the bug

When performing an ldl_factor on a matrix of zeros (or entries smaller than some epsilon value), I get the following error:
RuntimeError: false INTERNAL ASSERT FAILED at ```../aten/src/ATen/native/BatchLinearAlgebra.cpp":1504, please report a bug to PyTorch. torch.linalg.ldl_factor: Unknown error code: 1.```


Here is a simple example:

import torch
ld,pivot=torch.linalg.ld_factor(torch.zeros(5,5)) 

I wouldn't expect this to work, of course, because L is undefined in this case.  But perhaps it could do a saner exception check before (instead of) passing the input to LAPACK.  Incidentally, not sure why it's giving me a positive error code as I believe this conflicts with the LAPACK documentation...

### Versions

[pip3] numpy==1.21.5
[pip3] pytorch-lightning==1.7.0
[pip3] pytorch-wavelets==1.3.0
[pip3] rotary-embedding-torch==0.1.5
[pip3] torch==1.12.0
[pip3] torch-geometric==2.0.4
[pip3] torchfile==0.1.0
[pip3] torchinfo==1.7.0
[pip3] torchmetrics==0.9.3
[pip3] torchvision==0.13.0
[conda] numpy                     1.21.5                   pypi_0    pypi
[conda] pytorch-lightning         1.7.0                    pypi_0    pypi
[conda] pytorch-wavelets          1.3.0                    pypi_0    pypi
[conda] rotary-embedding-torch    0.1.5                    pypi_0    pypi
[conda] torch                     1.12.0                   pypi_0    pypi
[conda] torch-geometric           2.0.4                    pypi_0    pypi
[conda] torchfile                 0.1.0                    pypi_0    pypi
[conda] torchinfo                 1.7.0                    pypi_0    pypi
[conda] torchmetrics              0.9.3                    pypi_0    pypi
[conda] torchvision               0.13.0                   pypi_0    pypi
I am using the nightly build and running on the CPU.

cc @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano
test_profiler_experimental_tree_cuda_detailed is too unstable, and as its CUDA only difficult to regen
Labels: high priority, triage review, module: tests, triaged, oncall: profiler

### 🐛 Describe the bug

This test is a double whammy:

* It's an expect test, but for piles of internal data which can wobble on unrelated changes for no good reason...
* and it's a CUDA only test, which means that if you have a failure on a CPU build, you have no way of easily regenerating the output

This test has had to be repeatedly updated for unrelated changes multiple times in its lifetime. I am going to disable it for now and we should discuss how exactly this test should actually be written.

cc @ezyang @gchanan @zou3519 @mruberry @robieta @chaekit @aaronenyeshi @ngimel @nbcsm @guotuofeng @guyang3532 @gaoteng-git @tiffzhaofb 

### Versions

master
argmax/argmin returns the last index instead of the first when there are equally max/min elements
Labels: triaged, oncall: pt2

Here is the result when calling `a.argmax(0)`
```
a = torch.tensor([[3, 2, 2, 3, 2, 4, 3, 1, 3, 1],
        [2, 3, 4, 2, 1, 2, 4, 3, 1, 2],
        [3, 4, 3, 4, 3, 4, 1, 2, 1, 3],
        [4, 1, 2, 1, 1, 3, 1, 3, 2, 2],
        [1, 2, 4, 4, 3, 3, 2, 1, 1, 1],
        [1, 1, 4, 1, 4, 4, 2, 1, 1, 2],
        [3, 2, 3, 2, 1, 4, 2, 4, 2, 1],
        [4, 3, 4, 4, 1, 1, 2, 4, 3, 3],
        [2, 3, 4, 1, 2, 4, 4, 4, 4, 1],
        [1, 1, 3, 4, 1, 1, 1, 3, 3, 3]], device='cuda:0', dtype=torch.int32)
expected = tensor([3, 2, 1, 2, 5, 0, 1, 6, 8, 2], device='cuda:0')
actual   = tensor([3, 2, 8, 4, 5, 0, 8, 8, 8, 2], device='cuda:0')
```

Here is the generated triton kernel,
```
from ctypes import c_void_p, c_long
import torch
import random
from torch import empty_strided, as_strided
from torchinductor.codecache import CppCodeCache, TritonCodeCache

aten = torch.ops.aten

import triton
import triton.language as tl

from torchinductor.triton_ops.autotune import pointwise_heuristics
from torchinductor.triton_ops.autotune import reduction_heuristics
from torchinductor.triton_ops.autotune import grid

@reduction_heuristics(size_hints=[16, 16])
@triton.jit
def kernel0(in_ptr0, out_ptr0, ks0, xnumel, rnumel, XBLOCK : tl.constexpr, RBLOCK : tl.constexpr):
    xoffset = tl.program_id(0) * XBLOCK
    xindex = xoffset + tl.reshape(tl.arange(0, XBLOCK), [XBLOCK, 1])
    xmask = xindex < xnumel
    rbase = tl.reshape(tl.arange(0, RBLOCK), [1, RBLOCK])
    x0 = xindex
    _tmp1 = tl.zeros([XBLOCK, RBLOCK], tl.int64) + -2147483648
    _tmp1_index = tl.zeros([XBLOCK, RBLOCK], tl.int64)
    for roffset in range(0, rnumel, RBLOCK):
        rindex = roffset + rbase
        rmask = rindex < rnumel
        r1 = rindex
        tmp0 = tl.load(in_ptr0 + x0 + (ks0*r1), xmask & rmask, eviction_policy='evict_last')
        _tmp1_index = tl.where(xmask & rmask & (_tmp1 < tmp0),  rindex, _tmp1_index)
        _tmp1 = tl.where(xmask & rmask & (_tmp1 < tmp0), tmp0, _tmp1)
    _tmp1_index_reduce = tl.reshape(
    	tl.argmax(_tmp1, 1), [XBLOCK, 1]).to(tl.int32)
    _tmp1_index_mask = (tl.reshape(tl.arange(0, RBLOCK),
    	[1, RBLOCK]) == _tmp1_index_reduce)
    tmp1 = tl.reshape(tl.sum(
    	tl.where(_tmp1_index_mask, _tmp1_index, 0), 1), [XBLOCK, 1])
    tl.store(out_ptr0 + x0, tmp1, xmask)


def call(arg0_1):
    arg0_1_size = arg0_1.size()
    s0 = arg0_1_size[0]
    buf0 = empty_strided((s0, ), (1, ), device='cuda', dtype=torch.int64)
    kernel0[grid(s0)](arg0_1, buf0, s0, s0, s0)
    return (buf0, )


if __name__ == "__main__":
    from torchdynamo.testing import rand_strided
    from torchinductor.utils import print_performance
    arg0_1 = rand_strided((10, 10), (10, 1), device='cuda', dtype=torch.int64)
    call(arg0_1)
```

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
Speedup for adding images to tensorboard
Labels: oncall: visualization

### 🚀 The feature, motivation and pitch

This is my first contribution to pytorch, i will try to follow all protocols but please tell me if i am doing something wrong.
### The motivation
Adding very big matplotlib figures is slow due to unnecessary casting and scaling, this can be sped up significantly. 

 ### The concrete problem
I create big figures using matplotlib in a training callback after each epoch. I would like to add these figures to tensorboard. The add_figure method currently basically renders the image as a figure and then adds it as an image. 

https://github.com/pytorch/pytorch/blob/822a8e057fa4e6a6a8413d22bae2c1a5aa853134/torch/utils/tensorboard/writer.py#L748-L754

The method add_image creates a serialized summary and adds it to the file writer. 
https://github.com/pytorch/pytorch/blob/822a8e057fa4e6a6a8413d22bae2c1a5aa853134/torch/utils/tensorboard/writer.py#L614-L616

The creation of the image summary has these lines, where the image is casted to float, multiplied by a scale factor (either 255 or 1) and casted back to uint8. In my case the "tensor" is already an ndarray with type uint8, so the cast to float, multiplication and cast back to uint8 is unnecessary. 
https://github.com/pytorch/pytorch/blob/822a8e057fa4e6a6a8413d22bae2c1a5aa853134/torch/utils/tensorboard/summary.py#L443-L446


I'm very confident that these lines cause my speed problem . I localized them via printf debugging, then inserted condition (as shown below) and now it works much faster.


### Proposed solution 
Add a condition whether the cast-scale-cast-back is necessary, something to the effect of: 
```python
if tensor.dtype != np.uint8 or scale_factor != 1.0:
        tensor = tensor.astype(np.float32)
        tensor = (tensor * scale_factor).astype(np.uint8)
```





### Alternatives
I have considered ignoring the problem since it is only on epoch end anyway, but especially in debugging it can be very annoying when adding an image to tensorboard takes that long.

### Additional context

_No response_
Segfault when profiling with_stack=True on model with jit.optimize_for_inference
Labels: oncall: jit

### 🐛 Describe the bug

The Python process segfaults whenever I run the pytorch profiler using with_stack=True on a model that has had torch.jit.optimize_for_inference() called on it. 

```python
import torch
import torchvision.models as models
from torch.profiler import profile, record_function, ProfilerActivity
model = models.resnet18().cuda()
inputs = torch.randn(5, 3, 224, 224).cuda()
model = torch.jit.script(model)
model = torch.jit.optimize_for_inference(model)

with profile(activities=[ProfilerActivity.CPU], with_stack=True) as prof:
    model(inputs)
```

```
Segmentation fault (core dumped)
```

The .crash file in /var/crash is 605MB and this should be easy to reproduce, so I haven't attached that, but let me know if there's anything else you need.

### Versions

Collecting environment information...
PyTorch version: 1.12.1+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.1 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-10ubuntu2) 9.3.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.8.5 (default, Jul 28 2020, 12:59:40)  [GCC 9.3.0] (64-bit runtime)
Python platform: Linux-5.4.0-47-generic-x86_64-with-glibc2.29
Is CUDA available: True
CUDA runtime version: 10.2.89
GPU models and configuration:
GPU 0: NVIDIA GeForce RTX 2080 Ti
GPU 1: NVIDIA GeForce RTX 2080 Ti
GPU 2: NVIDIA GeForce RTX 2080 Ti
GPU 3: NVIDIA GeForce RTX 2080 Ti
GPU 4: NVIDIA GeForce RTX 2080 Ti
GPU 5: NVIDIA GeForce RTX 2080 Ti
GPU 6: NVIDIA GeForce RTX 2080 Ti
GPU 7: NVIDIA GeForce RTX 2080 Ti

Nvidia driver version: 510.73.05
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.0.4
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.2
[pip3] torch==1.12.1
[pip3] torchvision==0.13.1
[conda] Could not collect
Profiler can only print first 5 entries in stack traces because of hard-coded limit
Labels: oncall: profiler

### 🐛 Describe the bug

The pytorch profiler's key_averages() takes a "group_by_n" parameter with no documented upper limit. However, it only works up to 5. Printing with something like 
```python
import torch
import torchvision.models as models
from torch.profiler import profile, record_function, ProfilerActivity
model = models.resnet18().cuda()
inputs = torch.randn(5, 3, 224, 224).cuda()

with profile(activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA], with_stack=True) as prof:
    model(inputs)
print(prof.key_averages(group_by_stack_n=8).table(sort_by="self_cuda_time_total", row_limit=2))
```
only prints 5 lines of each stack context, instead of the expected up-to-8.

This seems to be due to the hard-coded line
```
MAX_STACK_ENTRY=5
```
in the _build_table function in torch.autograd.profiler_util.py  .

If I understand correctly, I think this hard-coded limit could just be removed? Since the relevant stack depth is typically limited by the group_by_stack_n parameter anyway.

This came up in practice because I'm using some of the deeply-nested faster-RCNN modules in torchvision.



### Versions

Collecting environment information...
PyTorch version: 1.12.1+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.1 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-10ubuntu2) 9.3.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.8.5 (default, Jul 28 2020, 12:59:40)  [GCC 9.3.0] (64-bit runtime)
Python platform: Linux-5.4.0-47-generic-x86_64-with-glibc2.29
Is CUDA available: True
CUDA runtime version: 10.2.89
GPU models and configuration:
GPU 0: NVIDIA GeForce RTX 2080 Ti
GPU 1: NVIDIA GeForce RTX 2080 Ti
GPU 2: NVIDIA GeForce RTX 2080 Ti
GPU 3: NVIDIA GeForce RTX 2080 Ti
GPU 4: NVIDIA GeForce RTX 2080 Ti
GPU 5: NVIDIA GeForce RTX 2080 Ti
GPU 6: NVIDIA GeForce RTX 2080 Ti
GPU 7: NVIDIA GeForce RTX 2080 Ti

Nvidia driver version: 510.73.05
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.0.4
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.0.4
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.2
[pip3] torch==1.12.1
[pip3] torchvision==0.13.1
[conda] Could not collect


cc @robieta @chaekit @aaronenyeshi @ngimel @nbcsm @guotuofeng @guyang3532 @gaoteng-git @tiffzhaofb
Silent promotion of bool to int in the dispatcher
Labels: triaged, module: type promotion, module: pybind, module: library

### 🐛 Describe the bug

I try to register a function that according to its signature should only accept int or a list of ints, but there's a silent bool promotion to integers:
```py
import torch

test = torch.library.Library("test", "DEF")
test_impl = torch.library.Library("test", "IMPL", "CompositeExplicitAutograd")

schema = "test(int[2] arg) -> Tensor"

test.define(schema)
test_impl.impl("test", lambda arg=None: torch.tensor(arg))

try:
    res = torch.ops.test.test(arg=True)
    print(res)
    print("Unexpected success!")
except RuntimeError as e:
    print(e)

# This doesn't work as expected:
try:
    res = torch.ops.test.test(arg=2.0)
    print(res)
except RuntimeError as e:
    print(e)
```

### Versions

Latest master.

cc: @ezyang, @albanD, @bdhirsh  

cc @nairbv @mruberry @anjali411
KL-divergence of two Generalized Dirichlet distributions
Labels: module: distributions, feature, triaged

### 🚀 The feature, motivation and pitch

Thank you for your great contribution. Could you please add KL-divergence of two Generalized Dirichlet distributions? 
A very clear of its derivation can be found in equation 11 of this paper: [https://www.sciencedirect.com/science/article/abs/pii/S003132031830311X?via%3Dihub]
[KL-Divergence.pdf](https://github.com/pytorch/pytorch/files/9355461/KL-Divergence.pdf)
.

### Alternatives

_No response_

### Additional context

_No response_

cc @fritzo @neerajprad @alicanb @nikitaved
OpenJDK libtorch_cpu.so stack guard warning
Labels: oncall: java

### 🚀 The feature, motivation and pitch

When deploying a pytorch model using JavaCPP presets for pytorch + libtorch 1.10.2, java `17.0.3+7-Ubuntu-0ubuntu0.20.04.1` emits the following warning: 
```
OpenJDK 64-Bit Server VM warning: You have loaded library /mnt/lib/cache/org.bytedeco.pytorch-1.10.2-1.5.7-linux-x86_64.jar/org/bytedeco/pytorch/linux-x86_64/libtorch_cpu.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
```
I reported this in https://github.com/bytedeco/javacpp-presets/issues/1208 but it's apparently caused by the way libtorch is built, not by JavaCPP. 

Shouldn't pytorch be built with stack guard? 

The answer may well be 'no', but it would be great to understand better 'why' in that case. Thanks!

### Alternatives

Ignore the warning. 

### Additional context

https://stackoverflow.com/questions/1629685/when-and-how-to-use-gccs-stack-protection-feature
PyTorch test suite regression test_module_backward_global_hook_writeable
Labels: triaged, oncall: pt2

Bisected to https://github.com/pytorch/torchdynamo/pull/763

Repro command

```
PYTORCH_TEST_WITH_DYNAMO=1 python test/test_nn.py -k test_module_backward_global_hook_writeable
```

error

```
======================================================================                               
FAIL: test_module_backward_global_hook_writeable (__main__.TestModuleGlobalHooks)
----------------------------------------------------------------------                             
Traceback (most recent call last):                                                                   
  File "test/test_nn.py", line 21073, in test_module_backward_global_hook_writeable                  
    def test_module_backward_global_hook_writeable(self):                                            
  File "test/test_nn.py", line 21073, in test_module_backward_global_hook_writeable
    def test_module_backward_global_hook_writeable(self):                                          
  File "test/test_nn.py", line 21073, in test_module_backward_global_hook_writeable
    def test_module_backward_global_hook_writeable(self):             
  [Previous line repeated 1 more time]
  File "/data/users/ezyang/pytorch-tmp/torch/testing/_internal/common_utils.py", line 2382, in assert
Equal
    assert_equal(                                                                                    
  File "/data/users/ezyang/pytorch-tmp/torch/testing/_comparison.py", line 1093, in assert_equal
    raise error_metas[0].to_error(msg)
AssertionError: Tensor-likes are not close!

Mismatched elements: 25 / 25 (100.0%)
Greatest absolute difference: 0.24967582497447965 at index (4, 2) (up to 1e-07 allowed)
Greatest relative difference: 0.5 at index (0, 0) (up to 1e-07 allowed)

----------------------------------------------------------------------
```

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh
I have the same issue as @samgelman on my MacOS.
Labels: triaged, module: macos, module: openmp, module: third_party

I have the same issue as @samgelman on my MacOS. 

Notably 1.10.1, 1.11.0 works fine but 1.10.2 and 1.12.1 shows:
```
OMP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.
Abort trap: 6
```
One observation is that both 1.10.1 and 1.11.0 also come with `libuv` installation, which might have made the difference.

The workaround `KMP_DUPLICATE_LIB_OK=TRUE` causes a segfault.

Below is a working env:
```
python3 -mtorch.utils.collect_env
Collecting environment information...
PyTorch version: 1.10.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.5 (x86_64)
GCC version: Could not collect
Clang version: 12.0.0 (clang-1200.0.31.1)
CMake version: Could not collect
Libc version: N/A

Python version: 3.8.13 (default, Mar 28 2022, 06:16:26)  [Clang 12.0.0 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.5
[pip3] torch==1.10.1
[conda] blas                      1.0                         mkl  
[conda] mkl                       2021.4.0           hecd8cb5_637  
[conda] mkl-service               2.4.0            py38h9ed2024_0  
[conda] mkl_fft                   1.3.1            py38h4ab4a9b_0  
[conda] mkl_random                1.2.2            py38hb2f4e1b_0  
[conda] mypy_extensions           0.4.3            py38hecd8cb5_1  
[conda] numpy                     1.21.5           py38h2e5f0a9_3  
[conda] numpy-base                1.21.5           py38h3b1a694_3  
[conda] pytorch                   1.10.1                  py3.8_0    pytorch
```

_Originally posted by @yuanmao in https://github.com/pytorch/pytorch/issues/78490#issuecomment-1217134036_

cc @malfet @albanD
Add a new argument `check_inf=True` (by default) or check_pos_inf / check_neg_inf to anomaly mode
Labels: module: autograd, triaged, enhancement

should if also have a new argument `check_inf=True` (by default) or check_pos_inf / check_neg_inf for symmetry with nan? (minor)

_Originally posted by @vadimkantorov in https://github.com/pytorch/pytorch/issues/83481#issuecomment-1215925856_

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
Adding Levenberg-marquardt optimizer in PyTorch
Labels: feature, module: optimizer, triaged, needs research

### 🚀 The feature, motivation and pitch

### Feature
Adding levenberg-marquardt (LM) algorithm to Torch.Optim. 

### Motivation
Levenberg-Marquardt (LM), also known as damped least squares, is used to solve non-linear least squares. It converges much faster than gradient descent based methods (orders of magnitude faster). The algorithm needs access to the Jacobian matrix which makes it not suitable for problems with huge data. However, for many applications, like optical design, non-convex optimization, imaging systems design, the size of the network (or model) and data is manageable that LM algorithm provides much better solutions in a fraction of the time (seconds instead of hours). It will be extremely helpful to implement that in torch.optim.

### Alternatives

Two alternatives exist
1.  I can try implementing it myself by calculating the Jacobian. However, the functions that PyTorch provides to do that are extremely slow. 
2. Another method that I routinely use is a recently published paper called: "DeepLM: Large-scale Nonlinear Least Squares on Deep Learning Frameworks using Stochastic Domain Decomposition". This paper and GitHub uses PyTorch and provides an approximate way to implement LM algorithm. However, the structure of their optimizer is very different than the familiar torch.optim and the code needs to be changed significantly to use this paper. 


cc @vincentqb @jbschlosser @albanD
quantize_per_tensor/quantize_per_channel operators should honor the quant_min/quant_max from observer
Labels: oncall: quantization, triaged

### 🐛 Describe the bug

quantize_per_tensor/quantize_per_channel are operators that converts a floating point Tensor to a quantized (int8) Tensor based on some quantization parameters, the formula is:
```
quantized_Tensor = clamp(fp_Tensor / scale + zero_point, quant_min, quant_max)
```
Currently, since we don't take `quant_min`/`quant_max` as arguments, e.g. signature for `quantize_per_tensor` is 
```
quantize_per_tensor(Tensor self, float scale, int zero_point, ScalarType dtype) -> Tensor
```
the default quant_min/quant_max for the dtype is applied right now. but ideally we should take `quant_min`/`quant_max` from observers.

Currently there is a workaround in https://github.com/pytorch/pytorch/pull/83438 to clamp the weight in the utility function to get the quantized weight, which is shared by both eager mode quantization and fx graph mode quantization. But ideally we should add `quant_min`/`quant_max` arguments to these operators.



### Versions

master

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo
Cdist backward dependent on compute_mode
Labels: module: autograd, triaged, actionable

### 🐛 Describe the bug

The function [torch.cdist](https://pytorch.org/docs/stable/generated/torch.cdist.html) raises a `NotImplementedError: the derivative for '_cdist_backward' is not implemented.` for default `compute_mode` during nested gradient calculation. 


Example:

```
import torch


def function(a, b):
    return torch.cdist(a, b)
    # return torch.cdist(a, b, compute_mode="use_mm_for_euclid_dist")  # works


# input data
a = torch.tensor([[1.0, 2.0], [3.0, 4.0]], requires_grad=True)
b = torch.tensor([[5.0, 6.0], [7.0, 8.0]])

# calculate forward pass
y = torch.nn.Linear(2, 1)(function(a, b))

# calculate gradient
gradient = torch.autograd.grad(
    y,
    a,
    grad_outputs=torch.ones_like(y),
    create_graph=True,
)[0]

# calc loss to reference
grad_ref = torch.tensor([[10.0, 11.0], [12.0, 13.0]])
loss = torch.nn.MSELoss(reduction="mean")(gradient, grad_ref)

loss.backward()  # NotImplementedError: the derivative for '_cdist_backward' is not implemented.
```

To us, this behavior is unclear from the documentation.
If this is not the supposed way to implement gradient based loss calculation, we are looking forward to your suggestions on alternative setups. 

Hence, suggestions on how to circumference this are welcome (aside from changing `compute_mode`).

If this were to be expected, perhaps the documentation could be improved?

### Versions

PyTorch version: 1.12.0

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
Build and Run QNNPACK on X86
Labels: module: build, triaged

### 🚀 The feature, motivation and pitch

Hi Pytorch team,

I want to build and run QNNPACK on X86 CPU and there are not many references to do the same. It would be great if I can get the references to build and execute QNNPACK on the X86 platform.

Thank you!






### Alternatives

_No response_

### Additional context

_No response_

cc @malfet @seemethere @dhruvbird @ljk53
`torch.pinverse` produces wrong output!
Labels: module: docs, triaged, module: linear algebra

### 🐛 Describe the bug

`torch.pinverse` produces wrong output for a 3*3 tensor! 
According to the [documentation](https://pytorch.org/docs/stable/generated/torch.pinverse.html), `torch.pinverse` is an alias for [`torch.linalg.pinv()`](https://pytorch.org/docs/stable/generated/torch.linalg.pinv.html#torch.linalg.pinv), and computes the pseudoinverse (Moore-Penrose inverse) of a matrix. Algebraically, the output of `torch.pinverse(A)`, denoted as `Apinv`, should satisfy:
`A  Apinv A = A`
However, for this input `A`, `torch.pinverse(A)` fails to generate correct result:
```
import torch
A  = torch.tensor([[0.0, 1.0, -1.0], [1.0, -1.0, 0.0], [-1.0, 1.0, 0.0]])
Apinv = torch.pinverse(A)
print(A @ Apinv @ A)
print(torch.dist(A @ Apinv @ A, A))
```
Outputs:
```
tensor([[ 0.0000e+00,  1.0000e+00, -1.0000e+00],
        [ 0.0000e+00, -2.9802e-08,  2.9802e-08],
        [ 0.0000e+00,  2.9802e-08, -2.9802e-08]])
tensor(2.)
```
On the other hand, `torch.linalg.pinv` and `numpy.linalg.pinv` can generate correct pseudo inverse given the same input matrix `A`.
```
import torch
A  = torch.tensor([[0.0, 1.0, -1.0], [1.0, -1.0, 0.0], [-1.0, 1.0, 0.0]])
Apinv2 = torch.linalg.pinv(A)
print(A @ Apinv2 @ A)
print(torch.dist(A @ Apinv2 @ A, A))


import numpy as np
Apinv_np = np.linalg.pinv(A)
print(A @ Apinv_np @ A)
print(torch.dist(A @ Apinv_np @ A, A))
```
Outputs
```
tensor([[-1.4901e-07,  1.0000e+00, -1.0000e+00],
        [ 1.0000e+00, -1.0000e+00,  2.9802e-08],
        [-1.0000e+00,  1.0000e+00, -2.9802e-08]])
tensor(5.5516e-07)
tensor([[ 8.9407e-08,  1.0000e+00, -1.0000e+00],
        [ 1.0000e+00, -1.0000e+00, -2.9802e-08],
        [-1.0000e+00,  1.0000e+00,  2.9802e-08]])
tensor(2.8430e-07)
```

### Versions

torch 1.12.1

cc @svekars @holly1238 @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano
`Frozen` module for transfer learning.
Labels: module: nn, triaged, needs design

### 🚀 The feature, motivation and pitch

"Freezing" a model to stop gradient propagation and parameter updates is a common step in transfer learning. The [tutorial on transfer learning](https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html#convnet-as-fixed-feature-extractor) currently suggests setting `requires_grad = False` for all parameters of the feature extractor and removing the parameters from the set of parameters passed to the optimizer. What would be your thoughts on adding a `Frozen` module that achieves the same task as a one-liner? Proposed implementation below.

```python
class Frozen(torch.nn.Module):
    def __init__(self, frozen_module):
        super().__init__()
        self.frozen_module = frozen_module

    def forward(self, *args, **kwargs):
        with torch.no_grad():
            return self.frozen_module(*args, **kwargs)

    def parameters(self, recurse: bool = True) -> typing.Iterator[th.nn.Parameter]:
        return
        yield

    def named_parameters(self, prefix: str = '', recurse: bool = True) \
            -> typing.Iterator[tuple[str, th.nn.Parameter]]:
        return
        yield
```

Usage example: 

```python
resnet18 = torchvision.models.resnet18(pretrained=True)
extractor = Frozen(resnet18)  # Stops all gradients and parameter lookups.
```

As a side-effect, this would allow gradients to propagate if there are multiple paths from loss to parameters, e.g., in multi-objective optimization. Calling `extractor` would not propagate gradients. Calling `resnet18` directly would propagate gradients.

### Alternatives

_No response_

### Additional context

_No response_

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
Move TorchInductor Triton autotuner to compile time, use common cache
Labels: triaged, oncall: pt2

We use the Triton autotuner here:
https://github.com/pytorch/torchdynamo/blob/8e4abe3f66fb6b8d97d6461d64f9c3be323a523e/torchinductor/triton_ops/autotune.py#L182
and, here:
https://github.com/pytorch/torchdynamo/blob/8e4abe3f66fb6b8d97d6461d64f9c3be323a523e/torchinductor/triton_ops/autotune.py#L216
To select tile sizes for generated kernels from a short list of options.  

This results in good performance, but has a high cost in terms of warm up time.  We will be running nearly identical autotuning runs to pick tilings that should be obvious.

I'd propose we create a small cost model to replace the autotuner.  As an input to this model we should take:
1) The size_hints (already rounded up to next power of two)
2) The indexing formulas of all the reads and writes
3) [Maybe] Number of `tmpX` variables used in kernel
4) [Maybe] Number of reduction loops in the kernel
5) [Maybe] Histogram of `ops.*` usage in the kernel

As an initial implementation, we could just build an on-disk cache/hash-table that runs the Triton autotuner on misses, but reuses a prior decision if the model inputs are identical.  This alone should dramatically speed up warmup times.

Longer term, we could take the data collected in this cache and train a small neural network to predict which tiling to use.  For this we should have the neural network predict the normalized runtime for each tiling choice.  This cost model can then replace the measurement step of the autotuner.






cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
Test public bindings in CI gives weird output on error
Labels: high priority, module: ci, module: tests, triaged, module: python frontend

### 🐛 Describe the bug

https://github.com/pytorch/pytorch/runs/7824028166?check_suite_focus=true

There is a bunch of irrelevant failures. The real failure is reproducible locally and tells me only that there are some public API in symbolic_shapes that are not exported, but the CI output is useless

### Versions

master

cc @gchanan @zou3519 @seemethere @malfet @pytorch/pytorch-dev-infra @mruberry
How to turn off determinism just for specific operations, e.g. upsampling through bilinear interpolation?
Labels: module: cuda, triaged, module: determinism

This is the error caused by upsampling through bilinear interpolation when trying to use deterministic algorithms:

`RuntimeError: upsample_bilinear2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True)'. You can turn off determinism just for this operation if that's acceptable for your application. You can also file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation.`

How to turn off determinism just for upsampling_bilinear2d (and any other operation)? Thanks!

cc @ngimel @mruberry @kurtamohler
PyTorch profiler is spammy
Labels: oncall: profiler

Running the TorchDynamo benchmarking scripts in coverage mode results in:
```
~/torchdynamo$ ./benchmarks/torchbench.py --isolate
cpu  eval  BERT_pytorch                       STAGE:2022-08-13 17:41:32 999867:999867 ActivityProfilerController.cpp:294] Completed Stage: Warm Up
STAGE:2022-08-13 17:41:32 999867:999867 ActivityProfilerController.cpp:300] Completed Stage: Collection
  4/  4 +0 frames   3s  1 graphs  1 graph calls  565/ 565 = 100% ops 100% time
cpu  eval  Background_Matting                 STAGE:2022-08-13 17:41:49 999993:999993 ActivityProfilerController.cpp:294] Completed Stage: Warm Up
STAGE:2022-08-13 17:41:51 999993:999993 ActivityProfilerController.cpp:300] Completed Stage: Collection
  1/  1 +0 frames   7s  1 graphs  1 graph calls  184/ 184 = 100% ops 100% time
cpu  eval  LearningToPaint                    STAGE:2022-08-13 17:41:58 1000068:1000068 ActivityProfilerController.cpp:294] Completed Stage: Warm Up
STAGE:2022-08-13 17:41:58 1000068:1000068 ActivityProfilerController.cpp:300] Completed Stage: Collection
  1/  1 +0 frames   1s  1 graphs  1 graph calls   72/  72 = 100% ops 100% time
cpu  eval  Super_SloMo                        STAGE:2022-08-13 17:42:23 1000105:1000105 ActivityProfilerController.cpp:294] Completed Stage: Warm Up
STAGE:2022-08-13 17:42:27 1000105:1000105 ActivityProfilerController.cpp:300] Completed Stage: Collection
  1/  1 +0 frames  15s  1 graphs  1 graph calls  541/ 541 = 100% ops 100% time
cpu  eval  alexnet                            STAGE:2022-08-13 17:42:32 1000208:1000208 ActivityProfilerController.cpp:294] Completed Stage: Warm Up
STAGE:2022-08-13 17:42:32 1000208:1000208 ActivityProfilerController.cpp:300] Completed Stage: Collection
  1/  1 +0 frames   1s  1 graphs  1 graph calls   23/  23 = 100% ops 100% time
cpu  eval  attention_is_all_you_need_pytorch  STAGE:2022-08-13 17:42:38 1000238:1000238 ActivityProfilerController.cpp:294] Completed Stage: Warm Up
STAGE:2022-08-13 17:42:38 1000238:1000238 ActivityProfilerController.cpp:300] Completed Stage: Collection
...
```

All of the `ActivityProfilerController.cpp` logging makes it hard to read the actual output of the script.

IMO we should disable this logging by default.


cc @robieta @chaekit @aaronenyeshi @ngimel @nbcsm @guotuofeng @guyang3532 @gaoteng-git @tiffzhaofb
[caffee2] Windows build / 'metanet_pb2' (a circular import)  Anaconda
Labels: caffe2

### 🐛 Describe the bug

I want to use [Detectron](https://github.com/facebookresearch/Detectron) for an old project .
 I have Anaconda ```conda 4.13.0```
 

```
UserWarning: Caffe2 support is not enabled in this PyTorch build. Please enable Caffe2 by building PyTorch from source with `BUILD_CAFFE2=1` flag.
  warnings.warn('Caffe2 support is not enabled in this PyTorch build. 
```

can't build PyTorch from source for various reasons 
I think that has nothing to do with the issue  
```
  from caffe2.proto import caffe2_pb2, metanet_pb2, torch_pb2
ImportError: cannot import name 'metanet_pb2' from partially initialized module 'caffe2.proto' (most likely due to a circular import) 
```

  I edit the script located in D:\Anaconda\lib\site-packages\caffe2\proto\__init__.py
  and run it then change it back to its original code and now somehow it doesn't install or output anything !!
  
 So it seems that Caffee2 is integrated into Pytorch and the API is changed .
 I need to run the script of which use Detectron and old api of caffee2 
 
when I do 
```
python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"
```
it fails 
 ```
ModuleNotFoundError: No module named 'detectron' 
 ```

to put it simply I need to run this Project with Anaconda --

https://github.com/krematas/soccerontable

### Versions

I can't run this script
Complex-Valued Gaussian distributions
Labels: module: distributions, triaged, module: complex, module: random

### 🚀 The feature, motivation and pitch

I had been working on complex-valued variational auto encoders (CVAE) and for this purpose I required complex-valued Gaussian distributions. I coded my own implementation based on [1] and used it successfully. I essentially model it as a composite real-valued distribution. Would it be of interest if I created a PR to add this functionality?

[1] P. J. Schreier and L. L. Scharf, Statistical signal processing of complex-valued data: the theory of improper and noncircular signals. Cambridge: Cambridge University Press, 2010. Accessed: Feb. 13, 2022. [Online]. Available: https://doi.org/10.1017/CBO9780511815911


### Alternatives

_No response_

### Additional context

_No response_

cc @fritzo @neerajprad @alicanb @nikitaved @ezyang @anjali411 @dylanbespalko @mruberry @Lezcano @pbelevich