gopikrsmscs/torch-issues huggingface dataset:
[Modes] no_dispatch is not the same as DisableTorchFunction, causing differences in modes
Labels: triaged, module: __torch_function__, module: __torch_dispatch__

### 🐛 Describe the bug

## Repro
The following passes
```
      def test_disable_subclass_not_mode(self):
          called = False
            
          class A(TorchFunctionMode):
              def __torch_function__(self, func, types, args=(), kwargs=None):
                  nonlocal called
                  if kwargs is None:
                      kwargs = {}
                  called = True
                  return func(*args, **kwargs)
                  
          class B(torch.Tensor):
              pass
              
          x = B(torch.randn(5))
          with torch.overrides.push_torch_function_mode(A):
              with torch._C.DisableTorchFunction():
                  self.assertNotIsInstance(torch.sum(x), B)
  
          self.assertTrue(called)
```

while the following fails because `torch.sum(x)` is of type B (also adjusting the test, called is not True)

```
      def test_disable_subclass_not_mode(self):
          called = False

          class A(TorchDispatchMode):
              def __torch_dispatch__(self, func, types, args=(), kwargs=None):
                  nonlocal called
                  if kwargs is None:
                      kwargs = {}
                  called = True
                  return func(*args, **kwargs)

          class B(torch.Tensor):
              pass

          x = B(torch.randn(5))
          with push_torch_dispatch_mode(A):
              with no_dispatch():
                  self.assertNotIsInstance(torch.sum(x), B)

          self.assertTrue(called)
```

## Problem
This suggests that the behavior of `no_dispatch` and `DisableTorchFunction` don't match

We should either make the behavior of these functions match or make a version of `DisableTorchDispatch` which matches `DisableTorchFunction`

cc @hameerabbasi @rgommers @peterbell10 @Chillee @ezyang @zou3519 @albanD @samdow @eellison 

### Versions

master
Add TORCH_SHOW_CPP_STACKTRACES when TORCH_DISTRIBUTED_DEBUG = detail
Labels: oncall: distributed, module: bootcamp, pt_distributed_rampup, module: c10d

### 🚀 The feature, motivation and pitch

when TORCH_DISTRIBUTED_DEBUG = detail, we should also enable cpp stacktraces so that if a collective is inconsistent, user can get the entire C++ stack to help debug. 

Arguably, we could also just do this for tdd = info. We could also discuss whether if TDD = detail, we also want to enable NCCL_DEBUG=warn or info, although one reason for not doing this is nccl is a separate library who's env vars should be explicitly set by the user.

### Alternatives

_No response_

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501
[ONNX] Re-design `torch.onnx.export`
Labels: module: onnx, triaged, needs design, onnx-triaged

### 🚀 The feature, motivation and pitch

## Motivation

* Reduce the complexity of this function.
* Make exporter more modularized for easier development and maintenance.
* Separate export process in modular responsibilities 
  * Export ONNX using different "backends"
    * torch jit tracing
    * torch jit scripting
    * fx symbolic execution
    * torch inductor
    * whatever backend
  * Validate/verify exported model
  * Custom operators support
    * User overloading existing operators for particular op_name/opset
    * User adding new operators 
  * Shape inference / Dynamic axes support
  * Optimization options
  * Debug options

### Alternatives

* Create new exporter API and incrementally deprecate current torch.onnx.export API

### Additional context

1p users often complain about the difficulty in using this API and end up creating higher order APIs to hide away its complexity
Mac M1 Build Failure on DEBUG=1
Labels: needs reproduction, module: build, triaged, module: macos, module: arm

### 🐛 Describe the bug

DEBUG=1 python setup.py install

> ld: can't open file, errno=1 file 'caffe2/CMakeFiles/torch_cpu.dir/__/torch/csrc/jit/frontend/exit_transforms.cpp.o' for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)


### Versions

PyTorch version: 1.13.0a0+git61a83fc
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.4 (arm64)
GCC version: Could not collect
Clang version: 13.0.0 (clang-1300.0.27.3)
CMake version: version 3.22.1
Libc version: N/A

Python version: 3.9.12 (main, Apr  5 2022, 01:52:34)  [Clang 12.0.0 ] (64-bit runtime)
Python platform: macOS-12.4-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] torch==1.13.0a0+git61a83fc
[conda] numpy                     1.22.3           py39h25ab29e_0  
[conda] numpy-base                1.22.3           py39h974a1f5_0  
[conda] torch                     1.13.0a0+git141238a          pypi_0    pypi


cc @malfet @seemethere @albanD
TorchScript inference get intermediate result?
Labels: oncall: jit, feature

### 📚 The doc issue

https://discuss.pytorch.org/t/how-to-get-the-intermediate-output-from-torchscript/152981

Seems many people ask a solution for this, any documentation about it? Does it able to do so? This is really needed!!

### Suggest a potential alternative/fix

_No response_
Feature Request: Hessenberg and Schur decompositions
Labels: feature, triaged, module: linear algebra

There are a handful of eigenproblem decompositions that I would like to use but are not provided by PyTorch (e.g. Hessenberg and Schur). This is one area where PyTorch noticeably lags behind other applications and libraries.

- [ ] Schur Decomposition (`torch.linalg.schur`)

[Julia](https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/#LinearAlgebra.schur)
[MATLAB](https://www.mathworks.com/help/matlab/ref/schur.html)
[SciPy 
](https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.schur.html)

- [ ] Hessenberg Decomposition (`torch.linalg.hessenberg`)

[Julia](https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/#LinearAlgebra.hessenberg)
[MATLAB](https://www.mathworks.com/help/matlab/ref/hess.html)
[SciPy](https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.hessenberg.html)

cc @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano
Feature request: Integer system decompositions
Labels: feature, triaged, module: linear algebra

I have started to work more and more with integer systems in PyTorch so it would be nice if `linalg` provided some common integer system decompositions (e.g., Hermite or Smith normal forms are obvious choices).

cc @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano
Efficiency of unary operations on CPU for large tensors
Labels: module: performance, triaged

### 🐛 Describe the bug

It seems that doing
```python
out = x.clone()
x.op_()
```
is between a 5% and a 15% faster than doing `x.op()` on tensors floats of size 10^8:
```
[-------------- Unary Ops --------------]
             |  out of place  |  in-place
4 threads: ------------------------------
      abs    |       58       |     52   
      sqrt   |       69       |     54   
      tanh   |      114       |    105   
      atan   |      124       |    115   
      cos    |       84       |     74   
      erf    |       83       |     73   
      log    |       74       |     63   
      log1p  |      603       |    598   
      neg    |       59       |     53   

Times are in milliseconds (ms).
```
When the tensors are smaller (10^7 elements or less), in my machine, the out of place version is comparable or faster than the in-place version. I have not found where's the exact cut-off for this behaviour, but by the looks of it, it may have something to do with pagination 

The script:
```python
from itertools import product

import torch
from torch.utils.benchmark import Compare, Timer


def get_timer(op, fn):
    x = torch.rand((10000, 10000), device="cpu")

    stm = fn(op)
    print(stm)

    timer = Timer(
        stm,
        globals={"x": x},
        label="Unary Ops",
        sub_label=op,
        description="in-place" if "clone" in stm else "out of place",
        num_threads=4)

    return timer.blocked_autorange(min_run_time=5)


def get_params():
    ops = ("abs", "sqrt", "tanh", "atan", "cos", "erf", "log", "log1p", "neg")
    fns = (lambda fn: f"x.{fn}()", lambda fn: f"out = x.clone(); out.{fn}_()")
    for op, fn in product(ops, fns):
        yield op, fn


compare = Compare([get_timer(*params) for params in get_params()])
compare.trim_significant_figures()
compare.print()
```

### Versions

master

cc @VitalyFedyunin @ngimel
Deprecate hardtanh type promotion behavior.
Labels: module: nn, triaged, module: primTorch

### 🚀 The feature, motivation and pitch

There's no clear reason that `hardtanh` should exist - however, it currently has its own kernel and its own `hardtanh_backward`.

See discussion here: https://github.com/pytorch/pytorch/pull/78689/files/618a28c397fbc66433de7b16e5a86a413fd8bf2b#diff-93c7b95139f636278cc494028e322a2c3c3c9ba1e83b2adb35d54ccabed5b47a

cc: @mruberry @ngim

### Alternatives

N/A

### Additional context

N/A

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @ezyang @ngimel
[FSDP] Customizable gradient pre-divide for mixed precision training
Labels: oncall: distributed, triaged, module: fsdp

### 🚀 The feature, motivation and pitch

When training FSDP with mixed precision and fp16, we need to be aware of the reduced range and avoid over/underflow. If we sum all gradients for a large world size and then divide, this might overflow the fp16 range and cause issues, while if we divide first and then sum, we may risk underflow. One possible solution is to divide by sqrt(N), run allreduce, and then divide the result again by sqrt(N) which is equivalent to a N = world size divide. As a result, allreduce only increases the magnitude by sqrt(N=world size), which is less likely to cause overflow.

### Alternatives

_No response_

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501
Add a gallery of examples with sphinx-gallery
Labels: module: docs, triaged

### 📚 The doc issue

A cornerstone of numeric and scientific Python libraries is a gallery of concise, straightforward examples that are longer than per-function examples but shorter than our current [tutorials](https://pytorch.org/tutorials/index.html). They usually have an entirely visual navigation (extremely useful for pattern matching). It would be nice if we could also provide this functionality.

Here are some examples:

[scikit-image](https://scikit-image.org/docs/stable/auto_examples/)
[scikit-learn](https://scikit-learn.org/stable/auto_examples/index.html)
[statsmodels](https://www.statsmodels.org/devel/examples/index.html)
[matplotlib](https://matplotlib.org/stable/gallery/index.html)
[seaborn](https://seaborn.pydata.org/examples/index.html)

cc @svekars @holly1238
Test approximation and numerical stability of numerical operators 
Labels: module: numerical-stability, triaged, module: testing

### 🚀 The feature, motivation and pitch

It would be nice if there was an automated way for us to report the approximate error of our numerical operators and a way to annotate the OpInfo for the operator with the expected error bounds. It’d be even nicer if there was a way to annotate error bounds for specific intervals (e.g., -1.0 and 1.0).

### Alternatives

We’ve made it this far!

### Additional context

_No response_
[primTorch] Sensible Error Messages
Labels: triaged, module: primTorch

In PyTorch today we typically write error messages as follows:

```
torch.foo expected X, but saw Y!
```

When an operator is implemented as a composite of other operations this often causes a choice to be made:

- either the operator has to redundantly implement error checks to produce a reasonable error message
- it has to rely on its constituent operators to throw errors appropriately, and for users to understand that the error generated from `torch.bar` is happening because `torch.foo` calls `torch.bar`

Neither of these solutions is great, of course, and we should decide which approach we want to pursue and how to improve it.

**Option 1: Each operator implements its own error checks.**

This would allow most error messages to be high quality, but...

- we'd want a mechanism to avoid redundant error checks (because they could be expensive)
- internal assertions (which usually involve obscure derived computations that operators shouldn't expect to emulate) will still produce odd error messages
- this approach also requires operators using another operator update whenever the error checks for the used operator update, too. For example, if torch.foo uses torch.bar, and torch.bar changes the input it accepts, then torch.foo will have to be explicitly updated, too, which is a potential maintenance headache

**Option 2: Operators rely on their constituents to generate errors wherever possible.**

This avoids redundant error checks and allows operators to inherit behavior updates, but...

- we'd want a mechanism to improve error message quality

**Proposal**

The second option seems preferable to me, but it requires developing a mechanism to improve error message quality (likely some kind of error context), and we need to be sure that it doesn't have a performance impact.

I can't imagine any mechanism that will correct all error message oddities, however. While it's straightforward to do things like cite the original operation the user called, messages with derived values seem very hard to improve. For example, let's say `torch.foo` has a parameter `height` that it passes to `torch.bar`'s `width` parameter as `width=(height/2)`. Then if that `width` is invalid the user will see an error about `width`, even though they specified `height`!

For those cases we can continue to add additional, redundant error messages,

cc @ezyang @mruberry @ngimel
New c10 constants
Labels: module: internals, triaged

### 🚀 The feature, motivation and pitch

It would be extremely helpful if c10 exposed constants beyond `c10::pi` for all the PyTorch dtypes. Minimally, the GNU predefined constants would suffice but I would advocate for something _slightly_ more expansive like NumPy or Boost. It would also provide the benefit of having consistent representations (e.g., implementors using different representations of constants).

### Alternatives

Continue to inline constants.

### Additional context

I use a lot of constants when implementing PyTorch functions (e.g., special functions)!

cc @ezyang @bhosmer @smessmer @ljk53 @bdhirsh
[Better Engineering] Make OpInfo-based test failures easy to reproduce
Labels: module: tests, triaged, better-engineering

Let's update our OpInfo-based tests to better support randomness and reproducibility by using hypothesis (or a hypothesis-like mechanism) and providing a mechanism to acquire the exact sample input being tested when a failure occurs.

This functionality may be inherently limited to acquiring the exact sample input only when using the same hardware (and software) as the failing test.

cc @mruberry
AlBert quantization
Labels: oncall: quantization, triaged

Dynamicly quantized AlBert model shows poor performance
When I quantized fine-tuned albert-base-v2 model, checkpoint size reduced from 43MB to almost 22MB, but F1 score drops to 0.05 whereas on non quantized version it was 0.8.

`
quantized_model = torch.quantization.quantize_dynamic(
    albert_model, { nn.Linear}, dtype=torch.qint8
)
`
I assume that dynamic quantization adds ops to quantize on each layer of encoder part, but because Albert model has shared layers, it quantizes same weights several times (to be precise 12 times). I have tried distillbert and it worked fine.

I fine-tuned on token-classification task, but I believe you can reproduce the issue by comparing quantized and original albert base model outputs.
Thanks in advance!

### Versions

PyTorch version: 1.11.0+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.3 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.8.10 (default, Mar 15 2022, 12:22:08)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.4.0-105-generic-x86_64-with-glibc2.29
Is CUDA available: True
CUDA runtime version: 10.1.243
GPU models and configuration: 
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090

Nvidia driver version: 470.103.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.1.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.4
[pip3] pytorch-lightning==1.4.9
[pip3] torch==1.11.0
[pip3] torchmetrics==0.7.2

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel
ValueError during `yaml.dump(dtype)`
Labels: module: serialization, triaged

### 🐛 Describe the bug

```python
import torch
import yaml

yaml.dump(torch.half)
```

```python
Traceback (most recent call last):
  File "/home/carmocca/git/repro.py", line 4, in <module>
    yaml.dump(torch.half)
  File "/home/carmocca/git/py39/lib/python3.9/site-packages/yaml/__init__.py", line 253, in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
  File "/home/carmocca/git/py39/lib/python3.9/site-packages/yaml/__init__.py", line 241, in dump_all
    dumper.represent(data)
  File "/home/carmocca/git/py39/lib/python3.9/site-packages/yaml/representer.py", line 27, in represent
    node = self.represent_data(data)
  File "/home/carmocca/git/py39/lib/python3.9/site-packages/yaml/representer.py", line 52, in represent_data
    node = self.yaml_multi_representers[data_type](self, data)
  File "/home/carmocca/git/py39/lib/python3.9/site-packages/yaml/representer.py", line 330, in represent_object
    dictitems = dict(dictitems)
ValueError: dictionary update sequence element #0 has length 1; 2 is required
```

Reported in https://github.com/PyTorchLightning/pytorch-lightning/issues/13158

### Versions

```python
torch==1.11.0+cu113
yaml.__version__==6.0
Python 3.9.7
```

cc @mruberry
Allow batch_norm_backward_elemt and batch_norm_gather_stats_with_counts handle 0 counts
Labels: triaged, enhancement, module: norms and normalization

### 🚀 The feature, motivation and pitch

#36530 requested empty batch support for SyncBatchNorm, which is later added in #74944. However, the fix in #74944 broke CUDA graph capturing as the new code introduced sync on CPU. See #78549.

For v1.12, the temporary fix is to guard the sync in new code with [is_current_stream_capturing](https://pytorch.org/docs/master/generated/torch.cuda.is_current_stream_capturing.html?highlight=is_current_stream_capturing#torch.cuda.is_current_stream_capturing). A better long-term fix would be updating the CUDA kernels for `batch_norm_backward_elemt` and `batch_norm_gather_stats_with_counts` to support zero counts, so that empty batch can work with CUDA graph capturing as well.
torch.distributed.init_process_group(backend="nccl") NCCL version error
Labels: oncall: distributed

### 🐛 Describe the bug

Initializing torch distributed with NCCL backend:
```
import torch
torch.distributed.init_process_group(backend="nccl")
```

Leads to the error of:
```
Traceback (most recent call last):
  File "main_task_caption.py", line 24, in <module>
    torch.distributed.init_process_group(backend="nccl")
  File "/shared/nas/data/users/yifung2/envs/py_univl/lib/python3.6/site-packages/torch/distributed/distributed_c10d.py", line 455, in init_process_group
    barrier()
  File "/shared/nas/data/users/yifung2/envs/py_univl/lib/python3.6/site-packages/torch/distributed/distributed_c10d.py", line 1960, in barrier
    work = _default_pg.barrier()
RuntimeError: NCCL error in: /opt/conda/conda-bld/pytorch_1607370120218/work/torch/lib/c10d/ProcessGroupNCCL.cpp:748, internal error, NCCL version 2.7.8
```

How should I handle such an issue? Pointers greatly appreciated

### Versions

python=3.6.9 
conda install pytorch==1.11.0 cudatoolkit=11.0 -c pytorch
NCCL version 2.7.8
NVIDIA-SMI 470.103.01   Driver Version: 470.103.01   CUDA Version: 11.0  (v11.0.221)

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501
linalg.pinv_singular tests are slow
Labels: module: autograd, module: tests, triaged, module: linear algebra

### 🐛 Describe the bug

More context in https://github.com/pytorch/pytorch/issues/78620

In https://github.com/pytorch/pytorch/pull/78623, we skipped the slowest one (was taking almost 2 hours to run!), but there are still a couple more tests that run for that op that are quite slow. Recently their runtime seemed to have doubled as well (maybe due to https://github.com/pytorch/pytorch/pull/74521).

linalg.pinv_singular tests (previously) combine to represent almost all of slow gradcheck CI runtime (for the first shard) - ~2.6 hours total

### Versions

main

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7 @mruberry @jianyuh @walterddr @IvanYashchuk @xwang233
Module parameters/submodules can be shadowed by class attributes silently
Labels: module: nn, triaged, actionable

### 🐛 Describe the bug

```python
import torch

class Bar(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.attr1 = torch.nn.ReLU()

    def attr1(self):
        return "on the class"
 
b = Bar()
print(b.attr1())
# prints "on the class"
```

This happens because the Class `__dict__` is checked before calling the custom `__getattr__` that `nn.Module` relies on to properly return Parameters and Modules.
We are able to detect such a case when the paramter/submodule is being registered and we should raise an error in that case.

### Versions

master

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345
[FSDP] Enhance sync_module_states for auto wrapping
Labels: oncall: distributed, better-engineering, module: fsdp

### 🚀 The feature, motivation and pitch

Currently `sync_module_states` syncs state wrapped layer by layer. This makes sense for `wrap()` API where the root FSDP instance is not known at construction time.

However, when using `auto_wrap_policy`, we can do a single sync at the root as opposed to layer by layer, possibly saving some overhead.

### Alternatives

_No response_

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501
torch.svd_lowrank fails for complex matrices
Labels: triaged, module: complex, module: linear algebra

The reason is that the function get_floating_dtype from _linalg_utils returns torch.float32 on complex input instead of torch.cfloat or torch.cdouble as appropriate.  This is in version 1.11.0.



cc @ezyang @anjali411 @dylanbespalko @mruberry @Lezcano @nikitaved @jianyuh @pearu @walterddr @IvanYashchuk @xwang233
RFC: Improve the performance and usability of linear algebra on CUDA devices
Labels: module: cuda, triaged, module: linear algebra, module: magma

### 🚀 The feature, motivation and pitch

Currently, the `torch.linalg` (https://pytorch.org/docs/stable/linalg.html) package provides linear algebra functionalities in pytorch. The CUDA backend is supported by cuSOLVER and MAGMA libraries.

For now, linear algebra operators in pytorch are implemented in either cuSOLVER or MAGMA, or both. Users can use 
```python
torch.backends.cuda.preferred_linalg_library(backend='cusolver')
```
to prefer one of the two backends. Available options (python `str`) are `default` (using heuristics), `cusolver`, or `magma`. See doc for details https://pytorch.org/docs/stable/backends.html#torch.backends.cuda.preferred_linalg_library.

However, libraries have limitations and heuristics can't be perfect on all devices, library versions, input batch sizes, and input shapes. We'd like to collect user feedbacks and feature requests on the performance and usability of pytorch linear algebra on CUDA devices. Please leave a comment if you have any suggestions. Thank you!

### Alternatives

_No response_

### Additional context

_No response_

cc @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano @ptrblck @ngimel
[JIT] autodiff implementation of rand_like function is outdated 
Labels: oncall: jit

### 🐛 Describe the bug

The autodiff implementation of rand)like in symbolic_script.cpp is outdated such that it doesn't match the updated function signature in the config file.

The function signature in symbolic_script.cpp is currently:
def rand_like(self, *, memory_format: Optional[int])

Which is incorrect

### Versions

N/A
LibTorch cannot be used without nvcc
Labels: module: build, triaged

### 🐛 Describe the bug

When using LibTorch in CMake via `find_package(Torch REQUIRED)` it tries to find CUDA via `nvcc`. When only the runtime libraries are installed, e.g. `sudo apt install cuda-libraries-11-3` then it will fail with:
```
CUDA_TOOLKIT_ROOT_DIR not found or specified
-- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY) 
CMake Warning at [...]/cmake/Caffe2/public/cuda.cmake:31 (message):
  Caffe2: CUDA cannot be found.  Depending on whether you are building Caffe2
  or a Caffe2 dependent library, the next warning / error will give you more
  info.
```

Since LibTorch is already compiled, it should not require `nvcc` and other development libraries to be installed.

Looking at `Caffe2/public/cuda.cmake` and https://github.com/pytorch/pytorch/blob/v1.11.0/cmake/public/cuda.cmake#L29, it seems that the [deprecated `find_package(CUDA)`](https://cmake.org/cmake/help/latest/module/FindCUDA.html) is used. Instead, the new [`FindCUDAToolkit`](https://cmake.org/cmake/help/latest/module/FindCUDAToolkit.html#module:FindCUDAToolkit) should be used. This will work without `nvcc` installed.

### Versions

1.11 from https://download.pytorch.org/libtorch/cu113/libtorch-cxx11-abi-shared-with-deps-1.11.0%2Bcu113.zip

cc @malfet @seemethere
clear input shape declaration  on pytorch model inputs and outputs
Labels: triaged, module: shape checking, module: python frontend

### 🚀 The feature, motivation and pitch

It is hard to read a new pytorch project, compared to keras like projects.  each DL models can be seen as implementation of certain function, with scheduled inputs and outputs. But there is no clear information about shape, dtype for inputs and outputs in pytorch projects.

### Alternatives

or, to provide some convenient tools to test input and output shape, dtype, for any pytorch model - before training or after.

### Additional context

_No response_
Parallel execution of multiple unrelated statements written sequentially
Labels: triaged, enhancement

### 🚀 The feature, motivation and pitch

Often, I write statements in a sequential manner that do not relate to each other - the order of their execution does not matter. Such statements can be parallelized to achieve better performance. One example: resize three different images using nearest, bilinear, and bicubic:

Old method:

```python
img1 = F.interpolate(img1, mode="nearest", ...)
img2 = F.interpolate(img2, mode="bilinear", ...)
img3 = F.interpolate(img3, mode="bicubic", ...)
```

New method:

```python
img1, img2, img3 = parallel_api_fn([
    {
        "function": F.interpolate, 
        "args": {"inputs": img1, "mode": "nearest"}
    },
    {
        "function": F.interpolate, 
        "args": {"inputs": img2, "mode": "bilinear"}
    },
    {
        "function": F.interpolate, 
        "args": {"inputs": img3, "mode": "bicubic"}
    }
])
```

Of course, the implementation mentioned above is just an example. The core idea is to take a list of functions (same or different), and their respective arguments. These functions are then executed in parallel using the new API function.

### Alternatives

Currently, the functionality can be implemented using vmap or python multiprocessing. However, each of these requires a lot of additional code and has a lot of added complexity. If we assume that every line to be executed in parallel is independent and all arguments are immutable, I believe the lines can easily be run in parallel on GPU.

### Additional context

_No response_

cc @ngimel @bdhirsh
[1.9.1] [collect_env] collect_env does not collect actual runtime-loaded cudnn version
Labels: module: cudnn, module: collect_env.py, triaged, enhancement

### 🐛 Describe the bug

In opposite, `torch.backends.cudnn.version()` does collect it correctly.
Originally reported by @grazder in a comment: https://github.com/pytorch/pytorch/issues/78475#issuecomment-1140516737

This occurred on old-ish version of 1.9.1, not sure if it still happens on 1.11.0

### Versions

1.9.1

cc @csarofeen @ptrblck @xwang233
New feature requested: vmap for torch.histc
Labels: high priority, triaged, module: functorch

Hi,

I hit the following warning
```
 UserWarning: There is a performance drop because we have not yet implemented the batching rule for aten::histc. Please file us an issue on GitHub so that we can prioritize its implementation. (Triggered internally at  /__w/functorch/functorch/functorch/csrc/BatchedFallback.cpp:85.)
  h = torch.histc(x, bins=n_buckets, min=min_dist, max=max_dist)
```

Version
```
>>> functorch.__version__
'0.1.1'
```

Thanks 


cc @ezyang @gchanan @zou3519 @Chillee @samdow @kshitij12345 @janeyx99 @soumith
torch.fx: symbolic_trace: ones() received an invalid combination of arguments
Labels: triaged, module: fx

### 🐛 Describe the bug

Run the following standalone python file.

```
import torch
from torch.fx import symbolic_trace
from transformers import AutoModelForSequenceClassification


class OnlyLogitsHuggingFaceModel(torch.nn.Module):
    """Wrapper that returns only the logits from a HuggingFace model."""

    def __init__(self):
        super().__init__()
        self.model = AutoModelForSequenceClassification.from_pretrained(
            "ProsusAI/finbert",  # The pretrained model name.
            num_labels=3,
            output_attentions=False,
            output_hidden_states=False,
            torchscript=True,
        )
        self.model.eval()

    def forward(self, input):
        # Return only the logits.
        return self.model(input)[0]



traced = symbolic_trace(OnlyLogitsHuggingFaceModel())
```

I get the error:
```
TypeError: ones() received an invalid combination of arguments - got (tuple, device=Attribute), but expected one of:
 * (tuple of ints size, *, tuple of names names, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
 * (tuple of ints size, *, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
```
Full error log: https://gist.github.com/silvasean/a11f5219e8d931014ae4046a1fafcef7

### Versions

PyTorch version: 1.13.0.dev20220530+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux rodete (x86_64)
GCC version: (Debian 11.2.0-19) 11.2.0
Clang version: 13.0.1-3+build2
CMake version: version 3.22.4
Libc version: glibc-2.33

Python version: 3.9.12 (main, Mar 24 2022, 13:02:21)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.17.6-1rodete1-amd64-x86_64-with-glibc2.33
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] iree-torch==0.0.1
[pip3] numpy==1.23.0rc1
[pip3] torch==1.13.0.dev20220530+cpu
[pip3] torch-mlir==20220530.482
[pip3] torchvision==0.14.0.dev20220530+cpu
[conda] Could not collect

cc @ezyang @SherlockNoMad
torch.lerp: discrepancy between CUDA and CPU (with extremal inputs)
Labels: triaged, module: NaNs and Infs

### 🐛 Describe the bug

```python
import torch

st = torch.tensor(0.2345)
en = torch.tensor(float("inf"))
weight = torch.tensor(-0.9890)

print(torch.lerp(st, en, weight))   # -inf
print(torch.lerp(st.cuda(), en.cuda(), weight.cuda()))  # nan
```

### Versions

master
is the issue resolved? windows not pytorch_jni in path
Labels: oncall: java

### 🐛 Describe the bug

same question in https://github.com/pytorch/pytorch/issues/44363

is it resolved? how can i download the new version code or Pom.xml by java code style
i have a idea :
    1.like djl ，it automatically download when the service start;
    or 2. give the full relation packages i download it by myself
RuntimeError: Event device type CUDA does not match blocking stream’s device type CPU 
Labels: module: autograd, module: cuda, module: tests, triaged

### 🐛 Describe the bug

when I do the unittest of test_autograd.py on pytorch1.11.0, it throw an RuntimeError: Event device type CUDA does not match blocking stream’s device type CPU. I want to know what causes this error.

```
Traceback (most recent call last):
  File "/root/.local/lib/python3.7/site-packages/torch/testing/_internal/common_utils.py", line 1754, in wrapper
    method(*args, **kwargs)
  File "/root/.local/lib/python3.7/site-packages/torch/testing/_internal/common_utils.py", line 1754, in wrapper
    method(*args, **kwargs)
  File "/root/.local/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 389, in instantiated_test
    raise rte
  File "/root/.local/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 376, in instantiated_test
    result = test(self, **param_kwargs)
  File "/root/.local/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 939, in multi_fn
    return fn(slf, devices, *args, **kwargs)
  File "pytorch/test/test_autograd.py", line 8644, in test_backward_device
    Identity.apply(v).backward()
  File "/root/.local/lib/python3.7/site-packages/torch/_tensor.py", line 363, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/root/.local/lib/python3.7/site-packages/torch/autograd/__init__.py", line 175, in backward
    allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
RuntimeError: Event device type CUDA does not match blocking stream's device type CPU.
```

### Versions

PyTorch version: 1.11.0a0
Is debug build: False
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: 4.3.22211

OS: CentOS Linux 7 (Core) (x86_64)
GCC version: (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)
Clang version: 14.0.0 
CMake version: version 2.8.12.2

Is CUDA available: True



cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7 @ngimel @mruberry
[onnx] RuntimeError: Attribute 'axes' is expected to have field 'ints'
Labels: module: onnx, triaged, onnx-needs-info

### 🐛 Describe the bug

RuntimeError: Attribute 'axes' is expected to have field 'ints'

When I want to export the transformer model containing the proj_adaptive_softmax layer to the onnx format, I get an error: RuntimeError: Attribute 'axes' is expected to have field 'ints', ==> Context: Bad node spec: input: "567" output: " 584" name: "Unsqueeze_445" op_type: "Unsqueeze" attribute { name: "axes" type: INTS }

It means that axes type must be int. I found the corresponding line of code:

````
nll.index_copy_(0, indices_i, -logprob_i)
````

It means that indices_i must be a tensor of type int, but the problem is that indices_i is already a tensor of int64. When I replace it with a constant tensor equal to indices_i, no error is reported

````
#nll.index_copy_(0, indices_i, -logprob_i)
a = torch.tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21 , 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35])
nll.index_copy_(0, a, -logprob_i)
print(indices_i == a)
#[true,true,true,true,true,true,true,true,true,true,true,true,true,true,true,true,true,true,true,true,true,true,true,true, true,true,true,true,true,true,true,true]
````

Why is this?

### Versions

Pytorch Version == 1.7.1+cu101
OS == Linux
onnx == 1.10.2
CUDA Version == 10.1
Softmax, LogSoftmax are over parameterized
Labels: module: nn, triaged

### 🚀 The feature, motivation and pitch

The current implementation of Softmax and LogSoftmax for a K class output involves an [NxK] input tensor. This is over parameterized and leads to issues of identifiability. It would be worthwhile to include an implementation that only requires an input tensor [NX(K-1)] dimensionality.

### Alternatives

_No response_

### Additional context

_No response_

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345
`layer_norm` triggers INTERNAL ASSERT with input requiring grad + zero-size int tensor
Labels: module: autograd, triaged, actionable

### 🐛 Describe the bug

`layer_norm` triggers INTERNAL ASSERT with input requiring grad + zero-size int tensor

```python
import torch

input = torch.randint(0, 8, [0, 0, 1024], dtype=torch.int64)
normalized_shape = [1024]
eps = 1e-05

weight = torch.rand([1024], dtype=torch.float64, requires_grad=True)
bias = torch.rand([1024], dtype=torch.float64, requires_grad=True)
torch.nn.functional.layer_norm(input, normalized_shape, weight=weight, bias=bias, eps=eps, )
# RuntimeError: isDifferentiableType(variable.scalar_type())INTERNAL ASSERT FAILED at "/Users/distiller/project/pytorch/torch/csrc/autograd/functions/utils.h":65, please report a bug to PyTorch
```

`layer_norm` does check the dtype of zero-size tensor, like `input` in this example. If `input` is not zero-size, it will raise an error that `RuntimeError: "LayerNormKernelImpl" not implemented for 'Long'`

### Versions

pytorch: 1.11.0

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
`index_fill` will trigger INTERNAL ASSERT when float tensor requiring grad + int tensor
Labels: module: autograd, triaged, actionable

### 🐛 Describe the bug

`index_fill` will trigger INTERNAL ASSERT when float tensor requiring grad + int tensor

```python
import torch
value = torch.rand([], dtype=torch.float64, requires_grad=True)

input = torch.randint(-2, 8, [], dtype=torch.int64)
index = torch.randint(-1, 1, [], dtype=torch.int64)
dim = 0
input.index_fill(dim, index, value)
# RuntimeError: isDifferentiableType(variable.scalar_type())INTERNAL ASSERT FAILED at "/Users/distiller/project/pytorch/torch/csrc/autograd/functions/utils.h":65, please report a bug to PyTorch. 
```

### Versions

pytorch: 1.11.0

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
Permutation of Sparse Tensor
Labels: module: sparse, triaged

### 🚀 The feature, motivation and pitch

I'm working with COO sparse tensors and would like to get a permutation of any COO sparse tensor.
In the current version, `torch.permute` throws the following error:
`RuntimeError: sparse tensors do not have strides`
Also, in-place modification of indices is not possible.

### Alternatives

I have considered the following solution (it wouldn't be the same as `torch.permute` since this one returns a view of the original tensor):

`torch.permute_sparse(input, dims) → Tensor`
Returns a copy of the original tensor input with its dimensions permuted.
- **input** (_Tensor_) – the input tensor.
- **dims** (_tuple of python:ints_) – The desired ordering of dimensions

The source code:
```
def permute_sparse(input, dims):
    dims = torch.LongTensor(dims)
    return torch.sparse_coo_tensor(indices=input._indices()[dims], values=input._values(), size=torch.Size(torch.tensor(input.size())[dims]))
```

An example:
```
>>> indices = torch.tensor([[1, 1],
                            [2, 1],
                            [1, 0],
                            [3, 3]])
>>> values = torch.tensor([ 1, -1])
>>> s = torch.sparse_coo_tensor(indices=indices, values=values, size=(2, 3, 4, 5))
>>> s.size()
torch.Size([2, 3, 4, 5])
>>> permute_sparse(s, (3, 2, 0, 1)).size()
torch.Size([5, 4, 2, 3])
```

### Additional context

_No response_

cc @nikitaved @pearu @cpuhrsch @amjames
.lldbinit for lldb debuger
Labels: feature, triaged, module: macos, actionable

### 🚀 The feature, motivation and pitch

found `.gdbinit`, but no `.lldbinit`, If more lldb feature(like prettyprint for tensor) is supported, the debuging progress using lldb would have better experience.

### Alternatives

Nop

### Additional context

Nop

cc @malfet @albanD
torch.angle differs from np.angle for -0.
Labels: triaged, module: numpy, module: primTorch

### 🐛 Describe the bug

Discovered in https://github.com/pytorch/pytorch/pull/78349

```python
>>> import torch
>>> import numpy as np
>>> t = torch.tensor(-0.)
>>> torch.angle(t)
tensor(0.)
>>> np.angle(t.numpy())
3.141592653589793
```

cc: @mruberry 

### Versions

master

cc @mruberry @rgommers @ezyang @ngimel
Torchdynamo for Deepspeed and FSDP
Labels: triaged, enhancement, module: fsdp, oncall: pt2

In AWS we are working with customers to enable gigantic transformer model training on EC2. Furthermore, we attempt to leverage compiler techniques to optimize Pytorch workloads due to its widely adoption. For example, we recently open-sourced [RAF](https://github.com/awslabs/raf), a deep learning compiler for training, that shows promising training acceleration for a set of models and verified it works with TorchDynamo. On the other hand, we do see the gap converting Pytorch programs to the IR we are using, especially when it comes to complex strategies such as distributed training.

One example is [Deepspeed](https://github.com/microsoft/DeepSpeed). It implements data parallelism (ZeRO) on top of Pytorch, introducing sharding for optimizer states, gradients and parameters. (FSDP is another approach to do ZeRO). The idea itself is pretty straightforward, but when trying to convert the Deepspeed implementation to RAF via lazy-tensor, it doesn’t work well. For instance, the [NaN check for gradients](https://github.com/microsoft/DeepSpeed/blob/master/deepspeed/runtime/zero/stage3.py#L2461) breaks the graph and result performance degradation; It doesn’t capture CUDA stream usage; etc.

In my understanding, those issues could potentially be resolved via TorchＤynamo as it has the capability to extract structures and call information via Python bytecode, apart from the lazy tensor tracing mechanism at higher level.

So we’d like to get suggestion from TorchDynamo community, what do you think about supporting such scenarios in TorchDynamo? Specifically, 1) how should TorchDynamo support ZeRO tracing? 2) would that be in the TorchDynamo soon or later? Due to our current roadmap and goals, we at AWS would be interested in collaboration along this direction if possible.

cc @zhaojuanmao @mrshenli @rohan-varma @awgu @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @comaniac @szhengac @yidawang @mli @hzfan @zachzzc
Split up and reorganize RPC tests
Labels: oncall: distributed, triaged, better-engineering, module: rpc

## Issue description

Right now all RPC tests are in https://github.com/pytorch/pytorch/blob/master/torch/testing/_internal/distributed/rpc/rpc_test.py which is a 6500+ line file containing tests for `CudaRpcTest`, `FaultyAgentRpcTest`, `RpcTest`, `RpcTestCommon`, `TensorPipeAgentRpcTest`, `TensorPipeAgentCudaRpcTest`. These classes are then imported by various files such as https://github.com/pytorch/pytorch/blob/master/torch/testing/_internal/distributed/rpc_utils.py. It would be helpful to separate the tests so it is easier to know what you are adding to. (I encountered this when accidentally adding to RpcTest and the test began to be run internally which I did not want)





cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501 @jjlilley @mrzzd
`gradcheck` fails for `torch.distribution.transform` APIs in forward mode
Labels: module: distributions, module: autograd, triaged, module: forward ad

### 🐛 Describe the bug

`gradcheck` fails for `torch.distribution.transform` APIs in forward mode when `cache` is not 0.

Take `AbsTransform` as an example,
```python
import torch

def get_fn():
    cache_size = 1
    arg_class = torch.distributions.transforms.AbsTransform(cache_size=cache_size, )
    def fn(input):
        fn_res = arg_class(input)
        return fn_res
    return fn
fn = get_fn()

input = torch.rand([3], dtype=torch.float64, requires_grad=True)
torch.autograd.gradcheck(fn, (input,), check_forward_ad=True)
# torch.autograd.gradcheck.GradcheckError: Jacobian mismatch for output 0 with respect to input 0,
# numerical:tensor([[0., 0., 0.],
#        [0., 0., 0.],
#        [0., 0., 0.]], dtype=torch.float64)
# analytical:tensor([[1., 0., 0.],
#        [0., 1., 0.],
#        [0., 0., 1.]], dtype=torch.float64)
```

Other APIs like `SigmoidTransform, TanhTransform, SoftmaxTransform` also have this issue

### Versions

pytorch: 1.11.0

cc @fritzo @neerajprad @alicanb @nikitaved @ezyang @albanD @zou3519 @gqchen @pearu @soulitzer @Lezcano @Varal7
TRACK: integral + floating inputs to an op with floating requiring grad result in INTERNAL_ASSERT
Labels: module: autograd, triaged, actionable

### 🐛 Describe the bug

This is the tracking of the issues that "integral + floating inputs to an op with floating requiring grad result in INTERNAL_ASSERT"

- tensordot #77517
- addmv #77814 
- mv #77814 
- bilinear #78087 
- matmul #78141 
- mm #78141 
- baddmm #78143 
- index_fill #78443
- layer_norm #78444

### Versions

pytorch: 1.11.0

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
Request for adding the possibility for training on sparse tensors
Labels: module: sparse, triaged

### 🚀 The feature, motivation and pitch

I'm working on tensors that most of the cells are 0's and some cells are 1's, say `100` 1's out of `50^3` 0's.
I tried to use [turn tensor into sparse tensor](https://pytorch.org/docs/stable/generated/torch.Tensor.to_sparse.html#torch-tensor-to-sparse) in order to turn my tensors into sparse ones.
The problem is that I can't use them in order to propagate inside a neural network for training and I'm getting the error message: `RuntimeError: Input type (torch.cuda.sparse.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same`.
It would be great if there be a possibility for training on sparse tensors too.

Thank you

### Alternatives

_No response_

### Additional context

_No response_

cc @nikitaved @pearu @cpuhrsch @amjames