gopikrsmscs/torch-issues huggingface dataset:
Investigate Lazy{*}Norm{*}d modules no batch dim support
Labels: module: nn, triaged

### 🐛 Describe the bug

LazyInstanceNorm is documented to support no_batch_dim (e.g. (C, L) is listed as a valid input shape [here](https://pytorch.org/docs/stable/generated/torch.nn.LazyInstanceNorm1d.html)) but in _LazyNormBase, [`in_features` is initialized as `.shape[1]`](https://github.com/pytorch/pytorch/blob/099b2801dbcf23aba0b5c6862451b7b18272afd0/torch/nn/modules/batchnorm.py#L221), which should be incorrect when there is no batch dimension.

e.g. the following snippet fails


```
import torch
lazy_instance_norm = torch.nn.LazyInstanceNorm()
lazy_instance_norm(torch.randn(3, 15))
# ValueError: expected input's size at dim=0 to match num_features (15), but got: 3.
```

This might also apply to batchnorm which also uses _LazyNormBase and needs further investigation.


### Versions

master

cc @albanD @mruberry @jbschlosser @walterddr
A parameterized fill value for triu and tril functions
Labels: triaged, enhancement, module: viewing and reshaping

### 🚀 The feature, motivation and pitch

to mask the multihead attention layers we must store a mask tensor and perform additive or fill operations on the attention matrix, this tensor occupies memory. For the cases where we need to apply a causal mask, we can simply select the lower part of the attention matrix, filling the rest of the matrix with the value -inf. Currently our tril function does not allow this simple functionality. 

I propose to add the fill_value parameter, default zero, to both functions.

`torch.tril(input, diagonal=0, *, fill_value=0, out=None) → [Tensor]`
`torch.triu(input, diagonal=0, *, fill_value=0, out=None) → [Tensor]`

```python
def multi_head_attention(q, k, v,  /, heads, causal = None):
    q = rearrange(q, '... l (h k) -> h ... l k', h=heads)
    k = rearrange(k, '... t (h k) -> h ... t k', h=heads)
    v = rearrange(v, '... t (h v) -> h ... t v', h=heads)

    attn = einsum(q, k, 'h ... l k, h ... t k -> h ... l t')
    attn = attn / np.sqrt(q.shape[-1])

    if causal is not None:
        attn = attn.tril(fill_value=-np.inf)
        
        # we dont need this anymore:
        #   mask = view_for_broadcast(mask, to=attn)
        #   attn = attn.masked_fill(mask, -np.inf)
        # or:
        #   attn = attn - mask * (-1e10)

    attn = pt.softmax(attn, dim=-1)

    output = einsum(attn, v, 'h ... l t, h ... t v -> h ... l v')

    return rearrange(output, 'h ... l v -> ... l (h v)'), attn
```

### Alternatives

_No response_

### Additional context

_No response_

cc @gchanan @mruberry
Type conversion between float/complex
Labels: triaged, module: complex, enhancement

### 🚀 The feature, motivation and pitch

It would be nice to have a simple way to convert between float and complex types of the same precision. Something like
```python
dtype = torch.float32
print(dtype.to_complex())
# torch.complex64

dtype = torch.complex128
print(dtype.to_float())
# torch.float64

dtype = torch.float64
print(dtype.to_float())
# torch.float64
```

### Alternatives

1. building a dictionnary that stores conversions from float to complex types of the same precision
2. using this unpleasant/difficult to read workaround: `getattr(torch, torch.finfo(dtype).dtype)`

### Additional context

We are building a library for the simulation of quantum systems built on torch (see [torchqdynamics](https://github.com/PierreGuilmin/torchqdynamics)), and conversion between float and complex types happens all the time. Quantum objects are always complex, but time is always float-like, and it's essential that they are of the same precision.

cc @ezyang @anjali411 @dylanbespalko @mruberry @Lezcano @nikitaved
Unable to install torch on python 3.8.16
Labels: needs reproduction, oncall: binaries, triaged

### 🐛 Describe the bug

We have the ci pipeline which will install python 3.8.16 and on top of that we install torch package.
python -m pip  install -r requirements.txt
requirements.txt contains
torch==2.0.0

Error we getting is 
ERROR: Ignored the following versions that require a different python version: 1.9.5 Requires-Python >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <3.7
ERROR: Could not find a version that satisfies the requirement torch==2.0.0 (from versions: none)
ERROR: No matching distribution found for torch==2.0.0

### Versions

3.8.16

cc @seemethere @malfet
torch.onnx.errors.OnnxExporterError: Unsupported: ONNX export of operator unsafe_chunk, unknown dimension size.
Labels: module: onnx, triaged

### 🐛 Describe the bug

from speechbrain.pretrained import Tacotron2
from speechbrain.pretrained import HIFIGAN
tacotron2 = Tacotron2.from_hparams(source="speechbrain/tts-tacotron2-ljspeech", savedir="tmpdir_tractron2")
hifi_gan = HIFIGAN.from_hparams(source="speechbrain/tts-hifigan-ljspeech", savedir="tmpdir_hifi")
torch.onnx.export(tacotron2.mods.model )

### Versions

torch=2.0
make tensor data const correct
Labels: module: internals, triaged

### 🚀 The feature, motivation and pitch

The layers of tensor data throughout our stack have historically been lax about const correctness. We don't, for example, consistently distinguish between a data accessor granting mutable vs const access. This makes it challenging to add instrumentation that can distinguish data reads from data writes.

Furthermore, we mark methods const that give out mutable handles to class internals.

We should be strict on our tensor class and its internals about these accesses.

There is a straightforward migration path that forces us to make a decision at each callsite: change the existing method to return a const pointer and introduce an equivalent variant named `mutable_<original_method_name>` that gives a non-const pointer. Then we just fix what breaks.

While doing this, we need to be careful about not just blindly changing to the mutable variant: we should only do so when we must.

Challenges:
Some of our libraries were designed without considering this and will require some redesign to accommodate this distinction. For example, TensorIterator levels all of its arguments to non-const. JIT function calls do the same.

### Alternatives

_No response_

### Additional context

We want this right now to instrument reads/writes with high fidelity to accurately simulate and also implement copy-on-write tensors.

cc @ezyang @bhosmer @smessmer @ljk53 @bdhirsh
TORCH_LIBRARIES variable leads to undefined reference function error in compiling while using libtorch in c++
Labels: module: cpp, triaged

### 🐛 Describe the bug

I'm using cmake to build my c++ project. When i add ${TORCH_LIBRARIES} into target_link_libraries's parameters, the compiling fails.

Here's some of my cmake codes:

> set(CMAKE_CXX_STANDARD 14)
> set(THIRD_PARTY_PATH "${PROJECT_SOURCE_DIR}/third_party")
> set(Torch_DIR /opt/libtorch_1.13/share/cmake/Torch)
> find_package(Torch REQUIRED)
> include_directories(${PROJECT_SOURCE_DIR}/third_party/include)
> 
> add_executable(torch_test main.cpp)
> target_link_libraries(torch_test ${PROJECT_SOURCE_DIR}/third_party/lib/libjsoncpp.so)
> target_link_libraries(torch_test -Wl,--no-as-needed /opt/torch_tensorrt/lib/libtorchtrt.so)
> 
> target_link_libraries(torch_test ${TORCH_LIBRARIES}
>         /opt/TensorRT-8.5.3.1/lib/libnvinfer.so
>         /opt/TensorRT-8.5.3.1/lib/libnvinfer_plugin.so
>         /opt/libtorch_1.13/lib/libtorch_global_deps.so
>         /opt/libtorch_1.13/lib/libc10_cuda.so
> )

### Versions

system: ubuntu2204
libtorch version: 1.13
cuda version: 11.7

cc @jbschlosser
CUDA 10.2 cudnn 8.2.4 run Conv2d error
Labels: needs reproduction, module: cudnn, module: cuda, triaged

### 🐛 Describe the bug

    
 File "/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 399, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 395, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED


### Versions
i try cudnn 7.6 8.0 as well
pytorch 1.8.0
cudnn 8.2.4
cuda 10.2
2080ti

cc @csarofeen @ptrblck @xwang233 @ngimel
Memory leak when saving an input tensor returned as-is if mark_dirty and running with dual tensors
Labels: module: autograd, triaged, module: edge cases, module: forward ad

```python
import weakref
import torch
import torch.autograd.forward_ad as fwAD
import gc

def scope():
  saved_tensors = []
  class A(torch.autograd.Function):
      @staticmethod
      def forward(x):
          return x

      @staticmethod
      def setup_context(ctx, inputs, output):
          ctx.mark_dirty(inputs[0])
          ctx.save_for_backward(output)
          saved_tensors.append(output)

      @staticmethod
      def backward(ctx, grad_output):
          return grad_output

      @staticmethod
      def jvp(ctx, x_t):
          x_t.add_(0)
          return x_t

  a = torch.tensor(2., device="cpu", requires_grad=True).clone()
  a_t = torch.tensor(2., device="cpu", requires_grad=True).clone()

  with fwAD.dual_level():
      a_dual = fwAD.make_dual(a, a_t)
      A.apply(a_dual)

  class Test():
      pass
  test = Test()
  ref = weakref.ref(test)
  saved_tensors[0].grad_fn.metadata["test"] = test

  return ref

ref = scope()
gc.collect()
print(ref())
```

The reason this happens is that the output node is actually an AsStridedBackward node, but the logic testing for whether saved tensors are outputs checks that the grad_fn is ABackward. 

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @Lezcano @Varal7
Support sparse COO/CSR/CSC/BSR/BSC return values in gradcheck input function
Labels: module: sparse, module: autograd, open source, release notes: autograd, release notes: sparse, skip-pr-sanity-checks, no-stale

As in the title.

Previously, the input functions to gradcheck were expected to return strided tensors: say, when the result of an input function would be a sparse tensor, it had to be converted to strided via `.to_dense(masked=...)` method call. With this PR, the conversion of input function return values to strided tensors is not required anymore.

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #97825
* #98292
* #98288



cc @alexsamardzic @nikitaved @cpuhrsch @amjames @bhosmer @ezyang @albanD @zou3519 @gqchen @soulitzer @Lezcano @Varal7
Some c++ library docstrings incorrectly linked/repeated
Labels: module: docs, module: cpp, triaged, module: mps

### 📚 The doc issue

For example, the docstring for [torch::mps::is_available()](https://pytorch.org/cppdocs/api/function_namespacetorch_1_1mps_1ac04d09cc0c1b3c42588745c70b9c7cac.html):

```
Returns true if MPS device is available.

Returns true if MPS device is available.

Returns true if at least one CUDA device is available.

If we compiled with CUDA but there is a driver problem, etc., this function will report CUDA is not available (rather than raise an error.)
```

Looks incorrect.

Another example: [at::cuda::is_available()](https://pytorch.org/cppdocs/api/function_namespaceat_1_1cuda_1abdf5679016d4e4149f80f3fe2e80890f.html).

```
CUDA is available if we compiled with CUDA, and there are one or more devices.

Returns true if MPS device is available.

Returns true if at least one CUDA device is available.

If we compiled with CUDA but there is a driver problem, etc., this function will report CUDA is not available (rather than raise an error.)
```

### Suggest a potential alternative/fix

_No response_

cc @svekars @carljparker @jbschlosser @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
Add SSIM as Loss Function
Labels: feature, module: loss, triaged, needs research

### 🚀 The feature, motivation and pitch

Hello, I am doing research in the field of Unsupervised Anomaly Detection. In Unsupervised Anomaly Detection, you can segment an anomaly area through Reconstruction Error, but in the case of areas with ambiguous boundaries, better performance is shown when using loss using SSIM than using existing losses. Therefore, I would like to be able to use SSIM as the loss function of pytorch. Thank you :) 

### Alternatives

_No response_

### Additional context

_No response_
consider bumping `DEFAULT_PROTOCOL`
Labels: module: serialization, triaged

### 🐛 Describe the bug

As per https://github.com/pytorch/pytorch/releases/tag/v2.0.0, PyTorch 2.0 dropped support for Python versions up to and including 3.7, but it's unclear whether `DEFAULT_PROTOCOL` in `torch/serialization.py` is intended to keep its current value (originally set in August 2016!) or was simply overlooked.

https://github.com/pytorch/pytorch/blob/08766b23de8ed50a8f52973df78114426500cb65/torch/serialization.py#L23

https://docs.python.org/3/library/pickle.html#data-stream-format says:

> * Protocol version 2 was introduced in Python 2.3. It provides much more efficient pickling of [new-style classes](https://docs.python.org/3/glossary.html#term-new-style-class). Refer to [PEP 307](https://peps.python.org/pep-0307/) for information about improvements brought by protocol 2.
> * Protocol version 3 was added in Python 3.0. It has explicit support for [bytes](https://docs.python.org/3/library/stdtypes.html#bytes) objects and cannot be unpickled by Python 2.x. This was the default protocol in Python 3.0–3.7.
> * Protocol version 4 was added in Python 3.4. It adds support for very large objects, pickling more kinds of objects, and some data format optimizations. It is the default protocol starting with Python 3.8. Refer to [PEP 3154](https://peps.python.org/pep-3154/) for information about improvements brought by protocol 4.
> * Protocol version 5 was added in Python 3.8. It adds support for out-of-band data and speedup for in-band data. Refer to [PEP 574](https://peps.python.org/pep-0574/) for information about improvements brought by protocol 5.

As such, it seems worthwhile to consider bumping `DEFAULT_PROTOCOL`. Otherwise, it would be worth at least documenting the reasoning for **_not_** bumping it?


### Versions

PyTorch 2.0 (`v2.0.0`)

cc @mruberry
torch.testing.assert_close: allow check to fail on part on the input
Labels: triaged, module: testing

### 🚀 The feature, motivation and pitch

**Request**: A feature in `torch.testing.assert_close()` that allows users to state: "It's OK if the assertion fails on less than x% of the input (or y entries in absolute terms)" e.g. `assert_close(a, b, atol=atol, rtol=rtol, ignore_failure_rate_below=.02)` (name TBD)


**Why**:
Very often in torchvision we have equality checks that *almost* pass, except for a tiny (negligible) portion of the input. E.g.:

```
Mismatched elements: 1 / 50 (2.0%)
Greatest absolute difference: 3.0863037109375 at index (0, 22) (up to 0.1 allowed)
Greatest relative difference: 0.16271504955728078 at index (0, 22) (up to 0.1 allowed)
```

In the example above we don't really mind that 2% of the input fails, we'd want to test to pass instead of fail.

### Alternatives

Increasing atol or rtol aren't viable alternative in most cases, nor is computing the MAE or other metrics (because the difference between the few values that differ might be arbitrarily high, and that should still be OK).

### Additional context

CC @pmeier as discussed offline
oneDNN 3.0+ support
Labels: triaged, module: intel

### 🚀 The feature, motivation and pitch

MKLDNN was renamed to oneDNN a long time ago and the cmake name is also different.
The internal copy of MKLDNN in PyTorch is very outdated.

While PyTorch can be built with external oneDNN 2.x (with patches to look for the new cmake name), oneDNN 3.0 may also be found that way but PyTorch doesn't build because of API differences in oneDNN.

Please support oneDNN 3.0+

### Alternatives

_No response_

### Additional context

_No response_

cc @frank-wei @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10
TypeError: 'torch._C._TensorMeta' object is not iterable
Labels: module: windows, module: rocm, triaged

### 🐛 Describe the bug

Error on initial use
pip install torch
import torch
x = torch.rand(5,3)
print(x)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\Clouds\OneDrive\Documents\dev\github\openai\ChatGPT-at-Home\env\lib\site-packages\torch\__init__.py", line 934, in <module>
    from .storage import _StorageBase, TypedStorage, _LegacyStorage, UntypedStorage, _warn_typed_storage_removal
  File "D:\Clouds\OneDrive\Documents\dev\github\openai\ChatGPT-at-Home\env\lib\site-packages\torch\storage.py", line 230, in <module>
    class UntypedStorage(torch._C.StorageBase, _StorageBase):
TypeError: 'torch._C._TensorMeta' object is not iterable

Running on -
# Python 3.10.0b2
# GNU bash, version 5.1.16(1)-release (x86_64-pc-msys)
# VS Code Version: 1.76.2 (system setup)
Commit: ee2b180d582a7f601fa6ecfdad8d9fd269ab1884
Date: 2023-03-14T17:55:54.936Z
Electron: 19.1.11
Chromium: 102.0.5005.196
Node.js: 16.14.2
V8: 10.2.154.26-electron.0
OS: Windows_NT x64 10.0.19045
Sandboxed: No

# Hardware - Nitro AN515-43
Processor	AMD Ryzen 5 3550H with Radeon Vega Mobile Gfx     2.10 GHz
Installed RAM	40.0 GB (37.9 GB usable)

# OS
Edition	Windows 10 Home Single Language
Version	22H2
OS build	19045.2728
Experience	Windows Feature Experience Pack 120.2212.4190.0


### Versions

$ python collect_env.py
Traceback (most recent call last):
  File "D:\Clouds\OneDrive\Documents\dev\github\openai\ChatGPT-at-Home\collect_env.py", line 15, in <module>
    import torch
  File "D:\Clouds\OneDrive\Documents\dev\github\openai\ChatGPT-at-Home\env\lib\site-packages\torch\__init__.py", line 934, in <module>
    from .storage import _StorageBase, TypedStorage, _LegacyStorage, UntypedStorage, _warn_typed_storage_removal
  File "D:\Clouds\OneDrive\Documents\dev\github\openai\ChatGPT-at-Home\env\lib\site-packages\torch\storage.py", line 230, in <module>
    class UntypedStorage(torch._C.StorageBase, _StorageBase):
TypeError: 'torch._C._TensorMeta' object is not iterable

cc @peterjc123 @mszhanyi @skyline75489 @nbcsm @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport
dynamo sometimes hits the cache size limit due to the foreach flag in optimizer.step()
Labels: module: optimizer, triaged, oncall: pt2, module: dynamo

### 🐛 Describe the bug

When one torch.compile()'s the optimizer.step() for certain models and optimizers, dynamo sometimes hits the cache size limit due to the foreach flag. It should not try recompiling a graph for the foreach flag currently since we should just be tracing single-tensor for now. This will likely be fixed by @mlazos's foreach + optimizer changes and this issue is a tracker to make sure it is not forgotten. 

### Error logs

```
Warning: 27 21:42:30,628] torch._dynamo.convert_frame: [WARNING] torch._dynamo hit config.cache_size_limit (64)
   function: 'nadam' (/home/ubuntu/miniconda3/envs/userbenchmarks-ci/lib/python3.10/site-packages/torch/optim/nadam.py:160)
   reasons:  ___check_obj_id(foreach, 7698112)
to diagnose recompilation issues, see https://pytorch.org/docs/master/compile/troubleshooting.html.
```

### Repro

I'm not sure this is easy to repro in a small way, but this happens when I run my benchmarking script from https://github.com/pytorch/benchmark/pull/1473 with `python -m userbenchmark.optim.__init__`. 

### Versions

2.0, nightlies

cc @vincentqb @jbschlosser @albanD @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @voznesenskym @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire @lazos
Compile targts cuda:0 rather than the device the model is on
Labels: triaged, oncall: pt2, module: inductor

### 🐛 Describe the bug

The pytorch compile works great when I only have one GPU (gtx 1660 super, no tensor cores) installed on my computer. However, when I installed a new GPU card RTX 3060 (has tensor cores), which becomes the new “cuda:0”, even though I still chose gtx 1660 (“cuda:1” now) to run the model, the pytorch compile still choose rtx 3060, which has tensor cores as the targeted archecture. And I got the following warning:

```
compile_fx.py:90: UserWarning: TensorFloat32 tensor cores for float32 matrix multiplication available but not enabled. Consider setting `torch.set_float32_matmul_precision('high')` for better performance.
```
and error:
```
compiler.py", line 1671, in _init_handles

mod, func, n_regs, n_spills = cuda_utils.load_binary(self.metadata["name"], self.asm["cubin"], self.shared, device)

RuntimeError: Triton Error [CUDA]: device kernel image is invalid
```

Would be great if the compile function can either detect which GPU the model is on or has a device option.

### Versions

pytorch 2.0
cuda 11.8

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @voznesenskym @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire
[FSDP] Consolidate test_fsdp_state_dict.py
Labels: oncall: distributed, triaged, module: fsdp

### 🚀 The feature, motivation and pitch

One issue today is that our test_fsdp_state_dict.py file was incrementally developed and now there is a nontrivial amount of redundancy between tests. It would be great if we can consolidate the tests to reduce time to signal and also improve developer efficiency (e.g. if adding a new feature, which test(s) should I test my feature with).



### Alternatives

_No response_

### Additional context

_No response_

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu
[FSDP] test model.eval() + keep_low_precision_grads
Labels: oncall: distributed, triaged

### 🚀 The feature, motivation and pitch

Need to test that keep_low_precision_grads in mixed_precision is nullified in model.eval() mode.

### Alternatives

_No response_

### Additional context

_No response_

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu
How do I get the original object wrapped by the torch.fx.proxy class？
Labels: triaged, oncall: fx

I want quant the swin transformers. This model need get some shape of tensors in the forward function, but the args is a proxy object, How do I get the original object(It is the window tensor below) wrapped by the Proxy class？Thanks.

## Issue description

Provide a short description.

## Code example
![image](https://user-images.githubusercontent.com/49281157/227918528-3db1ef62-0c41-483e-953e-f1c468224a5b.png)
![image](https://user-images.githubusercontent.com/49281157/227918593-0d700138-2112-472d-a575-b34ad48e4965.png)





## Traceback
**the windows is a tensor wrapped by the proxy, Is there some method to get the original tensor or its shape?**

![image](https://user-images.githubusercontent.com/49281157/227918682-1576ab9d-0dc7-445f-a32f-0f3372850a17.png)


## System Info

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] torch==1.12.0+cu116
[pip3] torchaudio==0.12.0+cu116
[pip3] torchvision==0.13.0+cu116
[conda] blas                      1.0                         mkl    defaults
[conda] cudatoolkit               11.3.1               h2bc3f7f_2    defaults
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.4.0           h06a4308_640    defaults
[conda] mkl-service               2.4.0            py38h7f8727e_0    defaults
[conda] mkl_fft                   1.3.1            py38hd3c417c_0    defaults
[conda] mkl_random                1.2.2            py38h51133e4_0    defaults
[conda] numpy                     1.23.3           py38h14f4228_0    defaults
[conda] numpy-base                1.23.3           py38h31eccc5_0    defaults
[conda] pytorch                   1.13.0              py3.8_cpu_0    pytorch
[conda] pytorch-mutex             1.0                         cpu    pytorch
[conda] torch                     1.12.0+cu116              <pip>
[conda] torchaudio                0.12.0+cu116              <pip>
[conda] torchaudio                0.13.0                 py38_cpu    pytorch
[conda] torchvision               0.14.0                 py38_cpu    pytorch
[conda] torchvision               0.13.0+cu116              <pip>

cc @ezyang @SherlockNoMad @EikanWang @jgong5 @wenzhe-nrv
transposed 2d copy bfloat16 support
Labels: triaged, intel

              @zhuhaozhe BTW, transposed 2d copy only supports float currently. Will you add bfloat16 support later?

_Originally posted by @jgong5 in https://github.com/pytorch/pytorch/issues/97147#issuecomment-1484523680_
            

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10
torch.onnx.export support sparse tensor format
Labels: module: onnx, triaged

### 🚀 The feature, motivation and pitch

I have a simply model just like a conv2d with a sparse weight in my case. The forward like following

```
# self.weight is sparsed with coo format
x = torch.sparse.mm(self.weight, x)
```


When i want to convert to onnx model. This error occur.


```
    tuple(x.clone(memory_format=torch.preserve_format) for x in args)
RuntimeError: unsupported memory format option Preserve
```


I think maybe torch.onnx.export module doesn't support SparseTensor?

### Alternatives

_No response_

### Additional context

_No response_
Regression in jit for f-strings with new lines
Labels: oncall: jit, triaged

### 🐛 Describe the bug

Consider the following code. It executes without error on v1.13.1 and python3.7

```python
import torch

@torch.jit.script
def xxx(
    box_preds: torch.Tensor, cls_scores: torch.Tensor, cls_labels: torch.Tensor
) -> None:
    assert (
        1 == 2
    ), f'"cls_labels" should have the same shape as "cls_scores", but got {cls_labels.shape} and \
        {cls_scores.shape}'

print(xxx)
```

But it fails with the following error on v1.13.1 and python3.9

```
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
[<ipython-input-4-7faa743d235b>](https://localhost:8080/#) in <module>
      2 
      3 @torch.jit.script
----> 4 def xxx(
      5     box_preds: torch.Tensor, cls_scores: torch.Tensor, cls_labels: torch.Tensor
      6 ) -> None:

11 frames
[/usr/local/lib/python3.9/dist-packages/torch/jit/frontend.py](https://localhost:8080/#) in get_char(index)
    730 
    731         def get_char(index):
--> 732             return chr(source[index])
    733 
    734         start_pos = base.range().end + 1

IndexError: index out of range
```

### Versions

v1.13.1

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel
torch.zeros_like on a zero-sized BSR/BSC tensor results invalid tensor
Labels: module: sparse, triaged, module: correctness (silent)


## Issue description

As in the title.

## Code example

```python
>>> torch.sparse.check_sparse_tensor_invariants.enable()
>>> x=torch.sparse_bsr_tensor([0, 3], [0, 1, 2], torch.arange(12).reshape(3, 2, 1, 2))
>>> torch._validate_sparse_bsr_tensor_args(x.crow_indices(), x.col_indices(), x.values(), x.shape)
>>> z=torch.zeros_like(x)
>>> z
tensor(crow_indices=tensor([0, 0]),
       col_indices=tensor([], size=(0,)),
       values=tensor([], size=(0, 2, 2, 1)), size=(2, 3, 2), nnz=0,
       layout=torch.sparse_bsr)
>>> torch._validate_sparse_bsr_tensor_args(z.crow_indices(), z.col_indices(), z.values(), z.shape)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: tensor shape[1] (=3) must be divisible with blocksize[1] (=2) as defined by values shape
```

The expected result is
```python
>>> z=torch.sparse_bsr_tensor(torch.zeros(x.crow_indices().shape, dtype=torch.int64), torch.zeros(0, dtype=torch.int64), torch.zeros(0, *x.values().shape[1:]), x.shape)
>>> torch._validate_sparse_bsr_tensor_args(z.crow_indices(), z.col_indices(), z.values(), z.shape)
>>> z
tensor(crow_indices=tensor([0, 0]),
       col_indices=tensor([], size=(0,)),
       values=tensor([], size=(0, 2, 1, 2)), size=(2, 3, 2), nnz=0,
       layout=torch.sparse_bsr)
```


## System Info

- PyTorch version: master


cc @alexsamardzic @nikitaved @cpuhrsch @amjames @bhosmer
MPS: grid_sampler_2d falls back to CPU, even though warning says it is natively supported on macOS >=13.1
Labels: triaged, module: mps

### 🐛 Describe the bug

The following script runs with a warning that the grid_sampler_2d op is falling back to CPU, but is supported natively on MPS with macOS >=13.1:

```python
import torch
import torchvision.transforms as transforms

img = torch.rand([1, 1, 256, 256], device="mps");
transform = transforms.RandomPerspective();
img = transform(img);
```

I ran the script on macOS 13.3, so it should run without warnings, as the op is apparently supported on versions equal to or above 13.1.

Either the warning is wrong about what versions it is supported on, or the compatibility check is not working correctly. I'll look into it and make a PR if I can figure it out.

### Versions

PyTorch version: 2.1.0.dev20230317
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.3 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.202)
CMake version: version 3.25.2
Libc version: N/A

Python version: 3.10.8 (main, Nov 24 2022, 08:08:27) [Clang 14.0.6 ] (64-bit runtime)
Python platform: macOS-13.3-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Pro

Versions of relevant libraries:
[pip3] numpy==1.24.1
[pip3] pytorch-lightning==1.8.6
[pip3] torch==2.1.0.dev20230317
[pip3] torchaudio==2.0.0.dev20230317
[pip3] torchmetrics==0.11.0
[pip3] torchvision==0.16.0.dev20230317
[conda] numpy                     1.24.1                   pypi_0    pypi
[conda] pytorch-lightning         1.8.6                    pypi_0    pypi
[conda] torch                     2.1.0.dev20230317          pypi_0    pypi
[conda] torchaudio                2.0.0.dev20230317          pypi_0    pypi
[conda] torchmetrics              0.11.0                   pypi_0    pypi
[conda] torchvision               0.16.0.dev20230317          pypi_0    pypi

cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
Insufficient MPS Documentation
Labels: module: docs, triaged, module: mps

### 📚 The doc issue

Whenever I try to have a look at MPS documentation for PyTorch, it is really difficult to figure it out and I couldn't find much in my way. On the other hand, when I look for CUDA documentation, every detail was set so that it can be understandable for everyday user. Checked that I have a functioning MPS.

### Suggest a potential alternative/fix

Creating a comprehensive documentation for MPS would be great.

cc @svekars @carljparker @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
can get_submodule be called within a ScriptFunction ?
Labels: oncall: jit, triaged

### 🚀 The feature, motivation and pitch

I'm writing a multi domain classifier where can first predict domains of a sentence, and then choose the corresponding  sequence classifer to run. It can certainly save FPOP in inference stage.

```
class MultiDomainClassifier(nn.Module):
    def __init__(self, classifiers: nn.ModuleDict, num_domain):
        super().__init__()
        self.classifiers = nn.ModuleList(classifiers.children())
        self.num_domain = num_domain

    def forward(self, inputs: Tensor, output_domains: Tensor) -> list[Tensor]:
        return [self.classifiers.get_submodule(str(idx)).forward(inputs) for idx in output_domains]
```

But I got this:

```
RuntimeError: 
Unknown type name 'Module':
  File "/home/luka.bao/anaconda3/envs/cpu/lib/python3.9/site-packages/torch/nn/modules/module.py", line 617
    def get_submodule(self, target: str) -> "Module":
                                            ~~~~~~~ <--- HERE
        """
        Returns the submodule given by ``target`` if it exists,
'ModuleList.get_submodule' is being compiled since it was called from 'MultiDomainClassifier.forward'
  File "/home/luka.bao/PycharmProjects/multitask-nlu-torch/torch_impl/output/classifier.py", line 109
    def forward(self, inputs: Tensor, output_domains: Tensor) -> list[Tensor]:
        return [self.classifiers.get_submodule(str(idx)).forward(inputs) for idx in output_domains]
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
```


### Alternatives

_No response_

### Additional context

_No response_

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel
Multi-output derivative formulas can save unnecessary tensors
Labels: module: autograd, triaged, module: nestedtensor, actionable

Usually when you have a derivative formula, e.g. `mm`:
```
- name: mm(Tensor self, Tensor mat2) -> Tensor
  self: mm_mat1_backward(grad, mat2, self.sym_sizes(), self.sym_strides(), self.layout(), 1)
  mat2: mm_mat2_backward(grad, self, mat2.sym_sizes(), mat2.sym_strides(), mat2.layout(), 1)
  result: at::mm(self_t, mat2_p) + at::mm(self_p, mat2_t)
```
The structure informs autograd codegen what tensors need to be saved to compute which gradients, i.e. if only `self` requires grad, I only need to save `mat2`, and if only `mat2` requires grad, I only need to save `self`. 

In VariableType, the following logic is generated for `mm`:
```cpp
if (_any_requires_grad) {
    grad_fn = std::shared_ptr<MmBackward0>(new MmBackward0(), deleteNode);
    grad_fn->set_next_edges(collect_next_edges( self, mat2 ));
    if (grad_fn->should_compute_output(1)) {
      grad_fn->self_ = SavedVariable(self, false);
    }
  ...
    if (grad_fn->should_compute_output(0)) {
      grad_fn->mat2_ = SavedVariable(mat2, false);
    }
}
```

However, when you have a single derivative formula that produces multiple outputs, autograd codegen no longer has visibility into what tensors need to be saved in order to compute which gradients. 

```
- name: matmul(Tensor self, Tensor other) -> Tensor
  self, other: matmul_backward(grad, self, other, grad_input_mask)
```
This yields the following unoptimized logic in autograd kernel:
```
 if (_any_requires_grad) {
    grad_fn = std::shared_ptr<MatmulBackward0>(new MatmulBackward0(), deleteNode);
    grad_fn->set_next_edges(collect_next_edges( self, other ));
    grad_fn->self_ = SavedVariable(self, false);
    grad_fn->other_ = SavedVariable(other, false);
  }
```
One common case where this can matter is if I'm doing some fine-tuning, e.g. I want to requires_grad=False for the parameters of a bunch of layers in the middle of a network.

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @Lezcano @Varal7 @cpuhrsch @jbschlosser @bhosmer @drisspg @gchanan
InfoNCE loss for contrastive learning
Labels: module: loss, triaged, enhancement

### 🚀 The feature, motivation and pitch

In the last couple of years contrastive self-supervised learning has been gaining popularity, as it allows one to use large amounts of unlabeled data to (pre) train a model. A loss function which is commonly used is the [InfoNCE](https://arxiv.org/abs/1807.03748) loss, having 4620 citations according to scholar. The loss is also used by [MoCo](https://arxiv.org/abs/1911.05722), which has 6416 citations, and [SimCLR](https://arxiv.org/abs/2002.05709), which has 8988 citations. I think this loss would be a good addition to PyTorch considering the popularity and the fact that a similar loss is not yet in PyTorch. 

### Alternatives

There are slight variations in the literature. MoCo added the temperature, which wasn't present in the original definition. SimCLR uses more negative examples from both 'views'. But at it's core there is little difference between these, as the loss is still about applying softmax on the similarity of embeddings. 

Other large papers in this field are [BYOL](https://arxiv.org/pdf/2006.07733.pdf) and [SimSiam](https://arxiv.org/abs/2011.10566), which calculate a mean squared error of embeddings, and [SwAV](https://arxiv.org/abs/2006.09882), which at a glance seems to use a similar loss although I haven't read the paper in detail yet.

### Additional context

For context: I have a repository with an implementation of this loss. Someone opened an issue there asking me to try to contribute it to PyTorch, hence this feature request. The goal of this feature request is to get the InfoNCE loss into PyTorch and not to get my implementation of it in particular in PyTorch. Although, I have no objection to that of course. I genuinely think that this loss, or a variant thereof, will make a valuable contribution.
[DTensor] Add a unittest to cover default PG condition for DeviceMesh
Labels: oncall: distributed, triaged

### 🚀 The feature, motivation and pitch

There is no UT to cover the default PG condition for DeviceMesh. The detail of the past breakages can be found in https://github.com/pytorch/pytorch/pull/97384. A UT is required to cover this case.

### Alternatives

_No response_

### Additional context

_No response_

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu
Improve collectives fingerprinting
Labels: good first issue, triaged, module: c10d

### 🚀 The feature, motivation and pitch

When using `TORCH_DISTRIBUTED_DEBUG=DETAIL` we collect collectives fingerprints and those are quite helpful when troubleshooting issues like stragglers.

One recurring problem in distributed jobs are stragglers and, in special, those triggered by python GC activity. We should extend CollectiveFingerPrint to include two pieces of information: python gc counts (for all 3 gens) and some monotonic clock source.

Those would enable us us to detect such issues as part of `TORCH_DISTRIBUTED_DEBUG=DETAIL`.

One complication of this idea is that we currently compare fingerprints in a bitwise fashion, which won't work since some of this information is just advisory.



### Alternatives

_No response_

### Additional context

_No response_
Change progressbar for hub
Labels: triaged, module: hub

### 🚀 The feature, motivation and pitch

i am changed progressbar for hub to use rich.progress, its more beautiful
![image](https://user-images.githubusercontent.com/15774585/227221613-40f002c1-45f0-45af-8722-2dbf97fa68a8.png)


### Alternatives

_No response_

### Additional context

I did it for myself, if you're interested - my pr https://github.com/pytorch/pytorch/pull/97438

cc @nairbv @NicolasHug @vmoens @jdsgomes
suspicious memory leak when increase DataLoader's prefetch_factor and enable pin_memory
Labels: module: dataloader, module: memory usage, triaged

### 🐛 Describe the bug

In my 8*V100 GPU training project I use torch's DataLoader and set partial arguments below.

`train_loader = torch.utils.data.DataLoader(train_dataset,
                    batch_size=8,
                    sampler=train_sampler,
                    num_workers=16,
                    pin_memory=True,
                    prefetch_factor=16,
                    persistent_workers=True)`

But while training I found the host memory cost too much(nearly 90% of total 768GB RAM), from our node's monitor.
![image](https://user-images.githubusercontent.com/8710725/227138898-bf22fe17-c100-491f-980e-484ac2eb0181.png)

When I set prefetch_factor back to 2 (default value), the memory problem disappeared.

Additional, I used NVIDIA Nsight Systems tool to profile my training job and found many cudaHostAlloc API calls even in 2000th+ steps, without any corresponding cudaFree calls, which also blocked execution of GPU Kernels.
![image](https://user-images.githubusercontent.com/8710725/227154210-afe12e86-85c0-4186-9e42-ec28deec4d01.png)

In further analysis, I found these cudaHostAlloc were called from pin_memory_loop thread in DataLoader.
So I doubt there may be some pinned memory leak bugs? Need help from other developpers .


### Versions

torch.__version__ 1.13.1+cu117
torch.version.cuda : 11.7
torch.backends.cudnn.version() : 8500
python version : 3.7.13 (miniconda)
cuda driver version : 470.57.02
GPU : Tesla V100 * 8

cc @SsnL @VitalyFedyunin @ejguan @NivekT @dzhulgakov
After the release of pytorch 2.0.0, the compilation of ACLs is problematic.
Labels: module: build, triaged

I found that when I installed torch2.0.0 using pip, I carried a torch.libs folder with three ACL so files and two libgomp files. I wanted to know the detailed compilation options and ACL versions of the three ACL so files. What are the sources of the other two libgomp files?
libarm_compute-860e7a78.so
libarm_compute_graph-ab857ce5.so  
libgomp-d22c30c5.so.1.0.0
libarm_compute_core-0793f69d.so  
libgomp-adceddde.so.1.0.0

As for why I want to know this, I want to compile pytorch in the source code to link my self-installed ACLs and find that it is much slower than the torch installed using pip, under the same version of torch.

My Compilation Environment:
GCC 10.2
ACL 23.0.1
torch 2.0.0

cc @malfet @seemethere
streamNone = get_cuda_stream(None) RuntimeError: invalid argument to getCurrentStream
Labels: triaged, oncall: pt2, module: inductor

### 🐛 Describe the bug

Steps to repro:

Patch in https://github.com/pytorch/pytorch/pull/97275/

```
PYTORCH_TEST_WITH_INDUCTOR=1 python test/test_torch.py -k test_upsample_nearest2d_meta
```

fails with

```
======================================================================
ERROR: test_upsample_nearest2d_meta (__main__.TestTorch)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/users/ezyang/c/pytorch/test/test_torch.py", line 7594, in test_upsample_nearest2d_meta
    def test_upsample_nearest2d_meta(self):
  File "/data/users/ezyang/c/pytorch/torch/_dynamo/eval_frame.py", line 235, in _fn
    return fn(*args, **kwargs)
  File "/data/users/ezyang/c/pytorch/torch/_functorch/aot_autograd.py", line 3067, in forward
    return compiled_fn(full_args)
  File "/data/users/ezyang/c/pytorch/torch/_functorch/aot_autograd.py", line 1187, in g
    return f(*args)
  File "/data/users/ezyang/c/pytorch/torch/_functorch/aot_autograd.py", line 2098, in runtime_wrapper
    all_outs = call_func_with_args(
  File "/data/users/ezyang/c/pytorch/torch/_functorch/aot_autograd.py", line 1212, in call_func_with_args
    out = normalize_as_list(f(args))
  File "/tmp/torchinductor_ezyang/cr/ccratlcqupckmlekrr3wiwo4xs77rfbn2x47ii6wi3722dmjv52e.py", line 61, in call
    streamNone = get_cuda_stream(None)
RuntimeError: invalid argument to getCurrentStream

----------------------------------------------------------------------
```

I don't see any user reports of this error so maybe low priority.

Generated code: https://gist.github.com/ezyang/199c5be44c0744f571620bfbc7d5993d

ATen graph looks valid, shouldn't be a bug in the DCE PR:

```
    def forward(self):
        # File: /data/users/ezyang/c/pytorch/test/test_torch.py:7602, code: x = torch.empty(4, 3, 8, 8, device='meta')
        empty: f32[4, 3, 8, 8] = torch.ops.aten.empty.memory_format([4, 3, 8, 8], device = device(type='meta'), pin_memory = False)
        
        # File: /data/users/ezyang/c/pytorch/test/test_torch.py:7604, code: torch._C._nn.upsample_nearest2d(x, (16, 16), out=out)
        upsample_nearest2d: f32[4, 3, 16, 16] = torch.ops.aten.upsample_nearest2d.default(empty, [16, 16])
        return (empty, upsample_nearest2d)
```

### Versions

master

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh @voznesenskym @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire
Further memcopy improvement at FX body level
Labels: triaged

              We may consider to improve this further as the next step. In general, we don't have to care about what data type is when we do memory copy. Maybe we can do some pattern matching on the kernel fx body to figure out if it is a memory copy and just apply a common memory copy intrinsics.

_Originally posted by @jgong5 in https://github.com/pytorch/pytorch/pull/97147#discussion_r1145540933_
DISABLED test_checkpoint_trigger (__main__.TestCheckpoint)
Labels: triaged, module: flaky-tests, skipped, module: unknown

Platforms: dynamo

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_checkpoint_trigger&suite=TestCheckpoint) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/12205638073).

Over the past 3 hours, it has been determined flaky in 7 workflow(s) with 14 failures and 7 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_checkpoint_trigger`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_utils.py`
Import fails when both `USE_TENSORPIPE=OFF` and `USE_DISTRIBUTED=ON`.
Labels: module: build, triaged, module: tensorpipe

### 🐛 Describe the bug

Installation succeeds, but importing torch fails when pytorch is built with both `USE_TENSORPIPE=OFF` and `USE_DISTRIBUTED=ON`.

Error Message:
```
>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "lib/python3.10/site-packages/torch/__init__.py", line 831, in <module>
    from .functional import *  # noqa: F403
  File "lib/python3.10/site-packages/torch/functional.py", line 8, in <module>
    import torch.nn.functional as F
  File "lib/python3.10/site-packages/torch/nn/__init__.py", line 1, in <module>
    from .modules import *  # noqa: F403
  File "lib/python3.10/site-packages/torch/nn/modules/__init__.py", line 2, in <module>
    from .linear import Identity, Linear, Bilinear, LazyLinear
  File "lib/python3.10/site-packages/torch/nn/modules/linear.py", line 7, in <module>
    from .. import functional as F
  File "lib/python3.10/site-packages/torch/nn/functional.py", line 18, in <module>
    from .._jit_internal import boolean_dispatch, _overload, BroadcastingList1, BroadcastingList2, BroadcastingList3
  File "lib/python3.10/site-packages/torch/_jit_internal.py", line 39, in <module>
    import torch.distributed.rpc
  File "python3.10/site-packages/torch/distributed/rpc/__init__.py", line 30, in <module>
    from torch._C._distributed_rpc import (
ImportError: cannot import name '_TensorPipeRpcBackendOptionsBase' from 'torch._C._distributed_rpc' (unknown location)
```

Missing symbols from that list include: `_TensorPipeRpcBackendOptionsBase`,
`TensorPipeAgent`, and `_DEFAULT_NUM_WORKER_THREADS`.

`grep TensorPipeRpcBackendOptionsBase lib/python3.10/site-packages/torch/lib/libtorch_python.so` turns up no matches.

Diagnosis: The bug's existence can be read off directly from the source code, since the `torch._C._distributed_rpc` imports those unconditionally, but https://github.com/pytorch/pytorch/blob/v1.13.1/torch/csrc/distributed/rpc/init.cpp#L532 injects _TensorPipeRpcBackendOptionsBase and some other things during `rpc_init()`, but only if `USE_TENSORPIPE` is defined.


### Versions

This issue has been noticed before #68002 
The TensorPipe python import first appeared in 1.8.0.
The bug persists in v2.0.0:

https://github.com/pytorch/pytorch/blob/v2.0.0/torch/csrc/distributed/rpc/init.cpp#L532

https://github.com/pytorch/pytorch/blob/v2.0.0/torch/distributed/rpc/__init__.py#L54

cc @malfet @seemethere @osalpekar @jiayisuse @lw @beauby @pritamdamania87 @mrshenli @jjlilley @gqchen @rohan-varma