gopikrsmscs/torch-issues huggingface dataset:
Adding batched CSR tensors with different sparsities produces an invalid tensor
Labels: module: sparse, triaged, module: correctness (silent), bug


## Issue description

As in the title.

Adding batched CSR tensors with the same sparsities produces correct results.
Also, adding CSR tensors without batch dimensions and different sparsities produce correct results.

## Code example

```python
>>> a = torch.ones(2, 2, 2); a[:, :, 0] = 0; a = a.to_sparse_csr()
>>> b = torch.ones(2, 2, 2).to_sparse_csr()
>>> c = a + b
>>> c
tensor(crow_indices=tensor([[0, 2, 4],
                            [0, 1, 2]]),
       col_indices=tensor([0, 1, 0, 1]),
       values=tensor([1., 2., 1., 2.]), size=(2, 2, 2), nnz=4,
       layout=torch.sparse_csr)
>>> torch._validate_sparse_csr_tensor_args(c.crow_indices(), c.col_indices(), c.values(), c.shape)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: crow_indices and col_indices dimensionalities must be equal but got 2 and 1, respectively
```

The expected result is:
```python
>>> (a.to_dense() + b.to_dense()).to_sparse_csr()
tensor(crow_indices=tensor([[0, 2, 4],
                            [0, 2, 4]]),
       col_indices=tensor([[0, 1, 0, 1],
                           [0, 1, 0, 1]]),
       values=tensor([[1., 2., 1., 2.],
                      [1., 2., 1., 2.]]), size=(2, 2, 2), nnz=4,
       layout=torch.sparse_csr)
```

## System Info

- PyTorch version: main

cc @alexsamardzic @nikitaved @cpuhrsch @amjames @bhosmer
[ROCm] Add gcnArchName to collect_env and torch.cuda.get_device_properties
Labels: module: rocm, triaged, open source, ciflow/trunk, release notes: rocm, rocm

Printing just the device name is not helpful when investigating PyTorch issues filed for specific AMD GPUs, as the support/issue might depend on the gfx arch, which is part of the gcnArchName property.

`torch.cuda.get_device_properties(0).gcnArchName` will print the value of the `gcnArchName` property: eg. 
```
>>> torch.cuda.get_device_properties(0).gcnArchName
'gfx906:sramecc+:xnack-'
```

```
root@6f064e3c19fb:/data/pytorch/test# python ../torch/utils/collect_env.py
...
GPU models and configuration: AMD Radeon Graphics(gfx906:sramecc+:xnack-)
```

cc @jeffdaily @sunway513 @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang
Reference cycles involving code -> co_extra -> compiled output -> reference to code
Labels: triaged, oncall: pt2, module: dynamo

### 🐛 Describe the bug

By default, libraries which hook into the frame evaluation API are encouraged to store compiled products on co_extra. However, storing arbitrary Python objects on code objects is a recipe for reference cycles, because most people assume that holding a reference to a code object is "safe" (unlikely to result in a cycle); e.g., that it is safe to store a CapturedTraceback on a compiled code object. I recently ran into this on the stack at https://github.com/pytorch/pytorch/pull/107457 where I wanted to store debugging tracebacks on internal data structures in PT2; I only noticed because we had two bugs in GC traverse/clear and this resulted in code getting leaked entirely.

I'm not sure how to 100% solve this problem.

* Stop storing compiled products in co_extra, store them somewhere else. However, if you torch.compile a global function that takes only Tensor inputs, there is no where else to store it, the code object really is your only logical choice.
* Stop storing so much stuff in our compiled products. This might be plausible but we'll need to take a close look at our object graph on the compiled code and see what can be dropped and what can't
* Something else?

cc @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov 

### Versions

main
Conversion from COO with two sparse dimensions to CSR with dense_dim specified fails
Labels: module: sparse, feature, triaged, bug

## Issue description

As in the title. Not specifying `dense_dim` in the conversion leads to an expected result.

## Code example

```python
>>> torch.ones(2, 2, 2).to_sparse(2).to_sparse(layout=torch.sparse_csr, dense_dim=1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: sparse_coo_to_sparse: conversion from Sparse to SparseCsr with dense_dim argument given is not supported
```

The expected behavior is no exception and with the following result:
```python
>>> torch.ones(2, 2, 2).to_sparse(2).to_sparse(layout=torch.sparse_csr)
tensor(crow_indices=tensor([0, 2, 4]),
       col_indices=tensor([0, 1, 0, 1]),
       values=tensor([[1., 1.],
                      [1., 1.],
                      [1., 1.],
                      [1., 1.]]), size=(2, 2, 2), nnz=4,
       layout=torch.sparse_csr)
```

## System Info

- PyTorch version: main

cc @alexsamardzic @nikitaved @cpuhrsch @amjames @bhosmer
[testing] dynamo testing: we should call `dynamo.reset` before running each test with dynamo.
Labels: module: tests, triaged, oncall: pt2, module: dynamo

### 🐛 Describe the bug

Running dynamo tests without calling `reset` allows tests to pass which would fail if they were run standalone.

Example: https://github.com/pytorch/pytorch/issues/107050#issuecomment-1680756057

cc: @zou3519

### Versions

main

cc @mruberry @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng @ipiszy
Determinism by using datapipes shuffle
Labels: module: dataloader, triaged, module: determinism

### 🐛 Describe the bug

I encountered a problem where iterating twice caused the dataset to lose determinism.

```python
dp1 = IterableWrapper(range(10)).shuffle().set_seed(1) 
dp2 = IterableWrapper(range(10)).shuffle().set_seed(1)

# First iteration
all_match1 = all(x == y for x, y in zip(dp1, dp2))  # true
# Second iteration
all_match2 = all(x == y for x, y in zip(dp1, dp2))  # false
```
After commenting out self._seed to None, it runs correctly.

### Versions

PyTorch version: 2.0.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.4 (x86_64)
GCC version: Could not collect
Clang version: 14.0.3 (clang-1403.0.22.14.1)
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.12 (main, Jul  5 2023, 15:34:07) [Clang 14.0.6 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M2 Pro

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.25.1
[pip3] pytorch-lightning==2.0.5
[pip3] torch==2.0.1
[pip3] torchdata==0.6.1
[pip3] torchmetrics==1.0.1
[conda] numpy                     1.25.1                   pypi_0    pypi
[conda] pytorch-lightning         2.0.5                    pypi_0    pypi
[conda] torch                     2.0.1                    pypi_0    pypi
[conda] torchdata                 0.6.1                    pypi_0    pypi
[conda] torchmetrics              1.0.1                    pypi_0    pypi

cc @SsnL @VitalyFedyunin @ejguan @dzhulgakov @mruberry @kurtamohler
[fbcode][RFC] Parallel fast cat on cpu in cat op
Labels: fb-exported

Summary:
The original implementation in D40433604 is not feasible to be directly upstreamed to OSS for two reasons:
- It is using folly executor, while I see all the intra-op parallelism in OSS repo is using torch parallel_for api (underlying is either native thread pool or simply OpenMP). I believe it is better to avoid additional dependencies there
- It requires some gflags directly specify a magic number of partitioning thresholds in terms of # of bytes. It is not clean

This diff is making use of the heuristic number (https://www.internalfb.com/code/fbsource/[297f89d5823451fe9f4b5985e6537e89a72bb75a]/fbcode/caffe2/aten/src/ATen/TensorIterator.h?lines=86) from PyTorch OSS repo to do a best effort partitioning based on element numbers with a rounding-up. It also uses torch parallel_for api for parallelism

Test Plan: Unit test and benchmarking, sending out for RFC

Differential Revision: D48286816
make backward function explicit in a layer which is a combination of some ops
Labels: module: nn, triaged, oncall: profiler

### 🚀 The feature, motivation and pitch

To get a top-down performance analysis, we'd like to know how the time is consumed in different components of the model. Currently, we are able to know the forward time of the layers, but have no separated time of the bwd time of different layers.

For example,  I have a layer (MyLayer) which is the combination of some pytorch standard OPs, see blow. I can add something (for example, record_function, or events at the beginning/end) in the forward function, and I can get the time of forward function. But, I'm unable to know how much time is consumed by the bwd of MyLayer. I can get the OP level profiling from pytorch profiler, but it is not easy to collect them back into MyLayer (many reasons, for example, there are other layers fwd+bwd may call the same OPs).
```
class MyLayer(torch.nn.Module):
    def __init__(self):
        super(MyLayer, self).__init__()
        self.mlp_up_proj = nn.Linear(args.hs, 4 * args.hs)
        self.mlp_act = nn.GELU(approximate='none')
        self.mlp_down_proj = nn.Linear(4*args.hs, args.hs)
        
    def forward(self, x):
        y1 = self.mlp_up_proj(x)
        y2 = self.mlp_act(y1)
        y3 = self.mlp_down_proj(y2)
        return y3
```

So, I tried to change MyLayer as below to show my idea. I know it does not work, it even crashes in the runtime. If we can modify the original MyLayer with least change to make the backward function explicit, we can do all kinds of profiling on it.
```
# it does not work
class MyLayerImpl(torch.autograd.Function):
    @staticmethod
    def forward(ctx, x, mlp_up_proj, mlp_act, mlp_down_proj):
        with torch.enable_grad():
            y1 = mlp_up_proj(x)
            y2 = mlp_act(y1)
            y3 = mlp_down_proj(y2)
        y4 = y3.detach()
        ctx.save_for_backward(x, y1, y2, y3)
        return y4

    @staticmethod
    def backward(ctx, dout, *args):
        x, y1, y2, y3 = ctx.saved_tensors
        din = y3.backward(dout)
        return din, None, None, None
    

class MyLayer(torch.nn.Module):
    def __init__(self):
        super(MyLayer, self).__init__()
        self.mlp_up_proj = nn.Linear(args.hs, 4 * args.hs)
        self.mlp_act = nn.GELU(approximate='none')
        self.mlp_down_proj = nn.Linear(4*args.hs, args.hs)
        
    def forward(self, x):
        return MyLayerImpl.apply(x, self.mlp_up_proj, self.mlp_act, self.mlp_down_proj)
```

### Alternatives

_No response_

### Additional context

_No response_

cc @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki @robieta @chaekit @aaronenyeshi @nbcsm @guotuofeng @guyang3532 @gaoteng-git @tiffzhaofb @dzhulgakov @davidberard98
DISABLED test_memory_format_nn_ConvTranspose1d_cuda_complex32 (__main__.TestModuleCUDA)
Labels: module: nn, triaged, module: flaky-tests, skipped

Platforms: linux

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_memory_format_nn_ConvTranspose1d_cuda_complex32&suite=TestModuleCUDA) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/15994731946).

Over the past 3 hours, it has been determined flaky in 3 workflow(s) with 6 failures and 3 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_memory_format_nn_ConvTranspose1d_cuda_complex32`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_modules.py`

cc @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki
DISABLED test_wait_i_6 (__main__.TestMultiThreadedWait)
Labels: oncall: distributed, module: flaky-tests, skipped

Platforms: linux

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_wait_i_6&suite=TestMultiThreadedWait) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/15994133440).

Over the past 3 hours, it has been determined flaky in 7 workflow(s) with 7 failures and 7 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_wait_i_6`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `distributed/test_store.py`

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu
DISABLED test_wait_i_5 (__main__.TestMultiThreadedWait)
Labels: oncall: distributed, module: flaky-tests, skipped

Platforms: linux

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_wait_i_5&suite=TestMultiThreadedWait) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/15994133440).

Over the past 3 hours, it has been determined flaky in 6 workflow(s) with 6 failures and 6 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_wait_i_5`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `distributed/test_store.py`

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu
[Test] add python binding for oneDNN Graph API
Labels: module: mkldnn, open source, Stale, release notes: jit

`import torch._C._onednn as llga`


cc @gujinghui @PenghuiCheng @XiaobingSuper @jianyuh @jgong5 @mingfeima @sanchitintel @ashokei @jingxu10 @min-jean-cho @yanbing-j @Guobing-Chen @Xia-Weiwen
max_pool1d, max_pool2d, max_pool3d Integers for cpu and cuda
Labels: triaged, actionable, module: pooling

### 🚀 The feature, motivation and pitch

max_pool1d Integers cuda and cpu, max_pool2d Integers cuda, max_pool3d Integers for cuda and cpu

currently max_pool2d integer is only supported for cpu https://github.com/pytorch/pytorch/pull/104225

### Alternatives

_No response_

### Additional context
probably i will work on it within the next couple of weeks.
Problems of torch.multinomial
Labels: module: distributions, module: docs, triaged, module: random

### 🐛 Describe the bug

There are few theoretical problems with sampling from multinomial distribution using `torch.multinomial`.

**Multinomial distribution**
x ~ Multinomial(n, p)
 

- n: the number of independent trials (e.g., n=100 rolls of a dice)

- K: the number of classes (e.g., K=6 for a dice, K=2 for a coin)
the outcome of each trial falls into one of K classes

- p: [p_1, p_2, …, p_k, …, p_K], probabability parameter, a vector 
p_k = the probability that the outcome of a single trial falls into class k 
p_1 + p_2 + ... + p_K = 1

- x: [x_1, x_2, …, x_k, …, x_K], a sample point, a vector
x_k = the number of trials for which the outcome falls into class k
x_1 + x_2 + … + x_K = n

**Problem 1**
The sample point x of multinomial distribution is a vector [x_1, x_2, …, x_K] as noted above.
However, `torch.multinomial(replacement=True)` generates "outcomes of n trials" instead of random variable x that consists of "the number of trial for which the outcome falls into class k". This is very confusing as exemplified here, https://discuss.pytorch.org/t/trying-to-understand-the-torch-multinomial/71643. 

**Example**
```python
import torch

p = torch.tensor([2/10, 3/10, 5/10])
n = 10
out = torch.multinomial(p, n, replacement=True)
print(out) # tensor([1, 2, 1, 2, 2, 2, 1, 2, 0, 2])
 
# User needs to convert out (the outcomes of n trials) 
# to random variable using method as suggested here, 
# https://discuss.pytorch.org/t/trying-to-understand-the-torch-multinomial/71643/4
# This leads to unsound sampling from multinomial distribution
# without requiring user's further understanding and care
x = out.unique(return_counts=True)[1]
print(x) # tensor([1, 3, 6])
```

**Problem 2**
`replacement` option should be removed. `torch.multinomial(replacement=False)` by default. Sampling without replacement makes no sense here.

**Example**
```python
import torch
 
p = torch.tensor([2/10, 3/10, 5/10])
n = 3
out = torch.multinomial(p, n)
print(out) # tensor([1, 0, 2])
 
x = out.unique(return_counts=True)[1]
print(x) # tensor([1, 1, 1])
```

1. By definition of multinomial probability distribution, each trial is independent. Thus, we can observe any outcomes (according to p_k) at any trials. If `replacement=False`, then each trial is not independent (e.g., we cannot observe eye 3 after we once observe eye 3).
2. By definition of multinomial probability distribution, n > 0 and there is no constraint on n. If `replacement=False`, then n is constrained to n <= K.
3. If `replacement=False`, the sample point x, [x_1, ..., x_K], is always [1, …, 1]. In other words, regardless of the values of the parameter p, [p_1, …, p_K], `torch.multinomial(replacement=False)` always behaves in a uniform fashion as if p were [1/K, …, 1/K].
 
Above reasons simply make no sense to multinomial distribution.


### Versions

master

cc @fritzo @neerajprad @alicanb @nikitaved @svekars @carljparker @pbelevich
Multiple runners shutdown for an autoupdate while still running jobs
Labels: triaged, ci: sev-mitigated


## Current Status
Mitigated

## Error looks like
Multiple jobs failing with an error similar to:

```
The self-hosted runner: i-0a28cb5549510555b lost communication with the server. Verify the machine is running and has a healthy network connection. Anything in your workflow that terminates the runner process, starves it for CPU/Memory, or blocks its network access can cause this error. 
``` 
[Example](https://github.com/pytorch/pytorch/actions/runs/5893231102/attempts/1)

Alternately, the job view simply says `This job failed` ([example](https://github.com/pytorch/pytorch/actions/runs/5893231102/job/15984768337))



## Incident timeline (all times pacific)
Approx 9:30am PST multiple runners shutdown for an auto update. Any jobs running on them were lost and appeared on github ci as a failure

## User impact
*How does this affect users of PyTorch CI?*

## Root cause

## Mitigation
Runners came back up after a restart

## Prevention/followups
Use this flag to tell the runners to not self-update: https://github.blog/changelog/2022-02-01-github-actions-self-hosted-runners-can-now-disable-automatic-updates/

And explore how to ensure the runners are not running any jobs before initiating an update.
[regression] Not getting `CUDA error: device-side assert triggered` on main for CUDA_KERNEL_ASSERT2
Labels: module: cuda, triaged, module: regression

### 🐛 Describe the bug

Motivated by the discussion here https://github.com/pytorch/pytorch/issues/106467#issuecomment-1677712857

Considering the following script,

```
import torch
print(torch.__version__)
import torch.nn as nn

torch.manual_seed(0)

x = torch.randn(10, 10, requires_grad=True, device="cuda")
y = torch.randint(0, 10, (10,), device="cuda")
criterion = nn.CrossEntropyLoss()

loss = criterion(x, y)
print(loss)
# tensor(2.8109, device='cuda:0', grad_fn=<NllLossBackward0>)

y[0] = 10
loss = criterion(x, y)
print(loss)
```

Setting `y[0] = 10` should trigger https://github.com/pytorch/pytorch/blob/4bfc55ba8b0a96a1c62069b703318a729f0cc9a9/aten/src/ATen/native/cuda/Loss.cu#L254

which is defined as such

https://github.com/pytorch/pytorch/blob/4bfc55ba8b0a96a1c62069b703318a729f0cc9a9/aten/src/ATen/native/cuda/Loss.cu#L165-L170

On 2.0.1 I accurately get 

```
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
```

On `main` this error is not raised and I get

```
2.1.0a0+git2da735c
tensor(2.8778, device='cuda:0', grad_fn=<NllLossBackward0>)
tensor(-2., device='cuda:0', grad_fn=<NllLossBackward0>)
```

### Versions

main

cc @ezyang @gchanan @zou3519 @ptrblck @ngimel @r-barnes
[LibTorch/iOS] Unknown custom class type quantized.Conv2dPackedParamsBase. Please ensure it is registered
Labels: triaged, oncall: mobile, module: ios

### 🐛 Describe the bug

I installed `LibTorch-Lite` on iOS using Cocoapods:

```ruby
platform :ios, '16.0'

use_frameworks!

target 'MyApp' do
	pod 'LibTorch-Lite'

	target 'MyAppTests' do
		pod 'LibTorch-Lite'
	end
end
```

I am able to build my app and use the model without any problems, however, whenever I tried to run my tests, it always fail with this error:

```
libc++abi: terminating due to uncaught exception of type torch::jit::ErrorReport: 
Unknown custom class type quantized.Conv2dPackedParamsBase. Please ensure it is registered.:
quantized::conv1d(Tensor qx, __torch__.torch.classes.quantized.Conv2dPackedParamsBase packed_weight, float output_scale, int output_zero_point) -> Tensor
                                                               ~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
```

Do you have any idea why I cannot run my tests? 

### Versions

  - LibTorch-Lite (1.13.0.1):
    - LibTorch-Lite/Core (= 1.13.0.1)
  - LibTorch-Lite/Core (1.13.0.1):
    - LibTorch-Lite/Torch
  - LibTorch-Lite/Torch (1.13.0.1)
DISABLED test_kineto_profiler_with_environment_variable (__main__.TestProfiler)
Labels: module: rocm, triaged, module: flaky-tests, skipped, oncall: profiler

Platforms: rocm

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/failure/test_kineto_profiler_with_environment_variable) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/15939234701).

Over the past 72 hours, it has flakily failed in 6 workflow(s).

**Debugging instructions (after clicking on the recent samples link):**
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Grep for `test_kineto_profiler_with_environment_variable`


Test file path: `profiler/test_profiler.py`

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang @robieta @chaekit @aaronenyeshi @nbcsm @guotuofeng @guyang3532 @gaoteng-git @tiffzhaofb @dzhulgakov @davidberard98
sparse_mask method ignores masked-in elements of sparse compressed input tensors
Labels: module: sparse, triaged, module: correctness (silent), bug

## Issue description

As in the title. The current behavior with sparse compressed inputs contradicts [sparse_mask](https://pytorch.org/docs/main/generated/torch.Tensor.sparse_mask.html#torch.Tensor.sparse_mask) documentation note: "The returned sparse tensor has the same indices as the sparse tensor mask, even when the corresponding values in self are zeros." as well as the current behavior with strided or sparse COO inputs.

## Code example

```python
>>> mask = torch.tensor([[0, 1], [1, 1]], dtype=torch.bool).to_sparse_csr()
>>> data = torch.tensor([[1, 2], [3, 0]]).to_sparse_csr()
>>> data.sparse_mask(mask)
tensor(crow_indices=tensor([0, 1, 2]),
       col_indices=tensor([1, 0]),
       values=tensor([2, 3]), size=(2, 2), nnz=2, layout=torch.sparse_csr)
```

The expected result is
```python
tensor(crow_indices=tensor([0, 1, 3]),
       col_indices=tensor([1, 0, 1]),
       values=tensor([2, 3, 0]), size=(2, 2), nnz=3, layout=torch.sparse_csr)
```
to meet the documented specification and the behavior with COO and strided inputs:
```python
>>> data.to_dense().sparse_mask(mask)
tensor(crow_indices=tensor([0, 1, 3]),
       col_indices=tensor([1, 0, 1]),
       values=tensor([2, 3, 0]), size=(2, 2), nnz=3, layout=torch.sparse_csr)
>>> data.to_sparse_coo().sparse_mask(mask)
tensor(crow_indices=tensor([0, 1, 3]),
       col_indices=tensor([1, 0, 1]),
       values=tensor([2, 3, 0]), size=(2, 2), nnz=3, layout=torch.sparse_csr)
```

## System Info

- PyTorch version: main

cc @alexsamardzic @nikitaved @cpuhrsch @amjames @bhosmer
[feature request] Support native ONNX export of FFT-related ops in opset17 (with `inverse=True`, it also includes inverse DFT)
Labels: module: onnx, triaged

### 🚀 The feature

Seems that more FFT-related ops are now supported by ONNX opset17 and onnxruntime:
- https://github.com/onnx/onnx/blob/main/docs/Operators.md#dft

It would be good to have torch.fft.rfft and friends to be natively exported without having to construct the basis manually.

I think this would also be import to torchaudio community

Also related: https://github.com/pytorch/pytorch/issues/107446

### Motivation, pitch

N/A

### Alternatives

_No response_

### Additional context

_No response_
Getting more human-readable input and output names in the onnx model exported by torch
Labels: module: onnx, triaged, oncall: pt2, module: export

### 🚀 The feature, motivation and pitch

In  pytorch, we use variable names with actual physical meaning for distinguishing the inputs and outputs of the model. The corresponding interface is provided in the torch.onnx.export interface to reflect this, but when `export_modules_as_functions` is used to package the nn.Module, the setting of `output_names` is not correctly reflected if the output of the graph is also the output of a function, as exemplified in the following.
~~~ python
import timm
import torch
import torch.nn as nn

class TestModule(nn.Module):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.conv = nn.Conv2d(3, 32, (3, 3))
        self.relu = nn.ReLU()
        self.pool = nn.AvgPool2d((3, 3), (2, 2))
        
    def forward(self, x):
        x = self.conv(x)
        x = self.relu(x)
        x = self.pool(x)
        return x

module = TestModule()
module.eval()
torch.onnx.export(
    module,
    (torch.ones([1, 3, 64, 64], dtype=torch.float32), ),
    "test.onnx",
    opset_version=15,
    input_names=["input"],
    output_names=["output"],
    export_modules_as_functions={
        nn.AvgPool2d
    }
)
~~~
![test](https://github.com/pytorch/pytorch/assets/30765323/c6f8b687-ce27-4924-a8e9-99cd3f34c0f0)
The output name of the graph is set to the meaningless number `8` instead of `output`.

In `torch.onnx.dynamo_export`, there is not yet an interface for setting input names and output names, and it is hoped that some of the interface settings of `torch.onnx.export` can be migrated to the `torch.onnx.dynamo_export` to improve the readability of the final exported model

### Alternatives

_No response_

### Additional context

_No response_

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305
dist.scatter is incompatible with transpose/permute operation
Labels: oncall: distributed, triaged

### 🐛 Describe the bug

I found that when using dist.scatter() (torch 2.0.1, backend=nccl) and the tensors in scatter tensor list have been modified by operations such as transpose, permute, etc., (so that it is not contiguous), the results are abnormal. Here is the demo.

```import os
 
import torch
import torch.distributed as dist
import torch.multiprocessing as mp
 
 
def main(rank):
    dist.init_process_group(backend="nccl", world_size=2, rank=rank)
    device = f"cuda:{rank}"
 
    a = torch.arange(0, 4).reshape((2, 2)).to(torch.float).to(device)
    b = torch.arange(4, 8).reshape((2, 2)).to(torch.float).to(device)
    a = a.transpose(0, 1) # [[0,2],[1,3]]
    b = b.transpose(0, 1) # [[4,6],[5,7]]
    src = [a, b] if rank == 0 else None
    dst = torch.zeros((2, 2)).to(device)
    dist.scatter(dst, src, 0)
    """
    Expected result of dst:   rank0 [[0,2],[1,3]] rank1 [[4,6],[5,7]]
    Real result of dst:   rank0 [[0,2],[1,3]] rank1 [[4,5],[6,7]]
    """
    print(rank, dst)
 
 
if __name__ == "__main__":
    os.environ["MASTER_ADDR"] = "127.0.0.1"
    os.environ["MASTER_PORT"] = "6666"
    mp.spawn(main, nprocs=2)
```

I guess the scatter() directly copy the corresponding memory (without consider the stride variation caused by tranpose/permute) to the dest. However, this is not what the users expect. A simple way to solve this issue is to add a check script at the beginning of the scatter operation like:

```
if not a.is_contiguous():
    a = a.contiguous()
```

Also, I'm not sure if the bug will appear for other DDP operations like gather, reduce, etc.



### Versions

PyTorch version: 2.0.1+cu117


cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu
commit for testing internal tools
Labels: Stale, topic: not user facing

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #107317

This pr will not be merged onto main, it is just here to test / develop some CLI tools

Differential Revision: [D48403122](https://our.internmc.facebook.com/intern/diff/D48403122)
NumPy 2.0 Support
Labels: triaged, module: numpy, module: python frontend

NumPy 2.0  is expected to be out in December 2023. See details at https://github.com/numpy/numpy/issues/24300

The major changes are going to be some Python API and behavior improvements. These don't impact PyTorch code itself and our compat layer (https://github.com/pytorch/pytorch/tree/main/torch/_numpy) is targeting 2.0+ already.

There are also C-API changes and in particular while 2.0 ABI will be backward compatible, the current ABI will not be forward compatible with 2.0. This will require some care on our end to make sure things work properly.

In terms of PyTorch future release, it means that, as of today, the expected support matrix looks like:
- PyTorch <=2.1 and nightly until EOY 2023 can live alongside any version of NumPy but the `.numpy()` and `.from_numpy()` functions will ONLY work if the version of NumPy installed is <2.0. Users can build from source against their version of NumPy if they need a more recent version of NumPy.
- PyTorch >=2.2 and nightly after EOY 2023 (once we have a NumPy 2.0 ABI-compatible build) will be fully compatible with all versions of NumPy.


The AIs to ensure this works:
- [ ] Add a helpful error message in PyTorch to give users a helpful error message if they try to use .numpy() or .from_numpy() with numpy 2.0+ today. This will replace the ABI error message from https://github.com/numpy/numpy/blob/7b56da8c46427d487b9337780417a9aad64eb295/numpy/core/code_generators/generate_numpy_api.py#L81-L82 that they will get today by a message asking them use NumPy <2.0 or build from source.
- [ ] Once we have a NumPy 2.0 compatible available, upgrade all our binary build to this version of NumPy and remove the error message above. This will ensure that our produced binaries work with any version of NumPy (including <2.0)
- [ ] Make `torch.compile` compatible with NumPy 2.0. In order to do that, we may need to update the `test/torch_np` tests that we pulled from NumPy


cc @mruberry @rgommers
Enable flake8-bugbear B028 lint
Labels: triaged, open source, module: amp (automated mixed precision), release notes: quantization, release notes: releng, ciflow/mps, module: inductor, module: dynamo, ciflow/inductor, module: export

Part of #106571

cc @mcarilli @ptrblck @leslie-fang-intel @jgong5 @voznesenskym @penguinwu @EikanWang @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @anijain2305
Sparse compressed tensor values autograd support is not implemented
Labels: module: sparse, module: autograd, triaged, bug

## Issue description

As in the title.

## Code example

```python
>>> x = torch.tensor([[10, 1], [2, 3]], dtype=torch.float64).to_sparse_csr().requires_grad_(True)
>>> torch.autograd.gradcheck(torch.Tensor.values, x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "torch/autograd/gradcheck.py", line 2051, in gradcheck
    return _gradcheck_helper(**args)
  File "torch/autograd/gradcheck.py", line 2080, in _gradcheck_helper
    _gradcheck_real_imag(
  File "torch/autograd/gradcheck.py", line 1482, in _gradcheck_real_imag
    gradcheck_fn(
  File "torch/autograd/gradcheck.py", line 1617, in _slow_gradcheck
    analytical = _check_analytical_jacobian_attributes(
  File "torch/autograd/gradcheck.py", line 768, in _check_analytical_jacobian_attributes
    vjps1 = _compute_analytical_jacobian_rows(vjp_fn, output.clone())
  File "torch/autograd/gradcheck.py", line 884, in _compute_analytical_jacobian_rows
    grad_inputs = vjp_fn(grad_out_base)
  File "torch/autograd/gradcheck.py", line 759, in vjp_fn
    return torch.autograd.grad(
  File "torch/autograd/__init__.py", line 394, in grad
    result = Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: indices expected sparse coordinate tensor layout but got SparseCsr
```

The expected behavior is no exception with return value True.

The exception message above is explained by
https://github.com/pytorch/pytorch/blob/ddba7a5a55aafa012b8b4ad5aaf64375296e805b/tools/autograd/derivatives.yaml#L1876-L1881
that defines why the sparse CSR related code path leads to calling sparse-COO-tensor-specific code (`indices`).

## System Info

- PyTorch version: main


cc @alexsamardzic @nikitaved @cpuhrsch @amjames @bhosmer @ezyang @albanD @zou3519 @gqchen @soulitzer @Lezcano @Varal7
DISABLED test_find_or_create_pg (__main__.TestPgTag)
Labels: oncall: distributed, module: flaky-tests, skipped

Platforms: linux, rocm

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_find_or_create_pg&suite=TestPgTag) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/15924471388).

Over the past 3 hours, it has been determined flaky in 2 workflow(s) with 2 failures and 2 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_find_or_create_pg`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `distributed/test_functional_api.py`

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu
Translation layer (similar to torch_np) that can reliably lift Python operations into Tensor operations
Labels: feature, triaged, oncall: pt2

### 🐛 Describe the bug

In https://github.com/pytorch/pytorch/pull/106211 we added a translation layer that can take Numpy programs and translate them into corresponding PyTorch programs, so that we could transparently rewrite Numpy programs into accelerated PyTorch code.

We need a similar translation layer for programs that operate on native Python ints/floats/bools/etc. This is necessary to properly exercise on the agreed upon plan in https://docs.google.com/document/d/1INSCdYu1PxXcr43HrD82OudeEuS-qxQe1yZmLg2wy6A/edit#heading=h.aytveihghlgn where we want to translate SymFloat operations into tensor operations. Dynamo sort of implements this today, but on a very ad hoc basis.

Naively, you might think this is pretty straightforward (a multiply on ints is just a multiple on 0-d int tensors), but doing this in complete generality has more edge cases than you might think:

* Sometimes the obvious translation doesn't actually match semantics
* Anywhere an operator accepts a Scalar, you need to translate it into an appropriate variant that accepts a Tensor. Sometimes there is no Tensor-ified variant and you have to get creative (with a decomp or something)
* Python scalars have different type promotion rules than tensors
* Python scalars always work with tensors that live on other devices. However, if you are compiling a call `cuda_tensor.item()`, you would prefer NOT to immediately move the corresponding tensor to CPU in case you can avoid the DtoH sync entirely (maybe Inductor can take care of this optimization directly)
* The precision of operations on Tensors may differ from Python precision. This may be fundamentally unsolvable, but it can problems when hard thresholds are applied against the results of floating point computation; e.g., see for example https://github.com/pytorch/vision/pull/7592
* Some APIs only take scalars and aren't even operators e.g. https://gist.github.com/ezyang/7ebe5e607451cdd2cc067c9eeb1de3b4

### Versions

main

cc @msaroufim @wconstab @bdhirsh @anijain2305
Fix batch_norm_cpu to check the sizes of tensors
Labels: module: cpu, Stale, ciflow/trunk, topic: not user facing

When using `torch.ops.aten._native_batch_norm_legit.default` directly, if you accidentally pass the wrong shape of input tensors for weight, bias, or the stats, you get an ASAN exception on reading out of bounds from an accessor.
Instead, it should have a digestible error message that avoids memory corruption bugs.

Add a `TORCH_CHECK` that all of the inputs have the right dimensionality. This will raise a RuntimeError to python
if the check is violated. This behavior matches that of `F.batch_norm` which also uses `TORCH_CHECK` on the dimensionality
of its input tensors.

Added these error inputs as a unit test

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10
`torch.float8_e4m3fn` does not support `torch.cat`
Labels: triaged, module: float8

As of right now, this is not blocking any work. However, I would imagine that concatenating `float8_e4m3fn` tensors should be well-defined.

<details>
<summary> Repro </summary>

```
import torch

def test_cat_fp8_cpu():
    t1 = torch.empty((3,), dtype=torch.float8_e4m3fn)
    t2 = torch.empty((2,), dtype=torch.float8_e4m3fn)
    t = torch.cat([t1, t2], dim=0)
    print(t)

def test_cat_fp8_cuda():
    t1 = torch.ones((3,), dtype=torch.float8_e4m3fn, device="cuda")
    t2 = torch.ones((2,), dtype=torch.float8_e4m3fn, device="cuda")
    t = torch.cat([t1, t2], dim=0)
    print(t)

try:
    test_cat_fp8_cpu()
except Exception as e:
    print(e)
try:
    test_cat_fp8_cuda()
except Exception as e:
    print(e)
```

```
Unsupported TypeMeta in ATen: nullptr (uninitialized) (please report this error)
Unsupported TypeMeta in ATen: nullptr (uninitialized) (please report this error)
```

Addendum: `torch.float8_e4m3fn` does not support `torch.ones` on CPU:
```
"fill_cpu" not implemented for 'Float8_e4m3fn'
```


</details>



cc @yanbing-j
Conda configuration shouldn't pollute $PATH variable
Labels: oncall: releng, module: ci, triaged

### Description

In the `install_conda.sh`, the environmental path is exported to include a conda path. This has caused rocMLIR as a dependency (MIOpen, and therefore) to Pytorch to fail the build.

https://github.com/pytorch/pytorch/blob/e1ee10e6f59d43cd7c0fc830954220558beb76f4/.ci/docker/common/install_conda.sh#L40

### Analysis

Our component (rocMLIR) is dependent on `lld`, and we clone a local copy to lld. In the build time, we would build and link with lld shipped in our component. However, under `/opt/conda`, Pytorch also ship with a `lld`. And since Pytorch has added the conda path to the environmental PATH variable, this implicitly impacted all cmake based build as `find_path` make its decisions based on the system environment variable (according to [here](https://cmake.org/cmake/help/latest/command/find_path.html)). This caused the build of every object file in our component to compile with a `-isystem /opt/conda/include`. Eventually it becomes a compilation failure because the conda `lld` is of different version than our locally shipped lld.

There is no good way for us as a dependency to fix this issue as this `/opt/conda` was incorrectly set to PATH and become a global setting.

### Recommended fix

I recommend one of the following ways to address this:
 - Encode the conda path to `CONDA_PREFIX`. This way, the successive build can reference this variable for conda related dependencies, and we also have a way to ignore this path in our build.
 - Use the officially shipped bash script to turn on and off the conda related environment variables. Don't use `activate/deactivate` pair when a certain dependency doesn't require conda dependencies.
    - Before build a component that depend on conda: `bash /opt/conda/bin/activate`
    - After build completes, deactivate the environment: `bash /opt/conda/bin/deactivate`

References:
[1] "CMake in unrelated projects pick up anaconda path" https://github.com/conda/conda/issues/4552
[2] "Cmake ignore anaconda libs" https://www.scivision.dev/cmake-ignore-anaconda-libs/



cc @seemethere @malfet @pytorch/pytorch-dev-infra
Fallback random in test
Labels: Stale, topic: not user facing, module: inductor, keep-going

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #107242



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @ngimel
torch.compile not tracing ops on tensor subclass
Labels: triaged, module: __torch_dispatch__, oncall: pt2, module: dynamo

### 🐛 Describe the bug

torch.compile doesn't seem to be tracing ops on tensor subclasses yet.

Minimal repro: https://gist.github.com/vkuzo/15b3390241957b31773162f680c0114b 
Expected output (see this without tensor subclass): https://gist.github.com/vkuzo/e04e00d430ce2985cd30470f90029a81
Actual output: https://gist.github.com/vkuzo/c56f8ffe217e28c90dac7b5bde75429d

### Versions

torch==2.1.0a0+gita14d99b

cc @Chillee @ezyang @zou3519 @albanD @samdow @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @aakhundov
[dynamo] sdp_kernel ctx manager support
Labels: fb-exported, Stale, module: dynamo, ciflow/inductor

Summary: Adding dynamo support in sdp_kernel context manager.

Test Plan: sandcastle

Differential Revision: D48354002



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng @anijain2305 @Xia-Weiwen @ipiszy
[CPP API] Add Adadelta, Adamax, ASGD, NAdam, RAdam and Rprop
Labels: module: cpp, module: optimizer, triaged, needs research

### 🚀 The feature, motivation and pitch

This feature request revolves around augmenting the PyTorch CPP API by integrating additional optimizers that are currently absent. Specifically, I propose the inclusion of the following optimizers from [Python's torch.optim](https://pytorch.org/docs/stable/optim.html#algorithms): Adadelta, Adamax, ASGD, NAdam, RAdam, and Rprop, as they are currently not part of the CPP API's offerings.

### Alternatives

_No response_

### Additional context

I would like to work on this issue.
We could split this issue into different issues if needed.

cc @jbschlosser @vincentqb @albanD @janeyx99 @crcrpar @soumith @iramazanli
DISABLED test_RNN_input_size_zero (__main__.TestNN)
Labels: module: rocm, triaged, skipped

Platforms: rocm

This test was disabled because it is failing on main branch ([recent examples](http://torch-ci.com/failure/test_nn.py%3A%3ATestNN%3A%3Atest_RNN_input_size_zero)).

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang
Documenting `__getitems__` for slicing support in `torch.utils.data`
Labels: triaged, module: data

### 📚 The doc issue

It seems `torch.utils.data` supports a magic method `__getitems__` (implemented [here](https://github.com/pytorch/pytorch/blob/935f2dd084d0bd7e471fe94a2a6c0e0bc3a9d3a2/torch/utils/data/dataset.py#L362) and [here](https://github.com/pytorch/pytorch/blob/935f2dd084d0bd7e471fe94a2a6c0e0bc3a9d3a2/torch/utils/data/_utils/fetch.py#L49)) that can be called on `Dataset` objects for batch fetching.

- `__getitem__`: designed for `int` items
- `__getitems__`: used with "batched indexes" of type `list[int]`

I think more generally, `__getitems__` can be expanded to `Sequence[int]` to accommodate `slice` support.

### Suggest a potential alternative/fix

What do you think of:
- Documenting `__getitems__`'s existence and purpose somewhere in [`torch.utils.data`](https://pytorch.org/docs/stable/data.html)?
- Expanding its definition to be a way of getting map-style datasets to support slicing?

cc @VitalyFedyunin @ejguan @dzhulgakov
Documenting `IterableDataset`'s needing `StopIteration` for finite data
Labels: triaged, module: data

### 📚 The doc issue

When using `IterableDataset` with finite data, one needs to raise `StopIteration` after exhausting the underlying data, or else iterations will eventually go out of bounds. This is relevant when directly iterating over the `IterableDataset`, or when passing the dataset into a `DataLoader`.

For example, in the below snippet, there will be an `IndexError` if the `StopIteration` inside `__next__` is removed:

```python
from collections.abc import Sequence
from typing import Self

from torch.utils.data import IterableDataset


class SomeIterableDataset(IterableDataset):
    def __init__(self, data: Sequence[int]):
        self._data = data
        self._index = 0

    def __iter__(self) -> Self:
        return self

    def __next__(self) -> int:
        index = self._index
        if index >= len(self._data):
            self._index = 0  # Reset for next cycle through the dataset
            raise StopIteration  # Stops for loop after
        value = self._data[self._index]
        self._index += 1
        return value


for i in range(2):
    for datapoint in SomeIterableDataset(data=[1, 2, 3, 4]):
        print(f"Loop {i}: value is {datapoint}.")
```

### Suggest a potential alternative/fix

I think it would be good to mention `StopIteration` somewhere to the [`IterableDataset` docs](https://pytorch.org/docs/stable/data.html#torch.utils.data.IterableDataset).  This can be done by augmenting one of the examples, or alternately adding text.

If interested, I am happy to pose and contribute an addition.

cc @VitalyFedyunin @ejguan @dzhulgakov