gopikrsmscs/torch-issues huggingface dataset:
test_Bilinear_empty_cuda: IndexError: pop from empty list
Labels: triaged

Repro:
```
 PYTORCH_TEST_WITH_INDUCTOR=1 python test/test_nn.py -k   test_Bilinear_empty_cuda
```
Error:
```
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/fsx/users/binbao/pytorch/torch/_inductor/ir.py", line 1229, in dynamic_reshape_indexer
    reindex = cls._dynamic_reshape_indexer(old_size, new_size)
  File "/fsx/users/binbao/pytorch/torch/_inductor/ir.py", line 1263, in _dynamic_reshape_indexer
    var2, size_new2 = stack_new.pop()
IndexError: pop from empty list
```
test_memory_format_ao_nn_quantized_MaxPool2d_cuda_float32: assert not memory_format, "TODO"
Labels: triaged

Repro: 
Comment out https://github.com/pytorch/pytorch/blob/cda0d5a57b9126c6d244fdd5b02198f05c742615/test/test_modules.py#L585 and then run
```
PYTORCH_TEST_WITH_INDUCTOR=1 python test/test_modules.py -k test_memory_format_ao_nn_quantized_MaxPool2d_cuda_float32
```
Error:
```
Traceback (most recent call last):
  File "/fsx/users/binbao/pytorch/torch/_inductor/graph.py", line 241, in call_function
    out = lowerings[target](*args, **kwargs)
  File "/fsx/users/binbao/pytorch/torch/_inductor/lowering.py", line 204, in wrapped
    return decomp_fn(*args, **kwargs)
  File "/fsx/users/binbao/pytorch/torch/_inductor/lowering.py", line 376, in _to_copy
    assert not memory_format, "TODO"
AssertionError: TODO
```
"No CUDA GPUs are available" coming from GHA g5 runners
Labels: oncall: releng, module: ci, triaged

### 🐛 Describe the bug

There is a new trend of flaky issues when the GPU seems to disappear from GHA g5 runners. There are 2 common errors:

* `RuntimeError: No CUDA GPUs are available`: https://hud.pytorch.org/failure/RuntimeError%3A%20No%20CUDA%20GPUs%20are%20available  
* `nvidia-smi` failure `nvidia-smi --query-gpu=driver_version --format=csv,noheader --> INSTALLED_DRIVER_VERSION='No devices were found'`.  They might be related
  * https://hud.pytorch.org/pytorch/pytorch/commit/bf113e38fad30fb1eec1f94563f419518ae3178c
  * https://hud.pytorch.org/pytorch/pytorch/commit/107f92a6830f61b88a7eb55934610f491623dc9b 

**Examples**:
https://hud.pytorch.org/pytorch/pytorch/commit/78b406932f0e4afd82b672f959b8cb9ce1e79f9d

### Versions

GHA g5 runners (NVIDIA A10G). Inductors and some trunk tests use them

cc @seemethere @malfet @pytorch/pytorch-dev-infra @ZainRizvi @jeanschmidt @DanilBaibak
RuntimeError: unable to mmap 29764 bytes from file </torch_10182_3020184674_63991>: Cannot allocate memory (12)
Labels: triaged, module: tensor creation

### 🐛 Describe the bug

    for _ in range(flags.num_buffers):
        for key in buffers:
            buffers[key].append(torch.zeros(**specs[key]).share_memory_())
    return buffers

Traceback (most recent call last):
  File "/home/cloudam/rl-baseline (copy)/monobeast.py", line 634, in <module>
  File "/home/cloudam/rl-baseline (copy)/monobeast.py", line 512, in train
  File "/home/cloudam/rl-baseline (copy)/monobeast.py", line 140, in create_buffers
  File "/home/cloudam/miniconda3/lib/python3.9/site-packages/torch/_tensor.py", line 619, in share_memory_
  File "/home/cloudam/miniconda3/lib/python3.9/site-packages/torch/storage.py", line 627, in share_memory_
  File "/home/cloudam/miniconda3/lib/python3.9/site-packages/torch/storage.py", line 208, in share_memory_
RuntimeError: unable to mmap 29764 bytes from file </torch_10182_3020184674_63991>: Cannot allocate memory (12)

### Versions

pytorch 10.0 cu116


cc @gchanan @mruberry
[Inductor] Support deterministic parallel reduction in CPP backend
Labels: triaged

CPP backend uses OMP to do parallel reduction which cannot guarantee a deterministic reduction order and might produce different run-to-run result while determinism is important to ease the debugging. We should replace omp parallel reduction with deterministic parallel reduction.

Relevant issue/PR: https://github.com/pytorch/pytorch/issues/93542, https://github.com/pytorch/pytorch/pull/87259
AssertionError: Unknown expression s2
Labels: triaged, bug

### 🐛 Describe the bug

```
TORCHDYNAMO_DYNAMIC_SHAPES=1 AOT_DYNAMIC_SHAPES=0 time python benchmarks/dynamo/huggingface.py  --accuracy --backend eager --training --only AllenaiLongformerBase
```

on master

```
  File "/private/home/ezyang/.local/lib/python3.8/site-packages/sympy/printing/printer.py", line 331, in _print
    return printmethod(expr, **kwargs)
  File "/private/home/ezyang/.local/lib/python3.8/site-packages/sympy/printing/str.py", line 779, in _print_Relational
    return '%s(%s, %s)' % (charmap[expr.rel_op], self._print(expr.lhs),
  File "/private/home/ezyang/.local/lib/python3.8/site-packages/sympy/printing/printer.py", line 331, in _print    return printmethod(expr, **kwargs)
  File "/raid/ezyang/pytorch-scratch2/torch/_dynamo/guards.py", line 526, in _print_Symbol
    assert expr in self.expr_to_tensor_ref, f"Unknown expression {expr}"
AssertionError: Unknown expression s2
```

### Error logs

_No response_

### Minified repro

_No response_
Libtorch windows binaries publishing
Labels: module: ci, triaged, topic: binaries

### 🐛 Describe the bug

Looks like following libtorch configuations are missing from windows while present in linux.
Research if they are required. And if they are add make sure we build them:

Windows Libtorch missing:
```
libtorch_variants = [
            "shared-without-deps",
            "static-with-deps",
            "static-without-deps",
        ]
```

### Versions

1.14

cc @seemethere @malfet @pytorch/pytorch-dev-infra
`torchtyping` annotations make saving to Torchscript fail
Labels: oncall: jit

### 🐛 Describe the bug

It seems that there is already some interest in offering a type-annotation system for PyTorch.
[This issue](https://github.com/pytorch/pytorch/issues/73359) mentions [torchtyping](https://github.com/patrick-kidger/torchtyping). However, on trying to save a `nn.Module` that includes such annotations, to TorchScript, I get an issue that looks like this:
```
RuntimeError:
       Unknown type constructor TensorType:
         File "my/code/vae.py", line 34
           def forward(
               self, x: TensorType["batch_size", "input_dim"]
                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
           ) -> TensorType["batch_size", "input_dim"]:
```
It's very unclear to me why a type annotation would impact the TorchScript saving process, but I don't know anything about TorchScript.

However, the `torchtyping` maintainer seems to have looked into it and they mentioned a year ago that the issue lies with the TorchScript implementation. See the full discussion here: <https://github.com/patrick-kidger/torchtyping/issues/13>. Since they recommended contacting the TorchScript maintainers, and I could not find an issue about it in this project, I'd like to bring this to the attention to whom it may concern, if they're not already aware of this.

Some kind people have shared [workarounds](https://github.com/patrick-kidger/torchtyping/issues/13#issuecomment-1243725436) in the comments of this issue, and they seem to work.

Thanks for your attention!

### Versions

PyTorch version: 1.12.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.5.1 (x86_64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.102)
CMake version: version 3.24.2
Libc version: N/A

Python version: 3.10.7 (main, Sep 15 2022, 01:51:29) [Clang 14.0.0 (clang-1400.0.29.102)] (64-bit runtime)
Python platform: macOS-12.5.1-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.3
[pip3] pytorch-lightning==1.7.7
[pip3] torch==1.12.1
[pip3] torchmetrics==0.10.0
[pip3] torchtyping==0.1.4
[conda] Could not collect

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel
Improvements to fuse optimization
Labels: oncall: quantization, triaged, module: fx

### 🚀 The feature, motivation and pitch

Some optimization packages call pytorch's fuse function to try optimization without knowing much about the model, notably ipex and packages which call ipex.
This operation relies on symbolic tracing, which fails frequently due to the limitations on conditional statements.

A possible improvement here is before tracing, filter the patterns by those that could work given the submodules, i.e. if there is no conv or batch norm, there is no reason to include the pattern. If there are none left after this step, just return the model.


### Alternatives

Alternatively or additionally, symbolic tracing (or its use in fuse) could be improved.
The points of failure in symbolic tracing are outlined in the ipex issue and are mainly:

* input checking on optional arguments, which could be resolved by allowing concrete_args in fuse
* dimension checks, which could be improved by allowing a proxy to have a dimension or even a shape (although this goes quite far, and I can see the potential for some strange bugs here)

### Additional context

[this issue](https://github.com/intel/intel-extension-for-pytorch/issues/250) contains some examples of typical failures.


cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @vincentqb @jbschlosser @ezyang @SherlockNoMad @soumith @EikanWang @wenzhe-nrv
`torch.min`/`torch.max` returns bogus values for default int tensors on MPS
Labels: triaged, module: correctness (silent), module: mps

### 🐛 Describe the bug

```
%  python -c "import torch;x=torch.tensor([1, 2, 3],device='mps');print(x, x.max(), x.min())"
tensor([1, 2, 3], device='mps:0') tensor(1, device='mps:0') tensor(0, device='mps:0')
```

### Versions

1.12.1, 1.13.0, nightly

cc @kulinseth @albanD @DenisVieriu97 @razarmehr @abhudev
Checkpointing Support for Modularized Optimizers
Labels: module: optimizer, triaged, needs design

### 🚀 The feature, motivation and pitch

Hi, I'm currently working on an implementation of distributed Shampoo in PyTorch (https://github.com/facebookresearch/optimizers/tree/main/distributed_shampoo). Because of the complexity of the optimizer, it is necessary to modularize the optimizer by creating additional classes that may also contain Tensor objects (i.e., Preconditioner objects). However, `torch.optim.Optimizer`'s checkpointing functionality (`state_dict` / `load_state_dict`) does not natively support this kind of modularization, which requires us to override the `load_state_dict` function to support other custom classes. This is in order to ensure that the tensors contained within these custom state objects are appropriately moved by the `cast` function after loading from checkpoint.

Rather than requiring the user to override and support this additional checkpointing functionality, would it be possible to create an `OptimizerModule` (similar to `nn.Module`) that contains a `load_state_dict` that recursively calls this function within other `OptimizerModule` objects and is compatible with `torch.optim.Optimizer`? Developers can then build their new classes on top of this `OptimizerModule` to ensure that the cast automatically supports new classes.

Thanks in advance!

CC'ing collaborators and TorchRec folks since we are also interested in making distributed Shampoo also compatible with TorchRec's `KeyedOptimizer`.

cc: @tsunghsienlee, @dmudiger, @mikerabbat, @YLGH, @albanD

### Alternatives

In order to currently support checkpointing for Shampoo, we created `.to` functions for all custom classes in our optimizer that enable us to move all Tensor objects within all of our custom classes to a different device. We then modified the `cast` function within `torch.optim.optimizer`'s `load_state_dict` function by including an additional case for our `Preconditioner` class. While this works, it is not ideal since it requires us to maintain the `.to` functions for each class or (non-Iterable or dict) data structure we create, which can be bug-prone. 

### Additional context

Related to this issue on Distributed Shampoo: https://github.com/facebookresearch/optimizers/issues/3.

cc @vincentqb @jbschlosser @albanD @janeyx99 @crcrpar
FakeTensorMode doesn't support two Scalar inputs, if we use prims' impl as the meta function 
Labels: triaged, module: primTorch, module: fakeTensor

### 🐛 Describe the bug

uncomment following line in _meta_registration.py

```
            # "aten::add.Tensor",  # ValueError: Receive two Number inputs to an elementwise binary operation! inductor/test_torchinductor.py -k test_both_scalars  # noqa: B950
            # "aten::sub.Tensor",  # ValueError: Receive two Number inputs to an elementwise binary operation! inductor/test_torchinductor.py -k test_both_scalars  # noqa: B950
            # "aten::mul.Tensor",  # ValueError: Receive two Number inputs to an elementwise binary operation! inductor/test_torchinductor.py -k test_both_scalars  # noqa: B950
            # "aten::div.Tensor",  # ValueError: Receive two Number inputs to an elementwise binary operation! test_fake_tensor.py -k test_scalar_inputs  # noqa: B950
            # "aten::div.Tensor_mode",  # ValueError: Receive two Number inputs to an elementwise binary operation! inductor/test_torchinductor.py -k test_div8_cpu  # noqa: B950

```

And this will use prims.add as the meta function. 

Further commenting out following line in _refs/__init__.py for `def add` fucntion
```
    # if isinstance(a, Number) and isinstance(b, Number):
    #     raise ValueError(
    #         "Receive two Number inputs to an elementwise binary operation!"
    #     )

```

Repo:

```
with FakeTensorMode():
    out = torch.add(3, 4)
    print(out)
```

Got 
```
Traceback (most recent call last):
  File "/scratch/bahuang/work/repos/pytorch/temp/t.py", line 28, in <module>
    out = torch.add(3, 4)
  File "/scratch/bahuang/work/repos/pytorch/torch/_subclasses/fake_tensor.py", line 831, in __torch_dispatch__
    return self.wrap_meta_outputs_with_default_device_logic(
  File "/scratch/bahuang/work/repos/pytorch/torch/_subclasses/fake_tensor.py", line 866, in wrap_meta_outputs_with_default_device_logic
    return tree_map(partial(wrap), r)
  File "/scratch/bahuang/work/repos/pytorch/torch/utils/_pytree.py", line 195, in tree_map
    return tree_unflatten([fn(i) for i in flat_args], spec)
  File "/scratch/bahuang/work/repos/pytorch/torch/utils/_pytree.py", line 195, in <listcomp>
    return tree_unflatten([fn(i) for i in flat_args], spec)
  File "/scratch/bahuang/work/repos/pytorch/torch/_subclasses/fake_tensor.py", line 879, in wrap
    return converter(self, e, device or common_device)
  File "/scratch/bahuang/work/repos/pytorch/torch/_subclasses/fake_tensor.py", line 268, in __call__
    assert t.device.type == "meta", f"device must be meta, got {t.device.type} instead"
AssertionError: device must be meta, got cpu instead
```

PyTorch eager supports this, and there is a test case in `inductor/test_torchinductor.py -k test_both_scalars` for this.  



### Versions

master


cc @ezyang @mruberry @ngimel @Lezcano @fdrocha @eellison, @ngimel, @ezyang
C++ Adagrad optimizer doesn't initialize parameter state
Labels: module: cpp, module: optimizer, triaged, actionable

### 🐛 Describe the bug

optim/adagrad.cpp lacks code to initialize parameter state.

Running this code:

```
 auto x=torch::tensor({0.5, 2.0, 4.0}, torch::requires_grad());
 auto y=torch::tensor({1.0, 2.0, 3.0});
 auto o=torch::optim::Adagrad(std::vector<torch::Tensor>{}, torch::optim::AdagradOptions(0.1));
 auto& p=o.param_groups();
 p[0].params().push_back(x);
 auto l=torch::nn::functional::mse_loss(x,y);
 l.backward();
 o.step();
```
will throw:
_'state_[c10::guts::to_string(p.unsafeGetTensorImpl())] != nullptr INTERNAL ASSERT FAILED at "../torch/csrc/api/src/optim/adagrad.cpp":80, please report a bug to PyTorch. state found NULL for the Tensor_ 

the other optimizers have code that searches for the state and initializes if not found, e.g. from sgd.cpp:

```
        auto param_state = state_.find(c10::guts::to_string(p.unsafeGetTensorImpl()));
        if(param_state == state_.end()) {
          buf = torch::clone(d_p).detach();
          auto state = std::make_unique<SGDParamState>();
          state->momentum_buffer(buf);
          state_[c10::guts::to_string(p.unsafeGetTensorImpl())] = std::move(state);
        } else {
          buf = static_cast<SGDParamState&>(*param_state->second).momentum_buffer();
          buf.mul_(momentum).add_(d_p, 1 - dampening);
        }
```
adagrad.cpp doesn't contain this kind of code.

### Versions

This error in all versions since PR  #29335

cc @jbschlosser @vincentqb @albanD
pytorch/pytorch cpu official Docker images
Labels: triaged, module: docker

### 🚀 The feature, motivation and pitch

First of all I want to thank you for maintaining PyTorch-based images. I think I can understand the complexity that caring about millions of times downloaded images has to have. I've been using these images since I switched to PyTorch and they never failed me.

Though, lately I've been missing cpu-only images.

The motivation behind this request is basically having a lighter version of them, focused especially in the ability to launch CI test (and potentially other kind of) jobs in which I don't pretend to test my models but just some tensor operations I do with PyTorch.

Not only that, this would also support the deployment of lightweight models that can run on cpu, when you can afford the time trade-off and want to save some cash.

Thank you,

Claudio




### Alternatives

_No response_

### Additional context

_No response_
Enable PostLocalSGDOptimizer on CUDA tensors
Labels: oncall: distributed, module: cuda, triaged

### 🚀 The feature, motivation and pitch

Hello!

I am trying to leverage the power of the `PostLocalSGDOptimizer` for averaging gradients during distributed training. However, as mentioned in the [docs](https://pytorch.org/docs/stable/distributed.optim.html#), I see that this optimization is not supported on CUDA tensors.

I think that enabling `PostLocalSGDOptimizer` for CUDA tensors would benefit tremendously the community, and thus, I was wondering: is this currently being worked on?

### Alternatives

_No response_

### Additional context

_No response_

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @vincentqb @jbschlosser @ngimel
Performance issue on Windows with a "benchmark" comparing to Linux and WLS
Labels: module: windows, triaged

### Description
- Started upon discussion on https://pytorch.slack.com/archives/C042FDXSH51/p1664990529414809
- No PyTorch issue created yet. We will need solid reproduction case. It was requested but not provided yet.
- The issue was reported on https://pytorch.org/vision/main/models/generated/torchvision.models.efficientnet_b3.html with  Linknet
- Some benchmarks that can be used:
  - https://github.com/pytorch/benchmark
  - https://github.com/LukasHedegaard/pytorch-benchmark
  - https://github.com/ryujaehun/pytorch-gpu-benchmark

### Acceptance Criteria
- The reported performance issue is identified and analyzed.
- If upstream issue will be created, it's resolved.

### Sub-tasks
- https://github.com/microsoft/ARM-on-Windows-Ecosystem/issues/191
- https://github.com/microsoft/ARM-on-Windows-Ecosystem/issues/192
- https://github.com/microsoft/ARM-on-Windows-Ecosystem/issues/193
- https://github.com/microsoft/ARM-on-Windows-Ecosystem/issues/221

### Documentation
- It seems that there are currently no publicly available PyTorch binaries with debug symbols https://github.com/pytorch/pytorch/issues/11982
INTERNAL ASSERT FAILED !(has_different_input_dtypes && !config.promote_inputs_to_common_dtype_ && (has_undefined_outputs || config.enforce_safe_casting_to_output_ || config.cast_common_dtype_to_outputs_))
Labels: needs reproduction, triaged, module: TensorIterator

### 🐛 Describe the bug

RuntimeError: !(has_different_input_dtypes && !config.promote_inputs_to_common_dtype_ && (has_undefined_outputs || config.enforce_safe_casting_to_output_ || config.cast_common_dtype_to_outputs_)) INTERNAL ASSERT FAILED

```
import torch

input_tensor = torch.rand(10, 10)
grad_tensor = torch.rand(10, 10)
index_tensor = torch.randint(low=0, high=10, size=(10,))

input_tensor.addcdiv(1, grad_tensor, index_tensor)

print(input_tensor)
```
```
RuntimeError: !(has_different_input_dtypes && !config.promote_inputs_to_common_dtype_ && (has_undefined_outputs || config.enforce_safe_casting_to_output_ || config.cast_common_dtype_to_outputs_)) INTERNAL ASSERT FAILED at "/opt/conda/conda-bld/pytorch_1659484806139/work/aten/src/ATen/TensorIterator.cpp":405, please report a bug to PyTorch. 
```

### Versions

PyTorch 1.12.1
`libtorch_cpu.so` is exposing some LLVM symbols
Labels: module: build, module: cpp-extensions, triaged

### 🐛 Describe the bug

#63272 had hidden LLVM symbols by making LLVM a `PRIVATE` dependency of `torch_cpu` in `caffe2/CMakeLists.txt`, but `libtorch_cpu.so` is still exposing some LLVM symbols.

```shell
nm -D ./build/lib/libtorch_cpu.so | grep llvm
000000000ca82122 B FLAGS_torch_jit_llvm_use_fast_intrinsics
0000000009fe4650 T llvm_orc_deregisterEHFrameSectionWrapper
0000000009fe4720 T llvm_orc_registerEHFrameSectionWrapper
000000000a048800 T llvm_regcomp
000000000a049120 T llvm_regerror
000000000a04bba0 T llvm_regexec
000000000a04cfb0 T llvm_regfree
000000000a04d030 T llvm_strlcpy
```

This issue doesn't affect PyTorch directly but can affect a [PyTorch C++ extension](https://pytorch.org/tutorials/advanced/cpp_extension.html) if it links to another library that also links to LLVM.
Even in that case, workarounds can be used, so it's probably not a high priority issue at this point. Thanks!

### Versions

Master branch


cc @malfet @seemethere @zou3519 @ZhennanQin
Add tests for ProcessGroup cpp extensions
Labels: oncall: distributed, triaged

### 🐛 Describe the bug

We have no test case covering for this tutorial (https://pytorch.org/tutorials/intermediate/process_group_cpp_extension_tutorial.html#customize-process-group-backends-using-cpp-extensions) to ensure that Process Group cpp extensions are supported. Creating this issue to track progress on this effort.

This ask is to prevent future regressions such as https://github.com/pytorch/pytorch/issues/87173

### Versions

main

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @kwen2501 @awgu
torchdynamo.export doesn't work with data-dependent control flow
Labels: triaged, enhancement

### 🐛 Describe the bug

```
def f(x):
    x_bool = torch.isfinite(x)
    if x.all():
        return x.cos() 
    return x.sin()

torchdynamo.export(f, torch.ones(6, 4), aten_graph=True, tracing_mode="symbolic")
```

### Error logs

torch._dynamo.exc.Unsupported: generic_jump TensorVariable()

### Minified repro

_No response_
eca_botnext26ts_256 fails with TORCHDYNAMO_DYNAMIC_ SHAPES=1: sympy infinite loop
Labels: triaged, bug, oncall: pt2

### 🐛 Describe the bug

```
  File "/raid/ezyang/pytorch-scratch2/torch/nn/modules/module.py", line 1363, in _call_impl
    return forward_call(*input, **kwargs)
  File "<eval_with_key>.54", line 7, in forward
    _unsafe_view = torch.ops.aten._unsafe_view.default(clone, [2048, 16]);  clone = None
  File "/raid/ezyang/pytorch-scratch2/torch/_ops.py", line 257, in __call__
    return self._op(*args, **kwargs or {})
RuntimeError: shape '[2048, 16]' is invalid for input of size 8192
```

repro

```
TORCHDYNAMO_DYNAMIC_SHAPES=1 time python benchmarks/dynamo/timm_models.py  --accuracy --backend aot_eager --training --only eca_botnext26ts_256
```

### Error logs

does anyone even look at these

### Minified repro

Minifier did not work

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh
Minifier should save config variables so you don't have to replicate env vars
Labels: triaged, bug, oncall: pt2

### 🐛 Describe the bug

Today you have to remember to run `minifier_launcher.py` with the same config vars / env vars that the original script. The minifier should record all configs explicitly (maybe only those that changed.)

### Error logs

_No response_

### Minified repro

_No response_

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh
ninja: build stopped: subcommand failed
Labels: needs reproduction, module: build, triaged

### 🐛 Describe the bug

Getting the following output when running `setup.py install` which results in the build failing.
![image](https://user-images.githubusercontent.com/54014218/197547147-b5a20035-1115-4e91-938f-17c37d1410ef.png)


### Versions

![image](https://user-images.githubusercontent.com/54014218/197547296-0f55577b-fc7b-4a07-aa82-5fbe654a149a.png)


cc @malfet @seemethere
sebotnet33ts_256 fails with TORCHDYNAMO_DYNAMIC_SHAPES=1: sympy infinite loop
Labels: triaged, bug, oncall: pt2

### 🐛 Describe the bug

```
  File "/raid/ezyang/pytorch-scratch2/torch/fx/graph_module.py", line 279, in __call__
    raise e                                                                                                    
  File "/raid/ezyang/pytorch-scratch2/torch/fx/graph_module.py", line 269, in __call__
    return super(self.cls, obj).__call__(*args, **kwargs)  # type: ignore[misc]
  File "/raid/ezyang/pytorch-scratch2/torch/nn/modules/module.py", line 1363, in _call_impl
    return forward_call(*input, **kwargs)                                                                      
  File "<eval_with_key>.54", line 7, in forward                                                                
    _unsafe_view = torch.ops.aten._unsafe_view.default(clone, [8192, 32]);  clone = None
  File "/raid/ezyang/pytorch-scratch2/torch/_ops.py", line 257, in __call__         
    return self._op(*args, **kwargs or {})                                                                     
RuntimeError: shape '[8192, 32]' is invalid for input of size 131072 
```

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh @eellison 

### Error logs

https://gist.github.com/da80e0d407c464a21cc49b84ce51ba74

### Minified repro

minifier didn't work

(it looks like it works but the resulting repro doesn't actually trigger error; after patching in [fix minifier 1](https://github.com/pytorch/pytorch/pull/84246/commits/55dfd189542df88860abe99897e42b97794d10e9) and [minor improvements to minifier 1](https://github.com/pytorch/pytorch/pull/84246/commits/d5d8240d66a651d471b0893c6fa601a5fe41597b) the minifier no longer runs.
#error "Expected GLOO_USE_CUDA to be defined"
Labels: module: build, triaged, module: third_party

### 🐛 Describe the bug

when I compiled torch 1.8.0a0+d1cb4a1 from source on GeForce RTX 4090, I always encountered following error:
![image](https://user-images.githubusercontent.com/73052387/197462340-13417fb1-53d8-4694-b608-f188adecb6cd.png)
I was helpless...


### Versions

driver version: 520.56.06
cudatoolkit: 11.3.1
cuda-nvcc: 11.3.122

cc @malfet @seemethere
Crash on backwards step when using `batch_first=True` for LSTMs on MPS (1.14 nightly build)
Labels: module: rnn, triaged, module: mps

### 🐛 Describe the bug

When using `batch_first=True` for LSTM layers, attempting to compute gradients causes a crash when running pytorch on MPS. This crash does not happen when running on CPU.

```python
import torch
import torch.nn as nn

device = "mps" if torch.backends.mps.is_available() else "cpu"
print("Running on", device)

data = torch.zeros((5, 3, 10), device=device)

lstm = nn.LSTM(10, 20, 1, batch_first=True).to(device)
h0 = torch.zeros((1, 5, 20), device=device)
c0 = torch.zeros((1, 5, 20), device=device)

expected = torch.randn((5, 3, 20), device=device)

output, _ = lstm(data, (h0, c0))
output = output.sum()

# crash occurs here
output.backward()
```

This produces the following crash, when running on MPS:

```
loc("total derivative last state"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/a0876c02-1788-11ed-b9c4-96898e02b808/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":219:0)): error: input types 'tensor<1x3x20xf32>' and 'tensor<1x5x20xf32>' are not broadcast compatible
LLVM ERROR: Failed to infer result type(s).
```


Additionally, I searched through the other issues, it seems there is a closed issue [here](https://github.com/pytorch/pytorch/issues/80306) which has a similar bug. However, despite being closed, it seems like it is still broken

### Versions

```
Collecting environment information...
PyTorch version: 1.14.0.dev20221023
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.6 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.102)
CMake version: version 3.24.1
Libc version: N/A

Python version: 3.10.7 (main, Sep 14 2022, 22:38:23) [Clang 14.0.0 (clang-1400.0.29.102)] (64-bit runtime)
Python platform: macOS-12.6-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.4
[pip3] pytorch-lightning==1.4.2
[pip3] torch==1.14.0.dev20221023
[pip3] torch-fidelity==0.3.0
[pip3] torchaudio==0.14.0.dev20221023
[pip3] torchmetrics==0.6.0
[pip3] torchvision==0.15.0.dev20221023
[conda] Could not collect
```

cc @zou3519 @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
Dynamic shapes exhaustive tests should fail (not xfail) if data mismatch
Labels: triaged, module: dynamic shapes

### 🐛 Describe the bug

In other words, it's OK to fail if you fail with an error message, but NOT ok to return silently incorrect outputs. I'm hoping to catch other occurrences of things like https://github.com/pytorch/pytorch/issues/87575 and https://github.com/pytorch/pytorch/issues/87571

### Versions

master
Functionalization does something wrong with pad backward when it uses as_strided
Labels: triaged, module: functionalization

### 🐛 Describe the bug

On master, make sure the unit test is using functionalization:

```
diff --git a/test/functorch/test_aotdispatch.py b/test/functorch/test_aotdispatch.py
index c058b3618e..d848068f3c 100644
--- a/test/functorch/test_aotdispatch.py
+++ b/test/functorch/test_aotdispatch.py
@@ -1317,7 +1317,7 @@ class TestEagerFusionOpInfo(AOTTestCase):
     @skipIfNoSympy
     @patch("functorch.compile.config.use_dynamic_shapes", True)
     @patch("functorch.compile.config.use_fake_tensor", True)
-    @patch("functorch.compile.config.use_functionalize", False)
+    @patch("functorch.compile.config.use_functionalize", True)
     @skipOps('TestEagerFusionOpInfo', 'test_aot_autograd_symbolic_exhaustive',
              aot_autograd_failures | symbolic_aot_autograd_failures)
     def test_aot_autograd_symbolic_exhaustive(self, device, dtype, op):
```

and then run this test:

```
python test/functorch/test_aotdispatch.py -k test_aot_autograd_symbolic_exhaustive_nn_functional_pad_circular_cpu_float32
```

It fails with a value mismatch error.

If you patch in (from https://github.com/pytorch/pytorch/issues/86427)

```
diff --git a/tools/autograd/gen_inplace_or_view_type.py b/tools/autograd/gen_inplace_or_view_type.py
index d79212a093..f9334d8c5b 100644
--- a/tools/autograd/gen_inplace_or_view_type.py
+++ b/tools/autograd/gen_inplace_or_view_type.py
@@ -158,7 +158,7 @@ at::_ops::${unambiguous_name}::call(${unpacked_args})"""
 SETUP_REPLAY_VIEW_IF_NOT_SUPPORT_AS_STRIDED_OR_VIEW_WITH_METADATA_CHANGE = CodeTemplate(
     """\
 std::function<at::Tensor(const at::Tensor&)> func=nullptr;
-if (${is_view_with_metadata_change} || !self.unsafeGetTensorImpl()->support_as_strided()) {
+if (${is_view_with_metadata_change} || !self.unsafeGetTensorImpl()->support_as_strided() || true) {
   ${replay_view_func}
 }
 """
```

the test starts passing. So it would seem that functionalization is not handling as_strided completely correctly in all cases.

cc @bdhirsh @soumith 

### Versions

master
DCE produced obviously wrong graph for pad, but test did not catch it
Labels: triaged, module: aotdispatch

### 🐛 Describe the bug

To reproduce, run on master:

```
AOT_FX_GRAPHS_JOINT=1 AOT_FX_GRAPHS=1 python test/functorch/test_aotdispatch.py -k test_aot_autograd_symbolic_exhaustive_nn_functional_pad_circular_cpu_float32
```

Inspect the third joint graph. You can see the operator is implemented with a bunch of slices and then mutations on the slices. Now, inspect the final forward/backward graph. All of the slice mutation code has been DCE'd away, and you're just returning empty tensors.

Yet somehow, the test passes! Very puzzling.

cc @Chillee 


### Versions

master
Testing insufficient to catch incorrect dispatch key for bernoulli.p re functionalization
Labels: triaged, module: functionalization

### 🐛 Describe the bug

I was investigating this dynamic shapes test failure:

```
python test/functorch/test_aotdispatch.py -k test_aot_autograd_symbolic_exhaustive_nn_functional_feature_alpha_dropout_with_train_cpu_float32
```

The bug boiled down to functionalization.  Pre functionalization:

```
        new_empty: f32[s0, s0, 1, 1] = torch.ops.aten.new_empty.default(primals_1, [sym_size, sym_size_1, 1, 1]
);  sym_size = sym_size_1 = None                                                                               
        bernoulli_: f32[s0, s0, 1, 1] = torch.ops.aten.bernoulli_.float(new_empty) 
```

post functionalization:

```
        new_empty: f32[s0, s0, 1, 1] = torch.ops.aten.new_empty.default(primals_1, [sym_size, sym_size_1, 1, 1]);  sym_size = sym_size_1 = None                                                                               
        empty_like: f32[s0, s0, 1, 1] = torch.ops.aten.empty_like.default(new_empty, memory_format = torch.cont
iguous_format);  new_empty = None                                                                              
        bernoulli_: f32[s0, s0, 1, 1] = torch.ops.aten.bernoulli_.float(empty_like)   
```

The `bernoulli_` never got functionalized! However, the empty_like is a clue: it kind of looks like the functional bernoulli did get applied, but we failed to trace it.

```
- func: bernoulli.p(Tensor self, float p, *, Generator? generator=None) -> Tensor
  device_check: NoCheck   # TensorIterator
  variants: function, method
  tags: nondeterministic_seeded
``` 

Bingo.

I have a fix for this, nbd. (Just add CompositeExplicitAutogradNonFunctional.) What is concerning, however, is the fact that we weren't able to catch this in testing. This is an "obvious" problem, maybe the functionalization codegen should have some test to make sure you aren't registering the functional variants as CompositeImplicitAutograd. Or maybe, why didn't composite compliance catch this? Hmm...

Another useful lint that would have caught this is, after functionalization, assert no more mutable ops in graph.

cc @bdhirsh @soumith @zou3519 

### Versions

master
diagonal of Jacobian matrix
Labels: module: autograd, triaged, enhancement

### 🚀 The feature, motivation and pitch

I'd like to calculate the diagonal of the jacobian matrix.
[d z_1/d x_1, d z_2/d x_2, ..., d z_n/d x_n]
This operator is widely used in the diffusion model.

### Alternatives

My current solution is calculating the whole Jacobian matrix, which takes a large memory and is slow.

### Additional context

_No response_

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
The behavior of cast `NaN` is different on cpu and cuda
Labels: triaged, module: NaNs and Infs, module: edge cases

### 🐛 Describe the bug

The behavior of cast `NaN` is different on cpu and cuda. CPU will return INT_MIN but CUDA returns 0

```py
a = torch.tensor(float('nan'))

b = a.clone().cpu().type(torch.int32)
c = a.clone().cuda().type(torch.int32)
print(b) # tensor(-2147483648, dtype=torch.int32)
print(c) # tensor(0, device='cuda:0', dtype=torch.int32)
```

### Versions

torch: 1.12.1
cuda: 11.6
Improve `c10d::ReduceOp` & `torch.distributed.distributed_c10d.ReduceOp`
Labels: oncall: distributed, triaged

`c10d::ReduceOp` is now a struct which contains an enum class of `RedOptype` in order to support `PREMUL_SUM` (premul_sum is only supported by NCCL backend).
This new reduce op type takes either a Python scalar or a Tensor and that scaling value needs to be stored somewhere while keeping the compatibility with dispatchable reduce ops (note that TorchScript compiler's support is limited) and keeping `torch.distributed.ReduceOp` instances enum like as possible (to name a few, allowing for `__members__` and `isinstance`).

As the op type itself is marked experimental for now but with this requirements and the changes caused, we have to improve the API.
The question is how to have users pass a scale value and how to create ReduceOp (while before premul_sum, there was no need to create a ReduceOp instance as it was closer to Python enum).



Related:
- https://github.com/pytorch/pytorch/pull/87303
- https://github.com/pytorch/pytorch/issues/87191
- https://github.com/pytorch/pytorch/pull/84243

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @wanchaol @carmocca
`bmm` will return wrong result on cpu with in-place
Labels: triaged, module: edge cases

### 🐛 Describe the bug

`bmm` will return wrong result on cpu with in-place. For example, when trying to output the result to the original input, the result is wrong.

More interestingly, the gpu will return the correct value given the same input

```py
import torch
input_1 = torch.randn(2, 1, 5)

def f(input_1):
    y = torch.bmm(input_1, input_1.transpose((- 1), (- 2)), out=input_1)
    return y

inp_cpu = input_1.clone().detach().cpu()
inp_gpu = input_1.clone().detach().cuda()

print(f(inp_cpu))
print(f(inp_gpu))
```

```
tensor([[[0.]], [[0.]]])
tensor([[[ 0.3501]], [[-0.2560]]], device='cuda:0')
```

### Versions

pytorch: 1.12.1
cuda: 11.6
RAM leak when copying tensor from cpu to cuda
Labels: module: cuda, module: memory usage, triaged

### 🐛 Describe the bug

If you profile run the following code then you'll see that line 8 grabs ~2GB of RAM, but it doesn't get released:

`
import gc
import torch
from memory_profiler import profile

@profile
def test(n):
    t1 = torch.ones(n)
    t2 = torch.ones(n).to("cuda")
    del t1
    del t2
    gc.collect()
    torch.cuda.empty_cache()

if __name__ == "__main__":
    test(3000000)
`

### Versions

PyTorch version: 1.12.0+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.27

Python version: 3.8.12 (default, Sep 10 2021, 00:16:05)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.4.214-120.368.amzn2.x86_64-x86_64-with-glibc2.27
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to:
GPU models and configuration: GPU 0: Tesla T4
Nvidia driver version: 510.47.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] botorch==0.2.1
[pip3] gpytorch==1.4.2
[pip3] mypy==0.982
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.4
[pip3] pytorch-lightning==1.7.7
[pip3] torch==1.12.0
[pip3] torchmetrics==0.10.0
[pip3] torchvision==0.13.1

cc @ngimel
invalid_arguments.cpp is busted
Labels: module: bootcamp, triaged, actionable, module: python frontend

### 🐛 Describe the bug

This file is responsible for multi-overload error printing when arguments mismatch. It is supposed to have color formatting, where it tells you which argument mismatched, where green says it matched, and red says it mismatched. It is also completely broken.

<img width="946" alt="image" src="https://user-images.githubusercontent.com/13564/197295055-befa59a9-4de3-4dd3-bd95-5a3bd91d4c6a.png">

Notice that tuple is red, even though it actually is correctly matching for the second argument.

The brokenness appears to stem from the fact that, to determine how to match up the arguments, invalid_arguments.cpp *re-parses* the string names that the python arg parser assigns:

```
std::string FunctionParameter::type_name() const {
  switch (type_) {
    case ParameterType::TENSOR:
      return "Tensor";
    case ParameterType::SCALAR:
      return "Number";
    case ParameterType::INT64:
      return "int";
    case ParameterType::SYM_INT:
      return "int (or SymInt)";
    case ParameterType::DOUBLE:
      return "float";
    case ParameterType::COMPLEX:
      return "complex";
    case ParameterType::TENSOR_LIST:
      return "tuple of Tensors";
```

however, this parsing is completely busted; for example, to parse a tuple appropriately, you should must return a string `tuple[int]`. Furthermore, the names must exactly match what Python typename reports for the type in question.

This is completely bonkers because we have a type system for the parameters already in python_arg_parser.cpp, invalid_arguments.cpp should have reused it.

This would be a good starter task for someone, I think.

### Versions

master

cc @albanD
Missing docstring for resize_as
Labels: module: docs, triaged

### 📚 The doc issue

As per the title, there is no docstring for `resize_as`.

```python
In [2]: t = torch.randn(6)

In [3]: t.resize_as?
Signature: t.resize_as(tensor)
Docstring: <no docstring>
File:      /usr/local/Caskroom/miniconda/base/envs/book/lib/python3.10/site-packages/torch/_tensor.py
Type:      method

In [4]: torch.resize_as_?
Docstring: <no docstring>
Type:      builtin_function_or_method
```

### Suggest a potential alternative/fix

Add docstring.

cc @svekars @carljparker
[JIT] Inconsistent handling of tracing dict output leads to assertion 
Labels: oncall: jit

### 🐛 Describe the bug

Repro: 
```python
import torch
from typing import Dict, List, Optional, Tuple

def impl(
    values: torch.Tensor,
) -> List[List[torch.Tensor]]:
    jt_tensors: List[List[torch.Tensor]] = []
    for i in range(2):
        jt_tensors.append([values[:], values[1:], values[2:], values[3:]])
    return jt_tensors

@torch.jit.script
def scripted_impl(
    keys: List[str],
    values: torch.Tensor,
) -> Dict[str, Tuple[torch.Tensor, torch.Tensor, torch.Tensor, Optional[torch.Tensor]]]:
    jt_tensors = impl(values)
    ret = torch.jit.annotate(
        Dict[str, Tuple[torch.Tensor, torch.Tensor, torch.Tensor, Optional[torch.Tensor]]],
        {k: (v[0], v[1], v[2], v[3]) for (k, v) in zip(keys, jt_tensors)},
    )
    return ret

def thing(values):
    return scripted_impl(["asdf", "fdsa"], values)

values = torch.rand((5))
traced_thing = torch.jit.trace(thing, (values,), strict=False)
```

The tracer seems like it tries to guess the return type based on the ivalue return type (https://github.com/pytorch/pytorch/blob/master/torch/csrc/jit/frontend/tracer.cpp#L293) and apparently it guesses wrong... perhaps there's a better way to do the annotations here, but I couldn't figure out a way to get around it.

### Versions

master branch
Categorical fails simplex validation after its own normalisation on CUDA
Labels: module: distributions, triaged

### 🐛 Describe the bug

`torch.distributions.Categorical` fails on simplex validation after normalising the input probabilities. 
So far, I could only reproduce this bug with CUDA.

I have uploaded two tensors to reproduce the behaviour here: https://oc.embl.de/index.php/s/fAhBE8KESs0i6vm

```python
import torch

p = torch.load("fishy_tensor.pt")

torch.distributions.Categorical(p)  # fails with simplex check
torch.distributions.constraints.simplex.check(p) # yields all true
```

![image](https://user-images.githubusercontent.com/16120273/197239666-b1baf4fc-e94f-49b9-b3c1-7a7d447bc754.png)

I raised this issue in the forum already https://discuss.pytorch.org/t/distributions-categorical-fails-with-constraint-simplex-but-manual-check-passes/163209/3 

### Versions

## Environment
cudatoolkit 11.3.1 h9edb442_10 conda-forge
python 3.10.5 h582c2e5_0_cpython conda-forge
python_abi 3.10 2_cp310 conda-forge
pytorch 1.12.0 py3.10_cuda11.3_cudnn8.3.2_0 pytorch
pytorch-lightning 1.6.5 pypi_0 pypi
pytorch-mutex 1.0 cuda pytorch

complete env here: https://oc.embl.de/index.php/s/UwFFBFDJy6s71t0

cc @fritzo @neerajprad @alicanb @nikitaved
[BUG] moco fails without suppress errors: RuntimeError: Tensors must be CUDA and dense                                                              
Labels: triaged, bug, oncall: pt2

### 🐛 Describe the bug

Patch in https://github.com/pytorch/pytorch/pull/87440 to turn off error suppression

run `python benchmarks/dynamo/torchbench.py  --accuracy --backend eager --training --only moco`

fails with

```
  File "/raid/ezyang/pytorch-scratch2/torch/_dynamo/variables/tensor.py", line 132, in <lambda>           
    lambda: _run_node(tx.output, node, args, kwargs, nnmodule)                                            
  File "/raid/ezyang/pytorch-scratch2/torch/_dynamo/variables/tensor.py", line 51, in _run_node       
    return node.target(*args, **kwargs)                                                                   
  File "/raid/ezyang/pytorch-scratch2/torch/distributed/distributed_c10d.py", line 2264, in all_gather    
    work = default_pg.allgather([tensor_list], [tensor])                                                  
  File "/raid/ezyang/pytorch-scratch2/torch/utils/_python_dispatch.py", line 101, in __torch_dispatch__   
    return old.__torch_dispatch__(func, types, args, kwargs)                                              
  File "/raid/ezyang/pytorch-scratch2/torch/_subclasses/fake_tensor.py", line 822, in __torch_dispatch__  
    r = func(*args, **kwargs)                                                                             
  File "/raid/ezyang/pytorch-scratch2/torch/_ops.py", line 257, in __call__
    return self._op(*args, **kwargs or {})                                                                
RuntimeError: Tensors must be CUDA and dense
```

### Error logs


https://gist.github.com/5a54f01612589820947d15ac2256a9a2

### Minified repro

minifier does not work because captured dynamo graph has distributed operations in it, but process groups are not initialized

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh