gopikrsmscs/torch-issues huggingface dataset:
Support for saving multiple storages/tensors that view same data as different dtypes
Labels: feature, module: serialization, triaged

### 🚀 The feature, motivation and pitch

Saving and loading multiple tensors or storages that view the same data with dfferent dtypes is not currently possible:

```python
>>> import torch
>>> t0 = torch.randn(10)
>>> t1 = t0.view(torch.int)
>>> torch.save([t0, t1], 'tmp.pt')
...
RuntimeError: Cannot save multiple tensors or storages that view the same data as different types
```

In the past, it would have been pretty difficult to add support for this because of the way storage types were defined (different Python and C++ classes for each storage dtype).

But now that storages have been refactored so that they all use the same untyped storage class underneath, enabling serialization of views of the same data with different dtypes should be much more straightforward now

### Alternatives

_No response_

### Additional context

_No response_

cc @mruberry @ezyang
Expand torch.utils._pytree.tree_map
Labels: feature, triaged, actionable, module: pytree, module: functorch

`tree_map(f, tree)` maps f to every leaf of tree. However, in many callsites in the codebase, we do something like a "tree zip" where we want to map a function `f(x, y)` over two trees, tree1 and tree2. JAX's tree_map has these semantics: https://jax.readthedocs.io/en/latest/_autosummary/jax.tree_util.tree_map.html

We should update our tree_map to be similar; e.g. `tree_map(f, tree, *rest)`.

cc @soumith @Chillee @samdow
Profiler is not properly recording seq number when any key above autograd is used
Labels: oncall: profiler

This is because we don't record function on redispatch. Meaning that the profiler is ONLY seeing the top key.
Anything changing the top key (dispatch subclass, dispatch mode, functorch, etc) will break the assignment.
https://github.com/pytorch/pytorch/blob/55749b9c41857050d4e875d4613c9ba7cddeae02/aten/src/ATen/core/dispatch/Dispatcher.h#L645

cc @robieta @chaekit @aaronenyeshi @ngimel @nbcsm @guotuofeng @guyang3532 @gaoteng-git @tiffzhaofb
AOT Autograd should allow backend compilers to see input mutations
Labels: triaged, oncall: pt2, module: functorch

Today, AOT Autograd has special handling for input mutations. Given a user code / captured graph like this:
```
def user_f(x):
    x.mul_(2)
    out = x * 3
    return out
```

AOT Autograd will split it into (1) a purely functional graph for the compiler, and (2) an epilogue that runs after, and applies any global side effects (aka input mutations):

```
// gets sent to a compiler
def compiled_functional_graph(x):
    x_updated = x.mul(2)
    out = x_updated * 3
    return x_updated_out

// wrapper that aot autograd calls
def wrapper_run_compiled_graph_and_epilogue(x):
    x_updated, out = compiled_functional_graph(x)
    x.copy_(x_updated)
    return out
```


This technically leaves some performance on the table: an out-of-place mul() + a copy_() takes twice as many operations as an inplace mul.

Once we start running functionalization all the time in AOT Autograd, and once we capture the optimizer step in a graph that gets sent to AOT Autograd, then this will likely have a larger impact on the performance of optimizer graphs: An optimizer graph consists of a large number of mutations to graph inputs (the parameters), and all of those mutations will now take 2 operations instead of one, and require extra memory - their out-of-place equivalent, and a copy_() call.

The most general way to fix this would be to give backend compilers the ability to indicate to AOT Autograd that they're ok with seeing input mutations in their graph and handling them.

One complication is that AOT Autograd also runs min-cut-partitioning and DCE, which either won't work, or will require even more special handling if there are input mutations in the graph. A simpler version of this github issue would be to only allow input mutations in the graph when we are not generating a backward graph and running partitioning, since graphs capturing the optimizer step should (mostly) fall into this bucket.

cc @ezyang @soumith @msaroufim @wconstab @ngimel @zou3519 @Chillee @samdow
[🚀 Feature Request] pdf and sampling from Alpha-stable distribution 
Labels: module: distributions, feature, triaged

### 🚀 The feature, motivation and pitch

I'm a long-time user of pytorch but a first-time contributor, and this is my first issue. I propose the implementation of the pdf and sampling of alpha-stable distribution into Pytorch. 

Recently, my team published a paper on the Neurips 2022 Workshop titled [Score-Based Generative Models using Lévy Processes](https://openreview.net/forum?id=ErzyBArv6Ue). In this study, gaussian noise on diffusion model is replaced with  alpha stable noise. Alpha-stable distribution is unfortunately not supported by PyTorch. Therefore, I implemented this feature to a different repository named [torchlevy](https://github.com/UNIST-LIM-Lab/torchlevy). However, it will be preferable if the pdf and sample features are included in PyTorch so that other AI researchers may quickly and readily utilize them.

### How to implement

- Implementation of pdf and sampling method is based on Zolotarev thm([Stable Diffusion](https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID2894444_code545.pdf?abstractid=2894444&mirid=1), page 7.)
- I want to add new file on `torch/distribution` for these features.

### Issue

- Pdf caculation requires numerical integration which is not supported on Pytorch. In [torchlevy](https://github.com/UNIST-LIM-Lab/torchlevy), I’ve used another python module called torchquad for efficient numerical integration. But I’m not sure I can use this module on Pytorch. If I cannot use this module, I want to implement only sampling from alpha stable distribution.

cc [@fritzo](https://github.com/fritzo) [@neerajprad](https://github.com/neerajprad) [@alicanb](https://github.com/alicanb) [@nikitaved](https://github.com/nikitaved)

### Alternatives

_No response_

### Additional context


I would be happy to work on solving this when the issue has been discussed further.

cc @fritzo @neerajprad @alicanb @nikitaved
torch.fx tracer emits type error when tracing module that directly contains and uses the torch.cat() function
Labels: oncall: fx

### 🐛 Describe the bug

### Problem:
When I use torch.fx's tracer to trace an `nn.Module` that uses the `torch.cat()` function directly (please see the `MyCat` module below), the tracing fails with the following type error message:
```
TypeError: cat() received an invalid combination of arguments - got (Proxy, int), but expected one of:
 * (tuple of Tensors tensors, int dim, *, Tensor out)
 * (tuple of Tensors tensors, name dim, *, Tensor out)
```
Interestingly however, when I use the torch.fx tracer to trace a nested `nn.Module` that uses `torch.cat()` both in the module itself as well as in the child module nested within (please see the `NestedCat` module below), the tracing completes with no error messages emitted. I am not sure if this is the expected behavior (and if so, the reasoning and mechanism behind this), or if this is a bug. I have also found that others seem to have very similar issues, and a fix in one GitHub issue ([https://github.com/pytorch/pytorch/issues/34294](https://github.com/pytorch/pytorch/issues/34294)) seems to address this problem; however, when the following code is run, I observed the aforementioned behavior and the problem still persists. If I understand correctly, I think the correct behavior should be that the tracing for both of the following modules should complete without emitting any error messages.

### Related issues:
[https://github.com/pytorch/pytorch/issues/79715](https://github.com/pytorch/pytorch/issues/79715)
[https://github.com/pytorch/pytorch/issues/34294](https://github.com/pytorch/pytorch/issues/34294) (Seems to contain an (incomplete) fix?)
[https://github.com/pytorch/pytorch/pull/43248](https://github.com/pytorch/pytorch/pull/43248)

### Code for reproducing the issue:
The `graph = tracer.trace(nested_cat)` line completes successfully with no problem, but the `graph = tracer.trace(my_cat)` line emits the type error messages.
```py
import torch
from torch import nn
from torch import fx

class MyCat(nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, inputs):
        cat = torch.cat(inputs, 1)
        return cat

class NestedCat(nn.Module):
    def __init__(self):
        super().__init__()
        self.my_cat_0 = MyCat()

    def forward(self, x):
        my_list = [self.my_cat_0([x])]
        cat = torch.cat(my_list, 1)
        return cat

tracer = fx.Tracer()

nested_cat = NestedCat()
graph = tracer.trace(nested_cat)

my_cat = MyCat()
graph = tracer.trace(my_cat)
```

### Versions

```
Windows 10

Python 3.9.10

CUDA 11.6

torch              1.13.1+cu117
torchaudio         0.13.1+cu117
torchvision        0.14.1+cu117
numpy              1.22.3
```

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @SherlockNoMad @EikanWang @jgong5 @wenzhe-nrv
torch.compile for calling func(**kwargs)
Labels: triaged, oncall: pt2

### 🐛 Describe the bug

See code:
```python
def func(a, b):
    return a + b

def func1(**kwargs):
    return func(a=1, **kwargs)

c_f = torch.compile(func1, fullgraph=True)
c_f(b=torch.rand([2]))
```


At the moment, when you compile with `torch.compile(func, fullgraph=True)`, it throws `Unsupported: missing: BUILD_MAP_UNPACK_WITH_CALL`. When you compile in normal mode, it produces a graph break, because the python op_code for "func(... **kwargs)" is not supported.

I know that torchscript does not support **kwargs. However, I wonder if torch.compile(fullgraph) is going to support this?



### Versions

Pytorch version: pre-release version 2.0.0a0+fb


cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
tensor.to_sparse() handling indices incorrectly under dynamo/fake tensor
Labels: triaged, bug, oncall: pt2, module: dynamic shapes

### 🐛 Describe the bug

to_sparse() is returning a FakeTensor where the indices attribute has the wrong shape/size.

### Error logs

_No response_

### Minified repro



Repro 1:
```python
import torch
import torch._dynamo
import logging

# torch._dynamo.config.verbose = True
# torch._dynamo.config.log_level = logging.DEBUG

def fn():
    x = torch.rand((7, 3))
    return x.to_sparse().indices().stride()

print(fn()) # returns (21, 1)
print(torch._dynamo.optimize("eager")(fn)()) # returns (1, 1)
```

Repro 2:
```python
import torch
import torch._dynamo
import logging

# torch._dynamo.config.verbose = True
# torch._dynamo.config.log_level = logging.DEBUG

fake_mode = torch._subclasses.fake_tensor.FakeTensorMode()

print("~~ fake mode below")
with fake_mode:
    x = torch.rand((7, 3))
    y = x.to_sparse()

print(f"fake mode indices size: {y.indices().size()}") # torch.Size([2, 0]); should be [2, 21]
```

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
Make quant_min/quant_max required for observer/fake_quant
Labels: oncall: quantization, triaged

### 🐛 Describe the bug

https://github.com/pytorch/pytorch/pull/91159#discussion_r1053901160

### Versions

master

cc @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel
Potential bug found with pybind11 dec_ref while gil released
Labels: triaged, module: pybind, shadow review, bug

### 🐛 Describe the bug

I tried updating pybind11 in this PR: https://github.com/pytorch/pytorch/pull/91248/ and in the new version, we added an assert to make sure that the GIL is held during all inc_ref / dec_ref of the python object's underlying reference count. This seems to be failing on one of the tests https://github.com/pytorch/pytorch/actions/runs/3750738878/jobs/6371337256 . Since the GIL isn't held here, this could cause invalid reference counts to develop which could corrupt the python / gc state and lead to memory leaks and or crashes.

### Versions

Ran on master https://github.com/pytorch/pytorch/actions/runs/3750738878/jobs/6371337256

cc @ezyang
Use dynamo to detect incorrect op schemas automatically
Labels: triaged, oncall: pt2, module: dynamo

A known issue that's bitten people in the past is when someone adds a new pytorch operator to the dispatcher that either mutates or aliases its inputs, but doesn't add proper alias annotations to the op's [schema](https://github.com/pytorch/pytorch/blob/15af4b1ceea3b689354ad664675b930486804c8e/test/test_schema_check.py#L331).

This will cause silent correctness issues across in the stack, both when using autograd, and when using functionalization (which runs by default when using `torch.compile()`).

We now have some [infra](https://github.com/pytorch/pytorch/blob/master/test/test_schema_check.py#L331) for automatically detecting improperly annotated schema using a `TorchDispatchMode`, which runs for a handful of ATen ops. However, it doesn't "automatically" run for custom user operators, and can cause models to silently break when using `torch.compile()`.

There's an open question of what the UX should look like though, because these checks are expensive to run. (Yet another) new debugging knob? Have it run automatically in the `aot_eager` backend?

One option would be to run this mode automatically under the `torch.compile(backend="aot_eager")`, likely inside of aot_autograd's tracing. The `aot_eager` backend is getting more debug checks added to it soon, so there's some precedent for using it as a catch-all for adding extra asserts.

cc @ezyang @soumith @msaroufim @wconstab @ngimel @mlazos @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire
Segmentation faults in DataLoader (in latest torch version).
Labels: needs reproduction, module: dataloader, triaged

### 🐛 Describe the bug

I'm experiencing segmentation faults in the dataloader.

# To Reproduce

This happens with `num_workers=16` or `12`, `8`, `4`, `3`. 

The Dataloader doesn't segfault when num_workers isn't set.

# Expected behavior

No segfaults

# Relevant snippet from code
```python
dataloader = torch.utils.data.DataLoader(train_data, shuffle=True,
                                         batch_size=batch_size,
                                         num_workers=16)
```

# Trace

```
ERROR: Unexpected segmentation fault encountered in worker.
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[20], line 38
     36 errG = criterion(output, labels)
     37 errG.backward()
---> 38 D_G_z2 = output.mean().item()
     39 optimizerG.step()
     41 # Save Losses for plotting later

File /lab/mlenv/lib/python3.10/site-packages/torch/utils/data/_utils/signal_handling.py:66, in _set_SIGCHLD_handler.<locals>.handler(signum, frame)
     63 def handler(signum, frame):
     64     # This following call uses `waitid` with WNOHANG from C side. Therefore,
     65     # Python can still get and update the process status successfully.
---> 66     _error_if_any_worker_fails()
     67     if previous_handler is not None:
     68         assert callable(previous_handler)

RuntimeError: DataLoader worker (pid 139803) is killed by signal: Segmentation fault. 
```

I've skimmed through earlier issues but those were closed and apparently fixed.

kind regards
Max

### Versions

PyTorch version: 1.13.1+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4090
Nvidia driver version: 525.60.11
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.24.0
[pip3] torch==1.13.1
[pip3] torchvision==0.14.1
[conda] Could not collect


cc @SsnL @VitalyFedyunin @ejguan @NivekT
first class dims leak memeory
Labels: triaged, module: functorch

The new PyCode_Get* functions return new references. So decref are missing and we leak memory right now.
https://github.com/pytorch/pytorch/blob/15af4b1ceea3b689354ad664675b930486804c8e/functorch/csrc/dim/dim.cpp#L1447

cc @zdevito 

cc @zou3519 @Chillee @samdow @soumith
ONNX export question (using torchdynamo)
Labels: triaged, bug, oncall: pt2

### 🐛 Describe the bug

I would like to export a model that can be run either as an ONNX file. My model fails with both the scripting and the trace options. Using PyTorch 2.0 I (uses the torchdynamo) I can create convert my model (to an optimized on) that runs. Is there a way to export the model to ONNX without a dependency on trace/script? 

Model_dynamo = torch.compile(model,backend="onnxrt_cpu",mode="max-autotune")
Here an export for ONNX? 
exported_model = torch._dynamo.export(Model_dynamo , input)

### Error logs

no error

### Minified repro

_No response_

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
JIT mishandles torch.round() in PyTorch 1.10.1
Labels: oncall: jit

### 🐛 Describe the bug

Behavior of `round()` is different when used in a function annotated with `@torch.jit.script`. Per documentation, `round()` should break ties to even; this is the behavior observed in a regular Python function. The JIT sometimes emits code that breaks ties by rounding up, as in the following example:

```python
import torch
import time

@torch.jit.script
def round_jit(x: torch.Tensor, inv_scale: float):
        x_scaled = x / inv_scale
        return x_scaled.round()

def round_baseline(x: torch.Tensor, inv_scale: float):
        x_scaled = x / inv_scale
        return x_scaled.round()

print(f'Torch version: {torch.__version__}')
inv_scale = 1
x0 = torch.zeros([4, 128, 16, 16], dtype=torch.float32)
x0 = x0 * 0 + 0.5
for n in range(2,3):
  for c in range(2,65):
    for h in range(2,9):
      for w in range(2,9):
        x = x0[1:n, 1:c, 1:h, 1:w]
        rbl = round_baseline(x, inv_scale)
        rji = round_jit(x, inv_scale)
time.sleep(1) # Give time for the JIT to compile...
x = x0[0:1, 0:1, 0:1, 0:1]
rbl = round_baseline(x, inv_scale)
rji = round_jit(x, inv_scale)
print(f'Input: {x}')
print(f'Output (baseline): {rbl}')
print(f'Output (JIT): {rji}')
assert rbl.equal(rji)
```

```
Torch version: 1.10.1+cu113
Input: tensor([[[[0.5000]]]])
Output (baseline): tensor([[[[0.]]]])
Output (JIT): tensor([[[[1.]]]])
Traceback (most recent call last):
  File "test-jit.py", line 31, in <module>
    assert rbl.equal(rji)
AssertionError
```

Disclaimer: This bug was discovered as part of work I performed as an IBM employee.

### Versions

PyTorch version: 1.10.1+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Red Hat Enterprise Linux release 8.6 (Ootpa) (x86_64)
GCC version: (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10)
Clang version: 9.0.1 (Red Hat 9.0.1-2.module+el8.2.0+5494+7b8075cf)
CMake version: version 3.20.2
Libc version: glibc-2.3.4

Python version: 3.6.8 (default, Jan 14 2022, 11:04:20)  [GCC 8.5.0 20210514 (Red Hat 8.5.0-7)] (64-bit runtime)
Python platform: Linux-4.18.0-372.26.1.el8_6.x86_64-x86_64-with-redhat-8.6-Ootpa
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.19.5
[pip3] torch==1.10.1+cu113
[pip3] torchaudio==0.10.1+cu113
[pip3] torchvision==0.11.2+cu113
[conda] Could not collect

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel
Odd/hand-wavy mathematical notation for Conv2D
Labels: module: docs, triaged

### 📚 The doc issue

Beginning with https://github.com/pytorch/pytorch/blob/cdbca3563e12097a8bf359e937d6d6196fa6dc89/torch/nn/modules/conv.py#L188
there is a description of the convolution result per out-channel  $C_{out_j}$ and batch sample $N_i$. While this is used to indicate that $i$ and $j$ iterate over $N$ and $C_{out}$, respectively, it is unusual/misleading notation (correct me if I am wrong). Despite a strong mathematical background it has confused me and my peers. E.g., using $N$ and $N_i$ in the same context usually means that $N_i$ is something like $N$ (i.e. a cardinality), but we generalize for different instances indexed by $i$, where $N$ would otherwise occur as a constant. This is not the case here.

### Suggest a potential alternative/fix

Use the more common and less confusing notation $(i\leq N,\ j\leq C_{out})$ or $(i\in [N],\ j \in [C_{out}])$:

```
\text{out}(i,j) = \text{bias}(j) +
        \sum_{k = 0}^{C_{in} - 1} \text{weight}(j, k)
        \star \text{input}(i, k) \quad i \leq N,\ j \leq C_{out}

cc @svekars @carljparker
DISABLED test_index_add_correctness (__main__.TestTorch)
Labels: triaged, module: flaky-tests, skipped, module: scatter & gather ops

Platforms: linux, rocm, win, windows

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_index_add_correctness&suite=TestTorch) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/10207821466).

Over the past 3 hours, it has been determined flaky in 5 workflow(s) with 6 failures and 5 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT BE ALARMED IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_index_add_correctness`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


cc @mikaylagawarecki
onednn(mkldnn) backend support for quantized operators
Labels: oncall: quantization, triaged, module: arm

### 🚀 The feature, motivation and pitch

Feature request: Onednn backend support for quantized operators, INT8 at least.

Motivation: Given the acceptable accuracies from INT8 quantized inference and significant performance gains, the feature is getting attention for server class use cases as well, not just the mobile cpus.
PyTorch currently supports only QNNPACK backend for quantization on arm cpus with the assumption that the feature is of interest only for mobile devices. As you know QNNPACK doesn't support any of the neoverse optimizations. However, there are optimized gemm libraries like Arm Compute Library already supported via onednn backend. So, enabling onednn for quantized operators similar to how it's been done for full precision operators (fp32 and bf16) would enable applications to make use of the optimized gemm kernels for latest arm socs.


### Alternatives

I haven't considered fbgemm as an alternative here because currently fbgemm doesn't have optimized kernels for aarch64.

### Additional context

_No response_

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel @malfet
not able to import pipelines as torch.distributed is missing
Labels: oncall: distributed, pipeline parallelism

### 🐛 Describe the bug

RuntimeError: Failed to import transformers.pipelines because of the following error (look up to see its traceback): No module named 'torch.distributed'

This is the error while im trying to import pipeline from transformer. My tensorflow is upraded to latesr version 2.11.0 and i am running this on windows 11 on visual code using python 3.10

Detailed Error if this helps:

File c:\Users\varun\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\utils\import_utils.py:1093, in _LazyModule._get_module(self, module_name)
   1092 try:
-> 1093     return importlib.import_module("." + module_name, self.__name__)
   1094 except Exception as e:

File c:\Users\varun\AppData\Local\Programs\Python\Python310\lib\importlib\__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:883, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File c:\Users\varun\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\pipelines\__init__.py:48
     40 from ..utils import (
...
   1097         f" traceback):\n{e}"
   1098     ) from e

### Versions

4.4.0

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu
`quantile` fails for `float16`/`half` inputs
Labels: module: cpu, triaged, enhancement, module: half

### 🐛 Describe the bug

```python
import torch

a = torch.randn(10, 10, dtype=torch.float16)
a.quantile(0.25)
```

results in

```python
RuntimeError: quantile() input tensor must be either float or double dtype
```

Looking at the [relevant piece of code](https://github.com/pytorch/pytorch/blob/511fbad830543c703c0b78a18ed015a4333cdbc2/aten/src/ATen/native/Sorting.cpp#L215), the intention was probably to disallow integer dtypes.

*Edit: the following code produces the same error, so it's not a CPU-specific issue*
```python
a = torch.randn(10, 10, dtype=torch.float16).cuda()
a.quantile(0.25)
```

### Versions

PyTorch version: 1.13.0+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Manjaro Linux (x86_64)
GCC version: (GCC) 12.2.0
Clang version: 14.0.6
CMake version: version 3.24.3
Libc version: glibc-2.36

Python version: 3.10.8 (main, Nov  1 2022, 14:18:21) [GCC 12.2.0] (64-bit runtime)
Python platform: Linux-5.15.78-1-MANJARO-x86_64-with-glibc2.36
Is CUDA available: True
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce GTX 1050
Nvidia driver version: 520.56.06
cuDNN version: Probably one of the following:
/usr/lib/libcudnn.so.8.6.0
/usr/lib/libcudnn_adv_infer.so.8.6.0
/usr/lib/libcudnn_adv_train.so.8.6.0
/usr/lib/libcudnn_cnn_infer.so.8.6.0
/usr/lib/libcudnn_cnn_train.so.8.6.0
/usr/lib/libcudnn_ops_infer.so.8.6.0
/usr/lib/libcudnn_ops_train.so.8.6.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] lovely-numpy==0.2.1
[pip3] mypy==0.982
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.5
[pip3] pytorch-lightning==1.7.7
[pip3] torch==1.13.0
[pip3] torchmetrics==0.10.1
[pip3] torchvision==0.14.0
[conda] Could not collect

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10
[Composable] Enable summon_full_params for fully_shard
Labels: oncall: distributed, triaged, module: fsdp

### 🚀 The feature, motivation and pitch

FSDP.summon_full_params does not work if pass in a composable module as the API uses some class-based only FSDP paths. 

There is also a larger discussion around whether it is correct / right to use FSDP class methods in composable API, or if we should have a different interface.

### Alternatives

_No response_

### Additional context

_No response_

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu
[BE] Investigate FSDP test _zero_model 
Labels: oncall: distributed, triaged, module: fsdp

### 🐛 Describe the bug

We shouldn't technically need summon_full_params in `_zero_model` in FSDP test, but removing this fails some DDP parity tests in `test_fsdp_core`. We should investigate and fix.

### Versions

main

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu
Is there a way to write passes?
Labels: triaged, enhancement, oncall: pt2

Is there a way torch.compile stack allows writing passes? For example, can I register a pass (FX IR => FX IR) that gets run after Dynamo generates FX graphs and before FX IR gets passed to inductor? 

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
[Dynamo] Graph Re-compilation Invoked by Changes of Unused Dict Values
Labels: triaged, oncall: pt2, module: dynamo

### 🐛 Describe the bug

We find that unused dict values are taken as constant in TorchDynamo, thus once the unused values change across iterations then the graph will re-compile. One example is that user may append the raw text prompt to the input dict, but the actually used values are the tokens of text prompt. I have brief search in the Github issues but didn't find realted issue, so I think this might be one bug.

Here's one minimal example to reproduce:

```python
#!/usr/bin/env python

import torch
from torch._dynamo import optimize, config
config.cache_size_limit = 4

@optimize("inductor")
def toy_example(adict):
    # adict["text"] not used here
    return adict["data"].norm()

def test():
    adict = {"data": torch.ones(10), "text": ""}
    for i in range(5):
        adict["text"] = f"text-{i}"
        toy_example(adict)

if __name__ == "__main__":
    test()
```

The result I got with PyTorch `1.14.0a0+410ce96`:

```
$ python bug-1.py
[2022-12-19 14:40:43,753] torch._dynamo.convert_frame: [WARNING] torch._dynamo hit config.cache_size_limit (4)
   function: 'toy_example' (bug-1.py:7)
   reasons:  ["adict['text'] == 'text-0'"]
to diagnose recompilation issues, see https://github.com/pytorch/torchdynamo/blob/main/TROUBLESHOOTING.md.
```

### Versions

```
Collecting environment information...
PyTorch version: 1.14.0a0+410ce96
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.5 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.24.1
Libc version: glibc-2.31

Python version: 3.8.10 (default, Jun 22 2022, 20:18:18)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.4.0-99-generic-x86_64-with-glibc2.29
Is CUDA available: True
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA A100-SXM4-80GB
Nvidia driver version: 515.65.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.7.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.2
[pip3] open-clip-torch==2.7.0
[pip3] pytorch-quantization==2.1.2
[pip3] torch==1.14.0a0+410ce96
[pip3] torch-tensorrt==1.3.0a0
[pip3] torchtext==0.13.0a0+fae8e8c
[pip3] torchvision==0.15.0a0
[conda] Could not collect
```

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @mlazos @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire @jjsjann123 @kevinstephano
Implement L1 and L2 gradient as hooks with the option of changing the weight decay value.
Labels: module: nn, module: optimizer, triaged, needs design

### 🚀 The feature, motivation and pitch

The L1 and L2 weight decay are important regularizations in deep learning, but their current implementation in PyTorch is not as flexible as one may desire. The L2 weight decay is available in PyTorch optimizers, but the weight decay value cannot be modified during training. L1 is not implemented in an efficient manner in the PyTorch library.
I propose implementing the L1 and L2 weight decay as gradient hooks and placing them in the `torch.nn.utils` namespace. Other regularizations, such as weight and spectral normalization, are already implemented there, as are pruning utilities, which often go with L1 regularization.
The implementation should allow easy attachment and detachment of the hook on any `nn.Module` with options to separate parameter groups. The weight decay value should be modifiable to allow weight decay scheduling.

### Alternatives

Implementing in the Optimizer base class is an option but has failed to gain traction.
Currently, the TorchLayers library offers a Keras-like interface with the L1 and L2 regularizations, but this requires a wrapper class and cannot separate parameter groups very efficiently or change the weight decay value.

### Additional context

I have already implemented the hooks and am willing to contribute. Separation of parameter groups has already been implemented, but I am considering a better interface for exposing the weight decay value for scheduling.

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are @vincentqb @janeyx99
If minifier test fails, stderr/stdout of subprocess calls is not printed
Labels: triaged, bug

### 🐛 Describe the bug

```
    # Runs the minifier launcher script in `repro_dir`, patched with `patch_code`.
    def _run_minifier_launcher(self, patch_code, repro_dir):
        self.assertIsNotNone(repro_dir)
        launch_file = os.path.join(repro_dir, "minifier_launcher.py")
        self.assertTrue(os.path.exists(launch_file))
        launch_code = self._inject_code(patch_code, launch_file)

        launch_proc = subprocess.run(
            ["python3", launch_file],
            capture_output=True,
            cwd=repro_dir,
        )

        return launch_proc, launch_code

```

this captures output, but then there is nothing that ensures this output is printed if, e.g., something else fails

### Error logs

_No response_

### Minified repro

_No response_
Simple deleting from the sys cache fails on reimport
Labels: triaged, enhancement, module: python frontend

### 🐛 Describe the bug

### Bug description
I am working on a project in which I need to throw `torch` package out of the sys cache in order to be able to reload new version of the torch package. However, this minimal example fails:
```
import torch
import sys

if __name__ == "__main__":
    sys.modules.pop("torch", None)
    import torch
```
With the following error:
```
Traceback (most recent call last):
  File "/home/andi/Memgraph/code/mage/python/test_mage/test_submodules.py", line 10, in <module>
    import torch
  File "/home/andi/.local/lib/python3.10/site-packages/torch/__init__.py", line 233, in <module>
    for name in dir(_C):
NameError: name '_C' is not defined
```

### Versions

```
Python platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.22.2
[pip3] numpydoc==1.5.0
[pip3] torch==1.12.1+cpu
[pip3] torch-tb-profiler==0.4.0
[pip3] torchaudio==0.12.1+cpu
[pip3] torchvision==0.13.1
[conda] blas                      1.0                         mkl  
[conda] cpuonly                   2.0                           0    pytorch
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py39h7f8727e_0  
[conda] mkl_fft                   1.3.1            py39hd3c417c_0  
[conda] mkl_random                1.2.2            py39h51133e4_0  
[conda] numpy                     1.23.4           py39h14f4228_0  
[conda] numpy-base                1.23.4           py39h31eccc5_0  
[conda] pytorch                   1.13.1              py3.9_cpu_0    pytorch
[conda] pytorch-mutex             1.0                         cpu    pytorch
[conda] torchaudio                0.13.1                 py39_cpu    pytorch
[conda] torchvision               0.14.1                 py39_cpu    pytorch
```


cc @albanD
A Simple Function Causing Graph Break
Labels: triaged, bug

### 🐛 Describe the bug

An error raised when calling `torch._dynamo.optimize` when optimizing huggingface GPT2 and enforcing a single optimized graph for the whole model.

```
  File "conda/dexport/lib/python3.9/site-packages/torch/_dynamo/variables/user_defined.py", line 243, in call_function
    return super().call_function(tx, args, kwargs)
  File "conda/dexport/lib/python3.9/site-packages/torch/_dynamo/variables/base.py", line 230, in call_function
    unimplemented(f"call_function {self} {args} {kwargs}")
  File "conda/dexport/lib/python3.9/site-packages/torch/_dynamo/exc.py", line 67, in unimplemented
    raise Unsupported(msg)
torch._dynamo.exc.Unsupported: call_function UserDefinedObjectVariable(_lru_cache_wrapper) [] {}

from user code:
   File "dexport/tests/loop.py", line 12, in forward
    if transformers.is_torch_tpu_available():

Set torch._dynamo.config.verbose=True for more information


You can suppress this exception and fall back to eager by setting:
    torch._dynamo.config.suppress_errors = True
```

Below is a minimal repro. If `transformers.is_torch_tpu_available()` is removed, then the code works fine.

```python
import torch
import torch._dynamo
from torch import nn
import transformers

class ToyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(10, 10)

    def forward(self, x):
        if transformers.is_torch_tpu_available():
            x = x + 1.0
        else:
            x = x + 2.0
        return self.linear(x)


model = ToyModel()
x = torch.rand(10, 10, dtype=torch.float32)
y = model(x)

class GraphCaptureCompiler:
    def __init__(self):
        self.captured_graph = None
        self.captured_graph_count =  0
    def compile(self, gm, _):
        assert self.captured_graph_count == 0
        self.captured_graph = gm
        self.captured_graph_count += 1
        return gm
compiler = GraphCaptureCompiler()
y_optimized = torch._dynamo.optimize(compiler.compile, nopython=True)(model)(x)
```

I'd expect dynamo can just trace through this simple function.

### Error logs

_No response_

### Minified repro

_No response_
node.stack_trace does not handle escaping correctly
Labels: oncall: fx

### 🐛 Describe the bug

While working on our importer in Torch-MLIR, I noticed that `node.stack_trace` is just a string rather than a more structured piece of data. This repro is extracted from that importer code. I am needing to regex match `node.stack_trace` to extract the source location, but of course that is perilous. The example below, if put in a file `repro".py` fails to extract a correct stack trace. I could make my regex smarter but I think the solution is to expose the source location info in a more structured manner.

```
from typing import List

import operator
import re

import torch
import torch._dynamo as dynamo

def my_backend(gm: torch.fx.GraphModule, example_inputs: List[torch.Tensor]):
    for node in gm.graph.nodes:
        if node.op == "call_function" and node.target == operator.mul:
            print(f"node.stack_trace is of type {type(node.stack_trace)}")
            print(node.stack_trace)
            m = re.search(
                r"""File "([^"]+)", line ([0-9]+),""", node.stack_trace)
            if m is None:
                print("FAILED TO MATCH")
            else:
                file, line = m.group(1), int(m.group(2))
                print(f"MATCHED {file}:{line}")
    return gm

@dynamo.optimize(my_backend)
def f(x):
    return x * x

example_inputs = (torch.randn(3, 4),)
f(*example_inputs)
```

Output if the file is called `repro".py`
```
$ python /tmp/repro'"'.py
node.stack_trace is of type <class 'str'>
Module stack: {}
  File "/tmp/repro".py", line 27, in f
    return x * x

FAILED TO MATCH
```

Example output if the file is called `repro2.py`
```
$ python /tmp/repro2.py
node.stack_trace is of type <class 'str'>
Module stack: {}
  File "/tmp/repro2.py", line 27, in f
    return x * x

MATCHED /tmp/repro2.py:27
```

### Versions

PyTorch version: 2.0.0.dev20221213+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux rodete (x86_64)
GCC version: (Debian 12.2.0-3) 12.2.0
Clang version: 14.0.6-2
CMake version: version 3.25.0
Libc version: glibc-2.35

Python version: 3.10.8 (main, Nov  3 2022, 15:17:13) [GCC 12.2.0] (64-bit runtime)
Python platform: Linux-5.19.11-1rodete1-amd64-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.24.0rc2
[pip3] torch==2.0.0.dev20221213+cpu
[pip3] torchvision==0.15.0.dev20221213+cpu
[conda] Could not collect

cc @ezyang @SherlockNoMad @EikanWang @jgong5 @wenzhe-nrv
overflow (?) on cuda tensor after matrix multiplication
Labels: module: numerical-stability, module: cuda, triaged

### 🐛 Describe the bug

Two ways of matrix division gives not the same results if using cuda context.
Using CPU context working as expected, difference matrix is filled zeroes..
Using Cuda gives difference matrix, where 255 pos is not zero.


```import torch

def test_func(GT):
        GT_1 = GT.clone()
        GT_2 = GT.clone()
        
        GT_1 /= GT_1[..., 2].unsqueeze(-1)
        GT_2 = GT_2[...] / GT_2[..., 2].unsqueeze(-1)

        dif = GT_1 - GT_2
        print(f"argmin: {dif.argmin().item()}, min: {dif.min().item()}")
        print(f"argmax: {dif.argmax().item()}, max: {dif.max().item()}")

if __name__ == "__main__":
        canonical = torch.rand([100,3]).type(torch.float32)
        Homo = torch.rand([3,3]).type(torch.float32)
        
        print("_________CPU EXEC_________")
        GT = canonical @ Homo.transpose(-1,-2)
        test_func(GT)
        
        print("_________GPU EXEC_________")
        canonical = canonical.cuda()
        Homo = Homo.cuda()
        GT = canonical @ Homo.transpose(-1,-2)
        test_func(GT)


OUTPUT:
_________CPU EXEC_________
argmin: 0, min: 0.0
argmax: 0, max: 0.0
_________GPU EXEC_________
argmin: 255, min: -0.9257388710975647
argmax: 0, max: 0.0
```




### Versions

[pip3] numpy==1.23.5
[pip3] pytorch-lightning==1.8.4.post0
[pip3] pytorchvideo==0.1.5
[pip3] torch==1.12.1+cu116
[pip3] torchmetrics==0.11.0
[pip3] torchvision==0.13.1+cu116

cc @ngimel
Crash in `index_select` with singleton `self`, non-singleton `index`
Labels: triaged

### 🐛 Describe the bug

Running the following can cause pytorch 1.13 to segfault (or give some other memory-related abort):

```python
torch.index_select(torch.tensor(10), 0, torch.tensor([0, 0, 0, 0, 0, 0, 0, 0]))
```
Example error messages:
```
munmap_chunk(): invalid pointer
Aborted (core dumped)
```
```
corrupted size vs. prev_size
Aborted (core dumped)
```
It doesn't happen every time, but i'm pretty much guaranteed to get it after ~3 runs.

I've also only found it to happen if `index` has at least four elements (though it probably just crashes less often with fewer).

### Versions

PyTorch version: 1.13.0+cpu
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.5 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: 10.0.0-4ubuntu1 
CMake version: version 3.25.1
Libc version: glibc-2.31

Python version: 3.8.10 (default, Nov 14 2022, 12:59:47)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.29
Is CUDA available: False
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: GPU 0: Quadro T2000 with Max-Q Design
Nvidia driver version: 510.108.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.4
[pip3] torch==1.13.0+cpu
[pip3] torchaudio==0.13.0+cpu
[pip3] torchvision==0.14.0+cpu
[conda] Could not collect
as_strided_scatter : INTERNAL_ASSERT_FAILED for requires_grad=True and non-config input
Labels: triaged, module: functionalization

### 🐛 Describe the bug

```python
import torch

from torch.testing._internal.common_methods_invocations import op_db

as_strided_scatter_opinfo = list(filter(lambda op_info : op_info.name == 'as_strided_scatter', op_db))[0]

device = 'cpu'
dtype = torch.float
requires_grad = True
for sample in as_strided_scatter_opinfo.sample_inputs(device, dtype, requires_grad=True):
    noncontig_sample = sample.noncontiguous()
    args = [sample.input] + list(sample.args)
    kwargs = sample.kwargs
    out = as_strided_scatter_opinfo.op(*args, **kwargs)
    out.sum().backward()
```

Fails with
`RuntimeError: !self.requires_grad() || self.is_contiguous() INTERNAL ASSERT FAILED at "ATen/native/TensorShape.cpp":3827, please report a bug to PyTorch. as_strided_scatter is currently only supported for contiguous inputs
`

This should probably be a `TORCH_CHECK` instead of `TORCH_INTERNAL_ASSERT`.

cc: @bdhirsh 

### Versions

master

cc @bdhirsh @ezyang @soumith
[Composable] Enable setting state_dict_type
Labels: high priority, triage review, oncall: distributed, triaged, module: fsdp

### 🚀 The feature, motivation and pitch

There are a couple issues we need to fix to enable set_state_dict type for composable path, such as the issue of `fsdp_modules` which is an FSDP method being used in set_state_dict_type.

One suggestion is to use the common composable State APIs landed in https://github.com/pytorch/pytorch/pull/89147/files. 

### Alternatives

_No response_

### Additional context

_No response_

cc @ezyang @gchanan @zou3519 @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu