gopikrsmscs/torch-issues huggingface dataset:
faster WeightedRandomSampler implementation based on alias method
Labels: module: dataloader, triaged

### 🚀 The feature, motivation and pitch

Since the weights are discrete and often fixed, I think the `WeightedRandomSampler` with replacement could be based on [alias method](https://en.wikipedia.org/wiki/Alias_method). In this implementation, the random values can be drawn from the distribution in O(1) time. The weight list has the same size as the dataset. When it comes to large datasets, the new method can be better.
Here is my solution, I implemented a `WeightedRandomSampler2` with scipy:
```python
from torch.utils.data import WeightedRandomSampler
from scipy.stats.sampling import DiscreteAliasUrn

class WeightedRandomSampler2(WeightedRandomSampler):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        urng = np.random.default_rng()
        self.rng = DiscreteAliasUrn(self.weights, random_state=urng)
        
    def __iter__(self):
        if self.replacement:
            yield from iter(self.rng.rvs(self.num_samples))
        else:
            return super().__iter__()
```

### Alternatives

https://www.keithschwarz.com/darts-dice-coins/
https://github.com/Tecnarca/Vose-Alias-Method
https://gist.github.com/orlp/e9b31d3397a7dd3e34d6bc862ce3b88d

### Additional context

```python
import numpy as np
w = np.random.random(10_000_000)
w /= w.sum()
```

```python
%%time
s1 = WeightedRandomSampler(w, 3)
```
>Wall time: 0 ns

```python
%%time
list(s1)
```
>Wall time: 63.1 ms
[729931, 4124692, 4903810]

```python
%%time
s2 = WeightedRandomSampler2(w, 3)
```
>Wall time: 235 ms

```python
%%time
list(s2)
```
>Wall time: 0 ns
[3887197, 7289214, 2292607]

We can see that although the initialization of `WeightedRandomSampler2` is slower, its random number generation is much faster. Typically, the initialization runs only once, while the generation runs tens of thousands of times. 
Furthermore, if given a larger weight list, say `w = np.random.random(100_000_000)`, the `WeightedRandomSampler` will raise a `RuntimeError: number of categories cannot exceed 2^24`, but the number generation of the `WeightedRandomSampler2` still tasks `Wall time: 0 ns`.
Therefore, I propose to implement the sampler using alias method.

cc @SsnL @VitalyFedyunin @ejguan @NivekT
`cat` fails the gradient computation in forward mode with empty tensors when used with legacy vmap
Labels: triaged, module: edge cases, module: forward ad

### 🐛 Describe the bug

`cat` fails the gradient computation in forward mode but succeeds in reverse mode

```py
import torch
from torch.autograd.functional import jacobian

torch.manual_seed(420)

x = torch.randn(4, 3, 32, 32)
empty = torch.Tensor([])

def func(x, empty):
    res1 = torch.cat([x, empty], dim=1)
    return res1

jacobian(func, (x, empty), vectorize=True, strategy="reverse-mode")
# succeed 
jacobian(func, (x, empty), vectorize=True, strategy="forward-mode")
# RuntimeError: Tensors must have same number of dimensions: got 5 and 2
```

### Versions

```
PyTorch version: 2.0.0.dev20230105
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: Could not collect
CMake version: version 3.22.1
Libc version: glibc-2.35

Python version: 3.9.15 (main, Nov 24 2022, 14:31:59)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.99
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090
GPU 2: NVIDIA GeForce RTX 3090

Nvidia driver version: 515.86.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.4.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.5
[pip3] torch==2.0.0.dev20230105
[pip3] torchaudio==2.0.0.dev20230105
[pip3] torchvision==0.15.0.dev20230105
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.4.0           h06a4308_640
[conda] mkl-service               2.4.0            py39h7f8727e_0
[conda] mkl_fft                   1.3.1            py39hd3c417c_0
[conda] mkl_random                1.2.2            py39h51133e4_0
[conda] numpy                     1.23.5           py39h14f4228_0
[conda] numpy-base                1.23.5           py39h31eccc5_0
[conda] pytorch                   2.0.0.dev20230105 py3.9_cuda11.7_cudnn8.5.0_0    pytorch-nightly
[conda] pytorch-cuda              11.7                 h67b0de4_2    pytorch-nightly
[conda] pytorch-mutex             1.0                        cuda    pytorch-nightly
[conda] torchaudio                2.0.0.dev20230105      py39_cu117    pytorch-nightly
[conda] torchtriton               2.0.0+0d7e753227            py39    pytorch-nightly
[conda] torchvision               0.15.0.dev20230105      py39_cu117    pytorch-nightly
```
Tracker for `scatter_reduce` additional reduction options requests
Labels: triaged, module: scatter & gather ops

Feel free to add further requests to this issue

- [ ] logsumexp #31394
- [ ] indices of max and min #80439 #83980
- [ ] [composite reductions in pytorch_scatter ](https://github.com/rusty1s/pytorch_scatter/tree/master/torch_scatter/composite) (softmax, std, logsumexp)
Set AVX2 is minimum supported instruction set for Linux X86
Labels: triaged, enhancement, module: intel

### 🐛 Describe the bug

Some of the ops already throwing this cryptic `Your CPU does not support FBGEMM`, which simply means CPU does not support AVX2 instruction set. Which makes me wonder, should we make it a uniform rule for the entire PyTorch codebase on x86?
https://github.com/pytorch/pytorch/blob/f84f89b1c3f2bc74512e7a7b05ae6185164a9b3e/aten/src/ATen/native/cpu/utils.h#L104

### Versions

CI

cc @frank-wei @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10
Type promotion mismatch between eager and inductor pow
Labels: triaged, oncall: pt2, module: cpu inductor

### 🐛 Describe the bug

`CpuTests.test_pow2` is failing because of the mismatch between scalar type promotion for eager and inductor, see 
https://github.com/pytorch/pytorch/blob/37a28255cb9c2a78fd2a27ed7921e8c9672a57ab/aten/src/ATen/native/cpu/PowKernel.cpp#L112-L115
vs
```c++
extern "C" void kernel(const float* __restrict__ in_ptr0,
                       float* __restrict__ out_ptr0,
                       float* __restrict__ out_ptr1)
{
    #pragma omp parallel num_threads(12)
    {
        {
            #pragma omp for 
            for(long i0=0; i0<256; i0+=1)
            {
                auto tmp1 = in_ptr0[i0];
                auto tmp0 = static_cast<float>(1000);
                auto tmp2 = std::pow(tmp0, tmp1);
                auto tmp3 = std::pow(tmp1, tmp0);
                out_ptr0[i0] = tmp2;
                out_ptr1[i0] = tmp3;
            }
        }
    }
}
```
generated for
https://github.com/pytorch/pytorch/blob/7db4d813c3e30cdc6c9937e0c2ff68f4a84edf49/test/inductor/test_torchinductor.py#L2991-L2992


Moving type to `torch.float64` make test pass

### Error logs

_No response_

### Minified repro

_No response_

### Versions

CI

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
test_nccl_warn_not_in_group_debug_detail is flaky
Labels: oncall: distributed, triaged, module: flaky-tests

### 🐛 Describe the bug

Error: https://github.com/pytorch/pytorch/actions/runs/4066116189/jobs/7002264428

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu 

### Versions

main
`linalg.lstsq` fails the gradient computation in forward mode
Labels: triaged, module: forward ad

### 🐛 Describe the bug

`linalg.lstsq` fails the gradient computation in forward mode but succeed in the reverse mode

```py
import torch
from torch.autograd.functional import jacobian

A = torch.randn(2, 3, 3)
b = torch.randn(2, 3)

def func(A, b):
    x = torch.linalg.lstsq(A, b, )
    return x

print(jacobian(func, (A, b), vectorize=True, strategy="reverse-mode"))
# succeed
print(jacobian(func, (A, b), vectorize=True, strategy="forward-mode"))
# fail
# RuntimeError: mat1 and mat2 shapes cannot be multiplied (6x3 and 2x3)
```

### Versions

```
PyTorch version: 2.0.0.dev20230105
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: Could not collect
CMake version: version 3.22.1
Libc version: glibc-2.35

Python version: 3.9.15 (main, Nov 24 2022, 14:31:59)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.99
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090
GPU 2: NVIDIA GeForce RTX 3090

Nvidia driver version: 515.86.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.4.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.5
[pip3] torch==2.0.0.dev20230105
[pip3] torchaudio==2.0.0.dev20230105
[pip3] torchvision==0.15.0.dev20230105
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.4.0           h06a4308_640
[conda] mkl-service               2.4.0            py39h7f8727e_0
[conda] mkl_fft                   1.3.1            py39hd3c417c_0
[conda] mkl_random                1.2.2            py39h51133e4_0
[conda] numpy                     1.23.5           py39h14f4228_0
[conda] numpy-base                1.23.5           py39h31eccc5_0
[conda] pytorch                   2.0.0.dev20230105 py3.9_cuda11.7_cudnn8.5.0_0    pytorch-nightly
[conda] pytorch-cuda              11.7                 h67b0de4_2    pytorch-nightly
[conda] pytorch-mutex             1.0                        cuda    pytorch-nightly
[conda] torchaudio                2.0.0.dev20230105      py39_cu117    pytorch-nightly
[conda] torchtriton               2.0.0+0d7e753227            py39    pytorch-nightly
[conda] torchvision               0.15.0.dev20230105      py39_cu117    pytorch-nightly
```
[RFC] Support Huge Model Init Without mallocs for Compile/Distributed Use Cases
Labels: oncall: distributed, triaged

### 🚀 The feature, motivation and pitch

<todo -- lets gather existing thoughts here and develop the RFC>

### Alternatives

_No response_

### Additional context

_No response_

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu
[FSDP] `summon_full_params(writeback=True, rank0_only=True)`
Labels: oncall: distributed, triaged, module: fsdp

`writeback=True` and `rank0_only=True` currently raises an error. We should be able to implement this setting by having rank 0 broadcast the unsharded `FlatParameter` to all ranks before writing back.

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501
onnx_torch.ModelProto exceeded maximum protobuf size of 2GB
Labels: module: onnx, triaged

### 🐛 Describe the bug

When I export onnx for large model, I get this error:

```bash
[libprotobuf ERROR /opt/pytorch/pytorch/third_party/protobuf/src/google/protobuf/message_lite.cc:457] onnx_tor
ch.ModelProto exceeded maximum protobuf size of 2GB: 4649393993
```

My code is:

```python
import onnx
import torch
from diffusers import UNet2DConditionModel

unet = UNet2DConditionModel.from_pretrained("CompVis/stable-diffusion-v1-4",
                                            torch_dtype=torch.float16,
																						revision="fp16",
                                            subfolder="unet",
																						use_auth_token=YOUR_TOKEN)
unet.cuda()

with torch.inference_mode(), torch.autocast("cuda"):
    inputs = torch.randn(2,4,64,64, dtype=torch.half, device='cuda'), torch.randn(1, dtype=torch.half, device='cuda'), torch.randn(2, 77, 768, dtype=torch.half, device='cuda')

    # Export the model
    torch.onnx.export(unet,               # model being run
                    inputs,                         # model input (or a tuple for multiple inputs)
                    "unet_v1_4_fp16_pytorch.onnx",   # where to save the model (can be a file or file-like object)
                    export_params=True,        # store the trained parameter weights inside the model file
                    opset_version=12,          # the ONNX version to export the model to
                    do_constant_folding=True,  # whether to execute constant folding for optimization
                    input_names = ['input_0', 'input_1', 'input_2'],
                    output_names = ['output_0'])
```

### Versions

docker run --gpus all -it --rm nvcr.io/nvidia/pytorch:22.12-py3
Dynamo uses CONSTANT_MATCH guards for string inputs
Labels: triaged, oncall: pt2, module: dynamo

### 🐛 Describe the bug

There are cases when we define pytorch modules that take strings or list[str] as inputs. For example, torchtext.vocab which does vocab lookup for string tokens, and modules that use torchtext.vocab underlying.

With jit script this works fine and string inputs are treated as variable input, but with dynamo they are somehow treated as constants, leading to recompilation every time the values change.

I'm not sure if this is expected behavior. If yes, this can be quite more restrictive than torchscript, especially for model export

### Error logs

![截屏2023-02-02 下午5 09 45](https://user-images.githubusercontent.com/90181230/216287269-fc42e8f0-eb2b-4fd9-801d-38f768d758d4.png)

The compiled graph should not have `[1, 0]` as constants

### Minified repro

_No response_

### Versions

PyTorch version: 2.0.0.dev20230201+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.10.5 | packaged by conda-forge | (main, Jun 14 2022, 07:04:59) [GCC 10.3.0] (64-bit runtime)
Python platform: Linux-5.10.130-118.517.amzn2.x86_64-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.4
[pip3] pytorch-lightning==1.5.10
[pip3] torch==2.0.0.dev20230201+cpu
[pip3] torch-tb-profiler==0.4.0
[pip3] torchdata==0.6.0.dev20230201
[pip3] torchmetrics==0.9.3
[pip3] torchtext==0.15.0.dev20230201+cpu
[conda] nomkl                     1.0                  h5ca1d4c_0    conda-forge
[conda] numpy                     1.22.4          py310h4ef5377_0    conda-forge
[conda] pytorch-lightning         1.5.10                   pypi_0    pypi
[conda] torch                     2.0.0.dev20230201+cpu          pypi_0    pypi
[conda] torch-tb-profiler         0.4.0                    pypi_0    pypi
[conda] torchdata                 0.6.0.dev20230201          pypi_0    pypi
[conda] torchmetrics              0.9.3                    pypi_0    pypi
[conda] torchtext                 0.15.0.dev20230201+cpu          pypi_0    pypi

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @mlazos @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire
[BUG] jit.trace not working for torchvision ViT models
Labels: oncall: jit

### 🐛 Describe the bug

torch.jit.trace is not working for torchvision vision transformer models, i.e. vit_b_16, vit_b_32, and vit_l_16, vit_l_32. It made a TracingCheckError.

### Versions

pytorch 1.13.0
torchvision v0.14

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel
[Dynamo] Don't graph break on einops
Labels: high priority, triaged, enhancement, oncall: pt2, module: dynamic shapes, module: dynamo

### 🚀 The feature, motivation and pitch

Einops is a very popular library in PyTorch code, so we should aim to not graph break on it. For example, see https://github.com/lucidrains/vit-pytorch/blob/main/vit_pytorch/cait.py#L95

It currently causes dynamo to graph break.
```
import torch
from einops import rearrange
from torch._dynamo import allow_in_graph

# allow_in_graph(rearrange)

@torch.compile(fullgraph=True)
def f(x):
    x = x.cos()
    return rearrange(x, 'b n -> (b n)').sin()

f(torch.randn(20, 20, device='cuda'))
```

If you uncomment the `allow_in_graph` though, it works.
`cast(Tensor, tensor)` causes the break (for pytorch tensors it's a no-op). 

### Alternatives

_No response_

### Additional context

_No response_

cc @ezyang @gchanan @zou3519 @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @soumith @ngimel @desertfire @mlazos @yanboliang
Why does the torch model have no memory leaks under gpu, but there is a memory leak under cpu, torch version 1.10.1
Labels: needs reproduction, triaged

### 🐛 Describe the bug

Why does the torch model have no memory leaks under gpu, but there is a memory leak under cpu, torch version 1.10.1

### Versions

Why does the torch model have no memory leaks under gpu, but there is a memory leak under cpu, torch version 1.10.1
Importing tensorflow (2.12) before torch (2.0) hangs at import torch
Labels: oncall: binaries, triaged

### 🐛 Describe the bug

I am trying to set up a bleeding edge machine (2x4090, TF 2.12-dev, torch 2.0-dev) and stumbled on something strange.
If I run this code
```
import tensorflow as tf
print('TF-version:',tf.__version__)
import torch
print('Torch-version:',torch.__version__)
```
It hangs on "import torch". And I have to ctrl-D out of it
It then prints
```
terminate called after throwing an instance of 'std::runtime_error'
  what():  random_device could not be read
```

While if I change the order of tensorflow and torch
```
import torch
print('Torch-version:',torch.__version__)
import tensorflow as tf
print('TF-version:',tf.__version__)
```
it works!

printing
```
Torch-version: 2.0.0.dev20230131+cu118
2023-02-01 21:59:13.899074: E tensorflow/tsl/lib/monitoring/collection_registry.cc:81] Cannot register 2 metrics with the same name: /tensorflow/core/bfc_allocator_delay
TF-version: 2.12.0-dev20230127
```


### Versions

Collecting environment information...
PyTorch version: 2.0.0.dev20230131+cu118
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: 14.0.0-1ubuntu1
CMake version: version 3.25.0
Libc version: glibc-2.35

Python version: 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.15.0-58-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 12.0.140
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: 
GPU 0: NVIDIA GeForce RTX 4090
GPU 1: NVIDIA GeForce RTX 4090

Nvidia driver version: 525.85.12
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.7.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.24.1
[pip3] pytorch-triton==2.0.0+0d7e753227
[pip3] torch==2.0.0.dev20230131+cu118
[pip3] torchaudio==2.0.0.dev20230131+cu118
[pip3] torchvision==0.15.0.dev20230131+cu118
[conda] Could not collect

TF-version: 2.12.0-dev20230127


cc @ezyang @seemethere @malfet
`PYTORCH_DEBUG_MODE`, better invalid index embedding lookup error message on cuda
Labels: high priority, triaged, needs design, module: python frontend

### 🚀 The feature, motivation and pitch

You can read the detailed presentation of the problem and the ensuing discussion here:
https://pytorch.slack.com/archives/C3PDTEV8E/p1675216222317729

The outcome of discussion with @ngimel and @Chillee is this:

1. introduce a new `PYTORCH_DEBUG_MODE` env var which allows for slow validations which we don't want in the normal mode. This mode is to be activated by users when things go wrong and additional hinting is needed from pytorch. Item (2) will be the first user of this mode, but then I hope more similar user-friendly validations will be added in the future. e.g. hinting at bad sizes of matrices which don't align with `% 16`, and then wave and tile quantization as another example. the other candidates would be any op that will now be able to pre-validate its inputs on the python side and thus give a better error than the async cuda error.
2. rewrite `nn.Embedding.__call__` to validate on the python side if `PYTORCH_DEBUG_MODE=1` that the index inputs aren't larger than the target matrix size and provide a user-friendly assert before the call is dispatched to cuda - since once it's gone to cuda the error message is a big disaster and very often doesn't tell where the problem is - see the slack thread for examples of such horrors.
3. bonus: while at it make the `IndexError: index out of range in self` on cpu more user friendly to align with other pytorch ops where actual sizes/dimensions are reported as part of the assert message, for example:

```
e = torch.nn.Embedding(10,10)
x = torch.tensor([[10]])
e(x)
---
before: IndexError: index out of range in self
after: IndexError: index 10 is out of max range 9
```
The actual check can be something as simple as:
```
if torch.where(x.flatten() >= e.weight.shape[0])[0].shape[0] > 0:
    raise ValueError(f"the inputs contain indices that are higher than {e.weight.shape[0]-1} which is the highest index of the embedding")
```
I'm not sure if we want to dump all the bad indices, perhaps just the highest one? But I trust you'd find a much more generic way to do that. I'm surely not thinking about the edge cases and only made it work with the simple example above.

p.s. in [this Issue thread](https://github.com/huggingface/transformers/issues/21378) that prompted this feature request you can see how badly it sometimes gets and the error messages are completely wrong and aren't pointing to the source of the problem


cc @ezyang @gchanan @zou3519 @albanD
Inductor miscompilation with dynamic shapes from Background_Matting
Labels: triaged, oncall: pt2, module: dynamic shapes, module: inductor

### 🐛 Describe the bug

Repro script: https://gist.github.com/100ea386eaad5041ed18303e277e39e2

Minifier did not work, due to https://github.com/pytorch/pytorch/issues/93857

I confirmed that this script fails accuracy with dynamic shapes, and doesn't fail without

Full model repro command is `TORCHDYNAMO_REPRO_FORWARD_ONLY=1 TORCHDYNAMO_REPRO_AFTER=dynamo TORCHDYNAMO_REPRO_LEVEL=4 python benchmarks/dynamo/torchbench.py --accuracy --backend inductor --explain --only Background_Matting  --float32  --dynamic-shapes --disable-cudagraphs` but to actually get the minifier to produce anything at all you need https://github.com/pytorch/pytorch/pull/93856 https://github.com/pytorch/pytorch/pull/93853 https://github.com/pytorch/pytorch/pull/93850 https://github.com/pytorch/pytorch/pull/93403

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh @mlazos @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire @Chillee 

### Versions

master
Minifier related: perhaps same_two_models should reseed between the regular and optimized runs?
Labels: triaged, oncall: pt2, module: minifier

### 🐛 Describe the bug

same_two_models doesn't reseed before running the two models. Maybe it should?

One countervailing factor here is that, we don't generally preserve RNG algorithm on compilation. So they won't match anyway, even if we reseed.

Maybe same_two_models should run the source model twice to see if there is RNG involved? But we also need to be careful about input mutations.

### Versions

master

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh
Bitwise-perfect method for (de)serializing tensors in base64
Labels: feature, module: serialization, triaged

### 🚀 The feature, motivation and pitch

- Convenient for pasting directly into repro code/comment on GitHub without attaching other files (also considering severe file extension limitations of GitHub file attachment)
- Currently (at least, some time ago), some properties of Tensors (such as coalesced-ness) might be lost after existing torch.save+torch.load roundtrip, so need a bitwise perfect C++ Tensor object reconstruction with its layout/sparse structures/bitfields/storage/strides for reproducing/demonstrating problems with these, so a concise text-based format (typically for small tensors) would be useful!

Original context and proposal in https://github.com/pytorch/pytorch/issues/73479#issuecomment-1056903431

Related on general use of utility for base64 torch.save/torch.load (?) format for repro purposes: https://github.com/pytorch/pytorch/issues/93366#issuecomment-1411988364

cc @mruberry @pearu

### Alternatives

_No response_

### Additional context

_No response_
Enable CUPTI
Labels: module: windows, triaged

Pytorch for Windows does not support CUPTI (Cuda Profiling Tools Interface) at this moment.
This is needed for Kineto (Pytorch Profiling library).

There was a try to enable it in [this PR](https://github.com/pytorch/pytorch/pull/65608), but after merge CI/CD started to fail so PR was reverted.

Expectations
- Enable CUPTI for Pytorch on Windows

cc @peterjc123 @mszhanyi @skyline75489 @nbcsm
torchdim can not be compiled for Python-3.11 on Windows
Labels: module: windows, triaged, module: functorch

### 🐛 Describe the bug

As it uses _PyOpcode_Deopt and _PyOpcode_Caches that are not guaranteed to be a public symbols
So attempts to compile fails as follows:
```
dim.cpp.obj : error LNK2019: unresolved external symbol _PyOpcode_Caches referenced in function "struct _object * __cdecl _dims<&struct py::object __cdecl create_dim(struct py::object,struct py::handle)>(struct _object *,struct _object * const *,__int64,struct _object *)" (??$_dims@$1?create_dim@@YA?AUobject@py@@U23@Uhandle@3@@Z@@YAPEAU_object@@PEAU0@PEBQEAU0@_J0@Z)
```

### Versions

CI

cc @peterjc123 @mszhanyi @skyline75489 @nbcsm @zou3519 @Chillee @samdow @soumith @kshitij12345 @janeyx99
save_config/load_config for torch._dynamo.config and friends hardcodes file paths
Labels: triaged, oncall: pt2, module: dynamo

### 🐛 Describe the bug

bad!!!

### Versions

master

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire
Failures in cuda11.7-py3.10-gcc7-sm86-periodic-dynamo-benchmarks
Labels: triaged, oncall: pt2

When Migrating our CI from CUDA 11.6 to CUDA 11.7. Here: https://github.com/pytorch/pytorch/pull/93406
I see multiple failures in cuda11.7-py3.10-gcc7-sm86-periodic-dynamo-benchmarks workflow.

Github Workflow failure: https://github.com/pytorch/pytorch/actions/runs/4060149836/jobs/6989215115

aot_eager_all [internal link](https://www.internalfb.com/intern/paste/P610917589/): 
```
Error: al  maml                                [2023-02-01 03:15:46,456] torch._dynamo.utils: [ERROR] Accuracy failed: allclose not within tol=0.0001

Error: ain tinynet_a                           [2023-02-01 04:32:59,763] torch._dynamo.utils: [ERROR] RMSE (res-fp64): 0.00769, (ref-fp64): 0.00072 and shape=torch.Size([32])
Error: 2-01 04:32:59,763] torch._dynamo.utils: [ERROR] Accuracy failed for key name bn1.weight.grad
FAIL

Error: ain gernet_l                            [2023-02-01 04:20:23,425] torch._dynamo.utils: [ERROR] RMSE (res-fp64): 0.02019, (ref-fp64): 0.00534 and shape=torch.Size([640])
Error: 2-01 04:20:23,425] torch._dynamo.utils: [ERROR] Accuracy failed for key name stages.3.0.shortcut.bn.running_var
FAIL

Error: ain gluon_xception65                    [2023-02-01 04:21:35,059] torch._dynamo.utils: [ERROR] RMSE (res-fp64): 0.00408, (ref-fp64): 0.00054 and shape=torch.Size([728])
Error: 2-01 04:21:35,060] torch._dynamo.utils: [ERROR] Accuracy failed for key name mid.block17.rep.bn1.weight.grad
FAIL
```

dynamic_aot_eager_torchbench [internal link](https://www.internalfb.com/intern/paste/P610917994):
```
Error: al  maml                                [2023-02-01 03:15:46,456] torch._dynamo.utils: [ERROR] Accuracy failed: allclose not within tol=0.0001
```

dynamic_aot_eager_timm 1 [internal link](https://www.internalfb.com/intern/paste/P610918613)
```
Error: ain gernet_l                            [2023-02-01 03:27:46,292] torch._dynamo.utils: [ERROR] RMSE (res-fp64): 0.02019, (ref-fp64): 0.00534 and shape=torch.Size([640])
Error: 2-01 03:27:46,292] torch._dynamo.utils: [ERROR] Accuracy failed for key name stages.3.0.shortcut.bn.running_var
FAIL
Error: ain gluon_xception65                    [2023-02-01 03:30:28,459] torch._dynamo.utils: [ERROR] RMSE (res-fp64): 0.00408, (ref-fp64): 0.00054 and shape=torch.Size([728])
Error: 2-01 03:30:28,459] torch._dynamo.utils: [ERROR] Accuracy failed for key name mid.block17.rep.bn1.weight.grad
FAIL
```

dynamic_aot_eager_timm 2  [internal link](https://www.internalfb.com/intern/paste/P610919025):
```
Error: ain tinynet_a                           [2023-02-01 03:47:04,591] torch._dynamo.utils: [ERROR] RMSE (res-fp64): 0.00769, (ref-fp64): 0.00072 and shape=torch.Size([32])
Error: 2-01 03:47:04,591] torch._dynamo.utils: [ERROR] Accuracy failed for key name bn1.weight.grad
FAIL
```

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @malfet @ptrblck 
### Versions

CI 31.01.2023
EmbeddingBag to support mini-batches with offsets
Labels: triaged, enhancement, module: nestedtensor

### 🚀 The feature, motivation and pitch

Currently, the `forward` method of `EmbeddingBag`, when offsets are passed, supports only 1D inputs. Hence, training / inference on mini-batches of data isn't supported with offsets.

Offsets are very useful when training on tabular datasets with "multi-valued" cells, such as movie genres, since we may want to sum / average the embeddings associated with several genres to a single vector.  There can also be weighted multi-valued cells, for example, when the multiple values are generated by an auxiliary model, and the weights represent the confidence of the model in its prediction. For example, consider automatic extraction of movie genres from their title and description.



### Alternatives

Two possible alternatives:

1. Using a regular `torch.nn.Embedding` class, extract the embedding vectors, multiply by weights manually, and aggregate them. In this case we lose the efficiency of the EmbeddingBag class, which doesn't have to actually create the full embedding tensor. This idea is relevant only if the number of features in each mini-batch item is the same.
2. Use an EmbeddingBag in our model, decompose the mini-batch to its constituent items, and compute the output of the model for each item using a for-loop.

### Additional context

_No response_

cc @cpuhrsch @jbschlosser @bhosmer @drisspg @mikaylagawarecki
ONNX Export Fails: Model input type is Dict[str, Tensor] 
Labels: module: onnx, triaged

### 🐛 Describe the bug

I tried to export a pytorch model to onnx and the model input type is Dict[str, Tensor],  but it fails with _**Couldn't lower all tuples**_ errror.
sorry I can't upload model as it's our internal one.

The same input, the model can output normally as below:
```python
import torch
from typing import Dict, List, Tuple
device = torch.device("cuda")
model = torch.load("model.pt")
input_dict: Dict[str, torch.Tensor] ={'input1': torch.tensor([1],dtype=torch.float64,device=device),
'input2': torch.tensor([1],dtype=torch.float64,device=device),
'input3': torch.tensor([1],dtype=torch.float64,device=device)
}
model(input_dict)
```
And I will get output as below:

```python
Output(prediction=tensor([[19.3254]], device='cuda:0', grad_fn=<ReluBackward0>))
```

But when I use the same input for onnx export：
```python
torch.onnx.export(user_model, (input_dict, {}), "onnx_model.onnx", verbose = True, input_names = ["input1","input2","input3"], output_names = ["output_spec"])
```
```python
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/torch/onnx/utils.py", line 504, in export
    _export(
  File "/usr/local/lib/python3.8/dist-packages/torch/onnx/utils.py", line 1529, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/usr/local/lib/python3.8/dist-packages/torch/onnx/utils.py", line 1115, in _model_to_graph
    graph = _optimize_graph(
  File "/usr/local/lib/python3.8/dist-packages/torch/onnx/utils.py", line 582, in _optimize_graph
    _C._jit_pass_lower_all_tuples(graph)
RuntimeError: Couldn't lower all tuples.
```



### Versions

PyTorch version: 1.13.1+cu117
CUDA used to build PyTorch: 11.7
OS: Ubuntu 20.04.5 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
CMake version: version 3.24.3
Libc version: glibc-2.31

Python version: 3.8.10 
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: 
GPU 0: Tesla V100-SXM2-32GB
Nvidia driver version: 450.51.06
Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.22.4
[pip3] torch==1.13.1
[pip3] torchsummary==1.5.1
Aot accuracy minifier with dynamic shapes doesn't work
Labels: triaged, oncall: pt2

### 🐛 Describe the bug

The typical symptom of this problem is that you have an accuracy problem, you successfully get the minifier to dump a minifier launcher (this in and of itself is nontrivial, see other issues I've filed), but then the minifier launcher claims there's nothing wrong with your program. Closer inspection (see also https://github.com/pytorch/pytorch/issues/93364 ) reveals that the minifier launcher is compiling your program differently than the actual, live execution. What's going on?

In fact, the algorithm the aot accuracy minifier and real execution take are quite different. So there actually isn't any reason to expect them to give the same result.

The aot minifier strategy looks like this:

1. Dump the GraphModule of post AOTAutograd operations (torch.ops.aten) into the test file
2. At minifier launch time, retrace it with `make_fx(gm)` (sic; notice that we don't pass `tracing_mode` here; more on this shortly)
3. Pass it directly `compile_fx_inner` from inductor

This is... problematic. Here are the reasons I know about, though there may be more.

* Most obviously, `make_fx` is being called without any tracing mode. So you will in fact get a static shape trace here. Oops.
* But let's say you fix that. Well, `make_fx` has a different algorithm for symbolicating the input fake tensors than Dynamo does. In particular, Dynamo creates a ShapeEnv, does some stuff which Dynamo might only know about, and then we reuse that when we run AOTAutograd. `make_fx` knows about none of this. In particular, you will lose guards that are not evident from the ATen graph trace.

The "correct" way to fix this is to try to more faithfully recreate the dynamic shapes environment as seen from real time execution.

### Versions

master

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh
Option for minifier to dump the actual tensor inputs/parameters to be used
Labels: triaged, oncall: pt2

### 🐛 Describe the bug

Often randn is fine, but sometimes it is not. Would be nice to be easily run the minifier launcher against the original inputs to see if the input data actually mattered.

### Versions

master

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh
Minifier should also dump compilation artifacts from the real execution for ease of sanity checking
Labels: triaged, oncall: pt2

### 🐛 Describe the bug

I just diagnosed a few minifier bugs which root caused down to "minifier didn't accurately replicate state from the original run and so the minifier ran compiled code that wasn't the same compiled code as the original run." It would be really good if it were easier to sanity check minifier results in this case.

One of the easiest things that can be done is to dump the inductor debug output from the real run (which contains the IR for all the Triton programs we compiled), and then make it easy to diff this against the debug output from the minifier launcher run. Hand inspection can make it clear if the code is not compiling the same way. Similarly, dumping the FX graph being fed into Dynamo can also be instructive.

### Versions

master

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh
Inductor miscompilation with dynamic shapes from LearningToPaint
Labels: triaged, oncall: pt2

### 🐛 Describe the bug

To run the repro script, you must patch in https://github.com/pytorch/pytorch/pull/93308 because of dynamic shape related problems in minifier infrastructure

The repro script is https://gist.github.com/0054b61f8e9cc5135e6e6d6f5d2caf0d

When run, it fails with

```
[2023-01-31 06:20:26,667] torch._dynamo.utils: [ERROR] Accuracy failed: allclose not within tol=0.001
Traceback (most recent call last):
  File "/data/users/ezyang/b/pytorch/repro.py", line 75, in <module>
    raise AccuracyError("Dynamo failed")
__main__.AccuracyError: Dynamo failed
```

By turning off dynamic shapes by modifying `USE_DYNAMIC_SHAPES`, you can see that inductor compiles without accuracy error. So it should be a dynamic shapes related miscompilation.

The minifier wasn't able to simplify this repro any further, unfortunately. The aot minifier does not work as it has fundamental problems with dynamic shapes.

This miscompilation is minified from LearningToPaint. Repro command is `TORCHDYNAMO_REPRO_AFTER=dynamo TORCHDYNAMO_REPRO_LEVEL=4 python benchmarks/dynamo/torchbench.py --accuracy --backend inductor --explain --only LearningToPaint  --float32  --dynamic-shapes --disable-cudagraphs` but you need at least the patches I quoted above and possibly more to actually get the minifier to work. My full branch state at time of successful minification was https://github.com/ezyang/pytorch/tree/LearningToPaint-successful-minify

**UPDATE.** Updated repro to avoid running backwards

### Versions

master

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh
Minifier launcher incorrectly runs backwards even when original reproducer didn't run backwards
Labels: triaged, oncall: pt2

### 🐛 Describe the bug

The launch script looks like

```
torch._dynamo.debug_utils.MINIFIER_SPAWNED = True
compiler_fn = BACKENDS["dynamo_accuracy_minifier_backend"]

dynamo_minifier_backend = functools.partial(
    compiler_fn,
    compiler_name="inductor",
)       
opt_mod = torch._dynamo.optimize(dynamo_minifier_backend)(mod)

with torch.cuda.amp.autocast(enabled=False):
    opt_mod(*args)
```

This calls

```
@register_backend
def dynamo_accuracy_minifier_backend(gm, example_inputs, compiler_name):
    from functorch.compile import minifier

    from torch._dynamo.optimizations.backends import BACKENDS

    if compiler_name == "inductor":
        from torch._inductor.compile_fx import compile_fx

        compiler_fn = compile_fx
    else:
        compiler_fn = BACKENDS[compiler_name]

    # Set the eval mode to remove randomness.
    gm.eval()

    # Check Accuracy
    if backend_accuracy_fails(gm, example_inputs, compiler_fn):
```

backend_accuracy_fails is not passed `only_fwd`.

### Versions

master

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh
minifier_launcher.py silently swallows "ran into runtime exception which is likely an unrelated an issue" warnings
Labels: triaged, oncall: pt2

### 🐛 Describe the bug

Sometimes, minifier launcher complains that "Input graph did not fail the tester". But in fact, there was just a problem in the minifier infrastructure code itself. However, minifier runs are done in a subprocess, and the stdout/stderr is piped to a file, so you can't see a message like this:

```
(/home/ezyang/local/b/pytorch-env) [ezyang@devgpu020.ftw1 ~/local/b/pytorch (4162c7e9)]$ python /data/users/ezyang/b/pytorch/torch_compile_debug/run_2023_01_30_14_12_59_772449/minifier/minifier_launcher.py
While minifying the program in accuracy minification mode,ran into a runtime exception which is likely an unrelated issue. Skipping this graph
Traceback (most recent call last):
  File "/data/users/ezyang/b/pytorch/torch/_dynamo/debug_utils.py", line 881, in backend_accuracy_fails
    compiled_gm = compiler_fn(copy.deepcopy(gm), clone_inputs(example_inputs))
  File "/home/ezyang/local/b/pytorch-env/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "/data/users/ezyang/b/pytorch/torch/fx/graph_module.py", line 708, in __deepcopy__
    fake_mod.__dict__ = copy.deepcopy(self.__dict__, memo)
  File "/home/ezyang/local/b/pytorch-env/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/ezyang/local/b/pytorch-env/lib/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/ezyang/local/b/pytorch-env/lib/python3.9/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/ezyang/local/b/pytorch-env/lib/python3.9/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/ezyang/local/b/pytorch-env/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/ezyang/local/b/pytorch-env/lib/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/ezyang/local/b/pytorch-env/lib/python3.9/copy.py", line 161, in deepcopy
    rv = reductor(4)
TypeError: cannot pickle '_thread._local' object
Traceback (most recent call last):
  File "/data/users/ezyang/b/pytorch/torch_compile_debug/run_2023_01_30_14_12_59_772449/minifier/minifier_launcher.py", line 517, in <module>
    minifier(
  File "/data/users/ezyang/b/pytorch/torch/_functorch/fx_minifier.py", line 97, in minifier
    raise RuntimeError("Input graph did not fail the tester")
RuntimeError: Input graph did not fail the tester
```

This is bad and makes it seem like there was something wrong with the minified example, when there actually was something wrong with the minifier.

### Versions

master

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh
when I want to use a new backend, how to deal with the op with 'device' argument? 
Labels: triaged, module: backend

### 🐛 Describe the bug

Hi
I saw the generated code in python_torch_functionsEverything.cpp line 4763， there are so many tricks for the op with 'device' argument, such as init CUDA device, `torch::utils::maybe_initialize_cuda(options);`
```
static PyObject * THPVariable_arange(PyObject* self_, PyObject* args, PyObject* kwargs)
{
  HANDLE_TH_ERRORS
  static PythonArgParser parser({
    "arange(Scalar end, *, Tensor out=None, ScalarType? dtype=None, Layout? layout=None, Device? device=None, bool? pin_memory=False, bool? requires_grad=False)",
    "arange(Scalar start, Scalar end, *, ScalarType? dtype=None, Layout? layout=None, Device? device=None, bool? pin_memory=False, bool? requires_grad=False)",
    "arange(Scalar start, Scalar end, Scalar step=1, *, Tensor out=None, ScalarType? dtype=None, Layout? layout=None, Device? device=None, bool? pin_memory=False, bool? requires_grad=False)",
  }, /*traceable=*/true);

  ParsedArgs<9> parsed_args;
  auto _r = parser.parse(nullptr, args, kwargs, parsed_args);
  if(_r.has_torch_function()) {
    return handle_torch_function(_r, nullptr, args, kwargs, THPVariableFunctionsModule, "torch");
  }
  switch (_r.idx) {
    case 0: {
      if (_r.isNone(1)) {
        // aten::arange(Scalar end, *, ScalarType? dtype=None, Layout? layout=None, Device? device=None, bool? pin_memory=None) -> Tensor
        const auto options = TensorOptions()
            .dtype(_r.scalartypeOptional(2))
            .device(_r.deviceWithDefault(4, torch::tensors::get_default_device()))
            .layout(_r.layoutOptional(3))
            .requires_grad(_r.toBool(6))
            .pinned_memory(_r.toBool(5));
        torch::utils::maybe_initialize_cuda(options);
```

when I  want to use a new backend which also need to init like CUDA, so I want to add some code to make my backend running fine, It is that ok ?
thanks.


### Versions

new backend
python:3.7.5
pytorch: 2.0.0
CUDA: None
aten::int_repr not supported in torch.onnx.export
Labels: module: onnx, triaged

### 🚀 The feature, motivation and pitch

```python
import torch.onnx
import torch

class Model(torch.nn.Module):
    def forward(self, x):
        y = torch.quantize_per_tensor(x, 1., 0, torch.qint8)
        return y.int_repr()

model = Model()
model.eval()
args = torch.ones(1, 1, 28, 28)
print(model(args))

torch.onnx.export(model,               # model being run
                  args,                         # model input (or a tuple for multiple inputs)
                  "out.onnx",   # where to save the model (can be a file or file-like object)
                  opset_version=16,          # the ONNX version to export the model to
                  input_names = ['input'],   # the model's input names
                  output_names = ['output'])
```
currently says
```
torch.onnx.errors.UnsupportedOperatorError: Exporting the operator 'aten::int_repr' to ONNX opset version 16 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub: https://github.com/pytorch/pytorch/issues.
```

I'd like to see this supported.

quantize_per_tensor+ int_repr + clamp is a useful way to represent quantization to less than 8 bits.

### Alternatives

_No response_

### Additional context

I have never developed on pytorch. But considering that dequantize(quantize_per_tensor(x)) already emits a cast op in ONNX,
I think the effort to do this would be small. I might be able to do it you could provide pointers about where to implement and where to add tests.
Minifier should not use pickle to save state into minifier launcher
Labels: triaged, module: fx, module: functorch

### 🐛 Describe the bug

So hard to read!

### Versions

master

cc @SherlockNoMad @soumith @EikanWang @jgong5 @wenzhe-nrv @zou3519 @Chillee @samdow @kshitij12345 @janeyx99
Minifier doesn't save/load functorch config
Labels: triaged, module: fx, module: functorch

### 🐛 Describe the bug

self explanatory

### Versions

master

cc @SherlockNoMad @soumith @EikanWang @jgong5 @wenzhe-nrv @zou3519 @Chillee @samdow @kshitij12345 @janeyx99