Repository: pytorch
Issue: [ONNX] Failed to export PyTorch-2-Export-Quantized model to onnx
Labels: module: onnx,triaged

### 🐛 Describe the bug

try to quantize a model like [this link](https://pytorch.org/tutorials/prototype/pt2e_quant_qat.html)
(only different in model structures and datasets)
then export the quantized model to onnx by `torch.onnx.export` (original model is able to output), and get
```Traceback (most recent call last):
  File "d:\my_project\train_quantized.py", line 798, in <module>
    onnx_program = torch.onnx.export(model, torch_input, "my_quantized.onnx")
  File "D:\anaconda3\envs\hm\lib\site-packages\torch\onnx\__init__.py", line 375, in export
    export(
  File "D:\anaconda3\envs\hm\lib\site-packages\torch\onnx\utils.py", line 502, in export
    _export(
  File "D:\anaconda3\envs\hm\lib\site-packages\torch\onnx\utils.py", line 1564, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "D:\anaconda3\envs\hm\lib\site-packages\torch\onnx\utils.py", line 1117, in _model_to_graph
    graph = _optimize_graph(
  File "D:\anaconda3\envs\hm\lib\site-packages\torch\onnx\utils.py", line 639, in _optimize_graph
    graph = _C._jit_pass_onnx(graph, operator_export_type)
  File "D:\anaconda3\envs\hm\lib\site-packages\torch\onnx\utils.py", line 1848, in _run_symbolic_function
    raise errors.UnsupportedOperatorError(
torch.onnx.errors.UnsupportedOperatorError: ONNX export failed on an operator with unrecognized namespace quantized_decomposed::quantize_per_tensor. If you are trying to export a custom operator, make sure you registered it with the right domain and version.
```

### Versions

Collecting environment information...
PyTorch version: 2.5.1
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 专业版 (10.0.22631 64 位)
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:40:08) [MSC v.1938 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.22631-SP0
Is CUDA available: True
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4090 D
Nvidia driver version: 560.94
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\cudnn_ops_train64_8.dll
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Name: Intel(R) Core(TM) i9-14900KF
Manufacturer: GenuineIntel
Family: 207
Architecture: 9
ProcessorType: 3
DeviceID: CPU0
CurrentClockSpeed: 3200
MaxClockSpeed: 3200
L2CacheSize: 32768
L2CacheSpeed: None
Revision: None

Versions of relevant libraries:
[pip3] efficientnet_pytorch==0.7.1
[pip3] flake8==7.1.0
[pip3] mypy-extensions==1.0.0
[pip3] numpy==2.1.3
[pip3] nvidia-cuda-runtime-cu12==12.6.77
[pip3] onnx==1.16.0
[pip3] onnx-tool==0.9.0
[pip3] onnxruntime==1.17.1
[pip3] onnxscript==0.1.0.dev20241218
[pip3] onnxsim==0.4.36
[pip3] optree==0.12.1
[pip3] pytorch-lightning==2.4.0
[pip3] segmentation-models-pytorch==0.3.4
[pip3] torch==2.5.1
[pip3] torch-pruning==1.5.1
[pip3] torch-tb-profiler==0.4.3
[pip3] torch_tensorrt==2.5.0
[pip3] torchaudio==2.5.1
[pip3] torchmetrics==1.4.2
[pip3] torchvision==0.20.1
[conda] blas                      1.0                         mkl    defaults
[conda] cuda-cudart               11.8.89                       0    nvidia
[conda] cuda-cudart-dev           11.8.89                       0    nvidia
[conda] cuda-cupti                11.8.87                       0    nvidia
[conda] cuda-libraries            11.8.0                        0    nvidia
[conda] cuda-libraries-dev        11.8.0                        0    nvidia
[conda] cuda-nvrtc                11.8.89                       0    nvidia
[conda] cuda-nvrtc-dev            11.8.89                       0    nvidia
[conda] cuda-nvtx                 11.8.86                       0    nvidia
[conda] cuda-opencl               12.5.39              he0c23c2_1    conda-forge
[conda] cuda-opencl-dev           12.5.39              he0c23c2_1    conda-forge
[conda] cuda-runtime              11.8.0                        0    nvidia
[conda] efficientnet-pytorch      0.7.1                    pypi_0    pypi
[conda] libcublas                 11.11.3.6                     0    nvidia
[conda] libcublas-dev             11.11.3.6                     0    nvidia
[conda] libcufft                  10.9.0.58                     0    nvidia
[conda] libcufft-dev              10.9.0.58                     0    nvidia
[conda] libcurand                 10.3.6.82            he0c23c2_0    conda-forge
[conda] libcurand-dev             10.3.6.82            he0c23c2_0    conda-forge
[conda] libcusolver               11.4.1.48                     0    nvidia
[conda] libcusolver-dev           11.4.1.48                     0    nvidia
[conda] libcusparse               11.7.5.86                     0    nvidia
[conda] libcusparse-dev           11.7.5.86                     0    nvidia
[conda] libnvjitlink              12.4.127                      0    nvidia
[conda] libnvjitlink-dev          12.4.127                      0    nvidia
[conda] mkl                       2021.4.0                 pypi_0    pypi
[conda] mkl-service               2.4.0           py310h2bbff1b_1    defaults
[conda] mkl_fft                   1.3.11          py310h827c3e9_0    defaults
[conda] mkl_random                1.2.8           py310hc64d2fc_0    defaults
[conda] numpy                     2.1.3                    pypi_0    pypi
[conda] nvidia-cuda-runtime-cu12  12.6.77                  pypi_0    pypi
[conda] optree                    0.12.1                   pypi_0    pypi
[conda] pytorch                   2.5.1           py3.10_cuda11.8_cudnn9_0    pytorch
[conda] pytorch-cuda              11.8                 h24eeafa_6    pytorch
[conda] pytorch-lightning         2.4.0                    pypi_0    pypi
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] segmentation-models-pytorch 0.3.4                    pypi_0    pypi
[conda] torch-pruning             1.5.1                    pypi_0    pypi
[conda] torch-tb-profiler         0.4.3                    pypi_0    pypi
[conda] torch-tensorrt            2.5.0                    pypi_0    pypi
[conda] torchaudio                2.5.1                    pypi_0    pypi
[conda] torchmetrics              1.4.2                    pypi_0    pypi
[conda] torchvision               0.20.1                   pypi_0    pypi