Repository: pytorch
Issue: [ONNX] `SymbolicValueError` with `onnx::quantized_gather`
Labels: module: onnx,triaged

### 🐛 Describe the bug

When attempting to export a quantized PyTorch GNN model to ONNX, I got the below error:
```
Traceback (most recent call last):
  File "/home/nathaniel/dev/hood/./ONNXExport.py", line 166, in <module>
    export_onnx_model(
  File "/home/nathaniel/dev/hood/./ONNXExport.py", line 78, in export_onnx_model
    torch.onnx.export(
  File "/home/nathaniel/.local/lib/python3.10/site-packages/torch/onnx/utils.py", line 516, in export
    _export(
  File "/home/nathaniel/.local/lib/python3.10/site-packages/torch/onnx/utils.py", line 1613, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/home/nathaniel/.local/lib/python3.10/site-packages/torch/onnx/utils.py", line 1139, in _model_to_graph
    graph = _optimize_graph(
  File "/home/nathaniel/.local/lib/python3.10/site-packages/torch/onnx/utils.py", line 677, in _optimize_graph
    graph = _C._jit_pass_onnx(graph, operator_export_type)
  File "/home/nathaniel/.local/lib/python3.10/site-packages/torch/onnx/utils.py", line 1957, in _run_symbolic_function
    return symbolic_fn(graph_context, *inputs, **attrs)
  File "/home/nathaniel/.local/lib/python3.10/site-packages/torch/onnx/symbolic_opset10.py", line 774, in dequantize
    return symbolic_helper.dequantize_helper(g, input)[0]
  File "/home/nathaniel/.local/lib/python3.10/site-packages/torch/onnx/symbolic_helper.py", line 1591, in dequantize_helper
    unpacked_qtensors = _unpack_quantized_tensor(qtensor)
  File "/home/nathaniel/.local/lib/python3.10/site-packages/torch/onnx/symbolic_helper.py", line 216, in _unpack_quantized_tensor
    raise errors.SymbolicValueError(
torch.onnx.errors.SymbolicValueError: ONNX symbolic expected the output of `%x.19 : Tensor = onnx::Gather[axis=0](%cloth_features, %1370), scope: models.core.postcvpr.EncodeProcessDecode::/models.core.basequant.GraphNetBlockQuant::0 # /home/nathaniel/dev/hood/models/core/basequant.py:114:0
` to be a quantized tensor. Is this likely due to missing support for quantized `onnx::Gather`. Please create an issue on https://github.com/pytorch/pytorch/issues  [Caused by the value 'x.19 defined in (%x.19 : Tensor = onnx::Gather[axis=0](%cloth_features, %1370), scope: models.core.postcvpr.EncodeProcessDecode::/models.core.basequant.GraphNetBlockQuant::0 # /home/nathaniel/dev/hood/models/core/basequant.py:114:0
)' (type 'Tensor') in the TorchScript graph. The containing node has kind 'onnx::Gather'.] 
    (node defined in /home/nathaniel/dev/hood/models/core/basequant.py(114): update_edge_features
/home/nathaniel/dev/hood/models/core/basequant.py(169): get_updated_edge_features
/home/nathaniel/dev/hood/models/core/basequant.py(311): propagate
/home/nathaniel/dev/hood/models/core/basequant.py(45): forward
/home/nathaniel/.local/lib/python3.10/site-packages/torch/nn/modules/module.py(1501): _slow_forward
/home/nathaniel/.local/lib/python3.10/site-packages/torch/nn/modules/module.py(1520): _call_impl
/home/nathaniel/.local/lib/python3.10/site-packages/torch/nn/modules/module.py(1511): _wrapped_call_impl
/home/nathaniel/dev/hood/models/core/postcvpr.py(499): _process
/home/nathaniel/dev/hood/models/core/postcvpr.py(647): forward
/home/nathaniel/.local/lib/python3.10/site-packages/torch/nn/modules/module.py(1501): _slow_forward
/home/nathaniel/.local/lib/python3.10/site-packages/torch/nn/modules/module.py(1520): _call_impl
/home/nathaniel/.local/lib/python3.10/site-packages/torch/nn/modules/module.py(1511): _wrapped_call_impl
/home/nathaniel/.local/lib/python3.10/site-packages/torch/jit/_trace.py(1074): trace_module
/home/nathaniel/.local/lib/python3.10/site-packages/torch/jit/_trace.py(806): trace
/home/nathaniel/dev/hood/./ONNXExport.py(156): <module>
)

    Inputs:
        #0: cloth_features defined in (%cloth_features : Tensor = prim::TupleConstruct(%804, %802, %803), scope: models.core.postcvpr.EncodeProcessDecode::/torch.nn.modules.container.Sequential::node_encoder/models.core.qops.LayerNormQuantized::1 # /home/nathaniel/.local/lib/python3.10/site-packages/torch/_ops.py:755:0
    )  (type 'Tensor')
        #1: 1370 defined in (%1370 : Long(*, device=cpu) = onnx::Gather[axis=0](%direct_world_edge_index, %679), scope: models.core.postcvpr.EncodeProcessDecode::/models.core.basequant.GraphNetBlockQuant::0 # /home/nathaniel/dev/hood/models/core/basequant.py:114:0
    )  (type 'Tensor')
    Outputs:
        #0: x.19 defined in (%x.19 : Tensor = onnx::Gather[axis=0](%cloth_features, %1370), scope: models.core.postcvpr.EncodeProcessDecode::/models.core.basequant.GraphNetBlockQuant::0 # /home/nathaniel/dev/hood/models/core/basequant.py:114:0
    )  (type 'Tensor')
```
As the error message states, I think this is caused by "missing support for quantized `onnx::Gather`" - could support for exporting models with this operator be added?

### Versions

```
nathaniel@R9128537:~/dev$ python collect_env.py
Collecting environment information...
PyTorch version: 2.2.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.3 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] (64-bit runtime)
Python platform: Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: Quadro T2000 with Max-Q Design
Nvidia driver version: 538.18
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Address sizes:                   39 bits physical, 48 bits virtual
Byte Order:                      Little Endian
CPU(s):                          12
On-line CPU(s) list:             0-11
Vendor ID:                       GenuineIntel
Model name:                      Intel(R) Xeon(R) W-10855M CPU @ 2.80GHz
CPU family:                      6
Model:                           165
Thread(s) per core:              2
Core(s) per socket:              6
Socket(s):                       1
Stepping:                        2
BogoMIPS:                        5616.01
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi ept vpid ept_ad fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 xsaves md_clear flush_l1d arch_capabilities
Virtualization:                  VT-x
Hypervisor vendor:               Microsoft
Virtualization type:             full
L1d cache:                       192 KiB (6 instances)
L1i cache:                       192 KiB (6 instances)
L2 cache:                        1.5 MiB (6 instances)
L3 cache:                        12 MiB (1 instance)
Vulnerability Itlb multihit:     KVM: Mitigation: VMX disabled
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Enhanced IBRS, IBPB conditional, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected

Versions of relevant libraries:
[pip3] numpy==1.26.3
[pip3] onnx==1.16.0
[pip3] onnxruntime-gpu==1.17.1
[pip3] torch==2.2.0
[pip3] torch_cluster==1.6.3+pt22cu121
[pip3] torch_geometric==2.5.3
[pip3] torch_scatter==2.1.2+pt22cu121
[pip3] torch_sparse==0.6.18+pt22cu121
[pip3] torch_spline_conv==1.2.2+pt22cu121
[pip3] torchaudio==2.1.2+cu121
[pip3] torchvision==0.16.2+cu121
[pip3] triton==2.2.0
[conda] Could not collect
```