Set correct dtypes for ONNX quantization
Author: severinsimmler
Pull request: yes

# What does this PR do?

It's currently not to possible to quantize an ONNX model with `transformers.convert_graph_to_onnx`.

Running the following snippet on `main`:

```python
from pathlib import Path

from transformers import pipeline
from transformers.convert_graph_to_onnx import convert_pytorch, quantize


# load a ner model
nlp = pipeline(task="ner", model="dbmdz/bert-large-cased-finetuned-conll03-english")

# name of the onnx file to be exported
output = Path("model.onnx")

# first transform pytorch to onnx model
convert_pytorch(nlp, output=output, opset=11, use_external_format=False)

# onnx model can now be quantized
quantize(output)
```

will result in:

```
Traceback (most recent call last):
  File "/home/severin/git/transformers/test-quantization.py", line 12, in <module>
    quantized_model = quantize(output)
                      ^^^^^^^^^^^^^^^^
  File "/home/severin/git/transformers/src/transformers/convert_graph_to_onnx.py", line 472, in quantize
    quantizer.quantize_model()
  File "/home/severin/git/transformers/.venv/lib/python3.11/site-packages/onnxruntime/quantization/onnx_quantizer.py", line 310, in quantize_model
    op_quantizer.quantize()
  File "/home/severin/git/transformers/.venv/lib/python3.11/site-packages/onnxruntime/quantization/operators/gather.py", line 29, in quantize
    ) = self.quantizer.quantize_activation(node, [0])
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/severin/git/transformers/.venv/lib/python3.11/site-packages/onnxruntime/quantization/onnx_quantizer.py", line 825, in quantize_activation
    return self.__quantize_inputs(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/severin/git/transformers/.venv/lib/python3.11/site-packages/onnxruntime/quantization/onnx_quantizer.py", line 915, in __quantize_inputs
    q_weight_name, zp_name, scale_name = self.quantize_initializer(
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/severin/git/transformers/.venv/lib/python3.11/site-packages/onnxruntime/quantization/onnx_quantizer.py", line 995, in quantize_initializer
    _, _, zero_point, scale, q_weight_data = quantize_data(
                                             ^^^^^^^^^^^^^^
  File "/home/severin/git/transformers/.venv/lib/python3.11/site-packages/onnxruntime/quantization/quant_utils.py", line 277, in quantize_data
    raise ValueError(f"Unexpected value for qType={qType}.")
ValueError: Unexpected value for qType=False.
```

The values are updated in this PR to be consistent with the default values in `optimum` (see [this](https://github.com/huggingface/optimum/blob/bb7b71a2c1f9c9220845b258afd88a5c1a24c013/optimum/onnxruntime/quantization.py#L367-L368) and also [this](https://github.com/huggingface/optimum/blob/bb7b71a2c1f9c9220845b258afd88a5c1a24c013/optimum/onnxruntime/configuration.py#L275-L277)).

Running the snippet from above in my branch outputs as expected:

```
Quantized model has been written at model-quantized.onnx: ✔
```

Tested with `onnxruntime` 1.16.3 (on Python 3.11.6) and 1.12.1 (on Python 3.10.13).


## Who can review?

@SunMarc and @younesbelkada (neither `bitsandbytes` nor `autogpt`, but quantization though)


Comments:
Thank you @severinsimmler, `transformers.convert_graph_to_onnx` is deprecated (https://github.com/huggingface/transformers/blob/dafd59512cf6376ee2f056e38935d83df77a213c/src/transformers/convert_graph_to_onnx.py#L362), I would recommend you to use optimum utilities.

@fxmarty Yes, makes sense to migrate to `optimum`. Should I close this PR then? However, would be great if quantization would work even with the deprecated module until version 5 is released.

Let's see what @michaelbenayoun thinks - I would just recommend to move to Optimum.

I agree with @fxmarty . Since ONNX related features are supported in Optimum, it does not make sense to add new features in Transformers at this point.

But this is not really a new feature, but rather a bugfix? It's definitely broken at the moment.

Let's move to Optimum anyways.

I understand that the module is deprecated and agree that everyone should move to Optimum, but `transformers.convert_graph_to_onnx.quantize` is broken and the fix is literally five changed lines. In case you are not going to release Transformers 5 next week and remove the whole module, it'd be worth merging this PR imho. If you don't think so, feel free to close this PR.

Okay, got it :D What a pity