Unable to quantize Google's LaBSE model using convert_graph_to_onnx.py
Author: swapnil3597
Pull request: no

## Environment Info:
Experiment was performed on Google Colab, RAM: 12.69GB
Also experiment on machine with ~20GB RAM available.

### Who can help
@LysandreJik @sgugger @SilvanK4t1qbit

## Information
I'm unable to quantize Google's [`setu4993/LaBSE`](https://huggingface.co/setu4993/LaBSE) model using script [`convert_graph_to_onnx.py`](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_graph_to_onnx.py) (Approximate model size LaBSE is **~1.8GB**).

The command I used to convert the graph is:
```bash
python convert_graph_to_onnx.py --framework pt --model setu4993/LaBSE --quantize saved_models_temp/labse_bert_onnx/labse_bert.onnx --pipeline sentiment-analysis --opset 11
```

The ONNX `convert` and `optimize` steps are executed, and after that process is killed while running `quantize`.
The process is **Killed** without any error.

**Terminal Output:**
```bash
2021-07-08 10:11:48.524534: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0

====== Converting model to ONNX ======
ONNX opset version set to: 11
Loading pipeline (model: setu4993/LaBSE, tokenizer: setu4993/LaBSE)
Downloading: 100% 560/560 [00:00<00:00, 763kB/s]
Downloading: 100% 1.88G/1.88G [00:39<00:00, 47.4MB/s]
tcmalloc: large alloc 1539547136 bytes == 0x5654616fc000 @  0x7fb8b248ab6b 0x7fb8b24aa379 0x7fb8469f526e 0x7fb8469f69e2 0x7fb88a260b49 0x7fb88a261897 0x7fb88a63dd89 0x7fb88ada2b9a 0x7fb88ad85cbe 0x7fb88a98aa05 0x7fb89d2cc451 0x565457706338 0x56545783a1ba 0x5654578337ad 0x5654577c6c9f 0x565457807d79 0x565457804cc4 0x5654577c5559 0x5654578394f8 0x5654578337ad 0x5654577c6a81 0x565457807d79 0x565457804cc4 0x5654577c5462 0x565457838715 0x5654578337ad 0x5654577c6a81 0x565457807d79 0x565457804cc4 0x5654577c5462 0x565457838715
tcmalloc: large alloc 1539547136 bytes == 0x5654d1b10000 @  0x7fb8b248ab6b 0x7fb8b24aa379 0x7fb8469f526e 0x7fb8469f69e2 0x7fb88b1129e9 0x7fb89d47d349 0x565457804c65 0x5654577c5462 0x565457838715 0x5654578337ad 0x5654577c6003 0x5654577c5b09 0x56545790d28d 0x56545787c1db 0x5654577c4bb1 0x5654578b5fed 0x565457838988 0x5654578337ad 0x565457705e2c 0x565457835bb5 0x5654578334ae 0x5654577c63ea 0x56545783532a 0x5654578334ae 0x5654577c6c9f 0x5654577c6ea1 0x565457835bb5 0x5654578334ae 0x5654577c6c9f 0x5654577c6ea1 0x565457835bb5
Some weights of BertForSequenceClassification were not initialized from the model checkpoint at setu4993/LaBSE and are newly initialized: ['classifier.weight', 'classifier.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Downloading: 100% 239/239 [00:00<00:00, 374kB/s]
Downloading: 100% 5.22M/5.22M [00:00<00:00, 53.7MB/s]
Downloading: 100% 9.62M/9.62M [00:00<00:00, 45.6MB/s]
Downloading: 100% 112/112 [00:00<00:00, 169kB/s]
Creating folder path_to_model/saved_models_temp/labse_bert_onnx
Using framework PyTorch: 1.9.0+cu102
Found input input_ids with shape: {0: 'batch', 1: 'sequence'}
Found input token_type_ids with shape: {0: 'batch', 1: 'sequence'}
Found input attention_mask with shape: {0: 'batch', 1: 'sequence'}
Found output output_0 with shape: {0: 'batch'}
Ensuring inputs are in correct order
position_ids is not present in the generated input list.
Generated inputs order: ['input_ids', 'attention_mask', 'token_type_ids']
tcmalloc: large alloc 1539547136 bytes == 0x5654d1b10000 @  0x7fb8b248ab6b 0x7fb8b24aa379 0x7fb8469f526e 0x7fb8469f69e2 0x7fb88b111a73 0x7fb88a647b7b 0x7fb88ada2bef 0x7fb88ad87480 0x7fb88a990454 0x7fb88a648890 0x7fb88ae9c26f 0x7fb88ac5af3e 0x7fb88c530f77 0x7fb88c5313f2 0x7fb88ac5af3e 0x7fb88c7e7fde 0x7fb88c7e8102 0x7fb88b0dd8a6 0x7fb89d163742 0x5654577c4d54 0x5654577c4a50 0x565457839105 0x5654578b6e36 0x5654578abe76 0x565457800484 0x565457804c65 0x5654577c5462 0x565457838715 0x5654578337ad 0x565457705eb1 0x7fb89d6c1275
/usr/local/lib/python3.7/dist-packages/transformers/modeling_utils.py:1974: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  input_tensor.shape[chunk_dim] == tensor_shape for input_tensor in input_tensors
tcmalloc: large alloc 1539547136 bytes == 0x5654d1b10000 @  0x7fb8b24aa887 0x7fb8b0da0c29 0x7fb8b0da0d47 0x7fb8b0da27a5 0x7fb88ce699c6 0x7fb88ce6bbf6 0x7fb88ce6d20a 0x7fb88ce6de23 0x7fb89d6564f9 0x7fb89d659004 0x7fb89d5cfc00 0x7fb89d054b88 0x5654577c4cc0 0x5654577c4a50 0x565457838be0 0x5654578334ae 0x5654577c63ea 0x56545783532a 0x5654578334ae 0x5654577c63ea 0x56545783532a 0x5654578334ae 0x5654577c63ea 0x56545783532a 0x5654578334ae 0x5654577c63ea 0x5654578387f0 0x5654578334ae 0x5654577c63ea 0x56545783532a 0x5654577c630a
tcmalloc: large alloc 1883791360 bytes == 0x56554dbbe000 @  0x7fb8b24aa887 0x7fb8b0da0c29 0x7fb8b0da1afb 0x7fb8b0da1bb4 0x7fb8b0da1f9c 0x7fb8987e7bb7 0x7fb8987e8064 0x7fb88ce66a1c 0x7fb89d699aff 0x7fb89d054b88 0x5654577c58a8 0x565457838fd5 0x5654578334ae 0x5654577c63ea 0x56545783532a 0x5654578334ae 0x5654577c63ea 0x5654578387f0 0x5654578334ae 0x5654577c63ea 0x56545783532a 0x5654577c630a 0x5654578343b5 0x5654578334ae 0x5654577c63ea 0x5654578343b5 0x5654578334ae 0x5654578331b3 0x5654578fd182 0x5654578fd4fd 0x5654578fd3a6

====== Optimizing ONNX model ======
tcmalloc: large alloc 2147483648 bytes == 0x5655a8fba000 @  0x7fb8b248ab6b 0x7fb8b24aa379 0x7fb7eb84b34c 0x7fb7eb8482f4 0x7fb7eb8027d1 0x7fb7eb8077b2 0x7fb7eb80a4d6 0x7fb7eb6783a0 0x7fb7ebb3e747 0x7fb7ebb84b53 0x7fb7ebbacb38 0x5654577c58a8 0x565457838fd5 0x5654578334ae 0x5654577c63ea 0x56545783460e 0x5654578334ae 0x5654577c6a81 0x565457807d79 0x565457804cc4 0x5654577c5462 0x565457838715 0x5654577c630a 0x5654578343b5 0x5654578334ae 0x5654578331b3 0x5654578fd182 0x5654578fd4fd 0x5654578fd3a6 0x5654578d4723 0x5654578d43cc
2021-07-08 10:14:05.009614471 [W:onnxruntime:, inference_session.cc:1303 Initialize] Serializing optimized model with Graph Optimization level greater than ORT_ENABLE_EXTENDED and the NchwcTransformer enabled. The generated model may contain hardware specific optimizations, and should only be used in the same environment the model was optimized in.
Optimized model has been written at path_to_model/saved_models_temp/labse_bert_onnx/labse_bert-optimized.onnx: ✔
/!\ Optimized model contains hardware specific operators which might not be portable. /!\
As of onnxruntime 1.4.0, models larger than 2GB will fail to quantize due to protobuf constraint.
This limitation will be removed in the next release of onnxruntime.
WARNING:root:onnxruntime.quantization.quantize is deprecated.
         Please use quantize_static for static quantization, quantize_dynamic for dynamic quantization.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator FusedMatMul. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Gelu. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator FusedMatMul. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Gelu. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator FusedMatMul. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Gelu. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator FusedMatMul. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Gelu. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator FusedMatMul. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Gelu. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator FusedMatMul. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Gelu. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator FusedMatMul. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Gelu. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator FusedMatMul. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Gelu. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator FusedMatMul. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Gelu. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator FusedMatMul. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Gelu. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator FusedMatMul. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Gelu. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator FusedMatMul. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator Gelu. No schema registered for this operator.
Warning: Unsupported operator LayerNormalization. No schema registered for this operator.
Warning: Unsupported operator FusedGemm. No schema registered for this operator.
tcmalloc: large alloc 3079086080 bytes == 0x5656f6cf0000 @  0x7fb8b24aa001 0x5654577f7b30 0x5654577ce655 0x7fb8ae61c5a9 0x5654577c4c47 0x5654578b5fed 0x565457838988 0x5654578334ae 0x5654577c63ea 0x56545783460e 0x5654578334ae 0x5654577c63ea 0x56545783460e 0x5654578337ad 0x5654577c63ea 0x56545783460e 0x5654577c630a 0x56545783460e 0x5654578334ae 0x5654577c63ea 0x56545783532a 0x5654577c630a 0x5654578343b5 0x5654578334ae 0x5654578331b3 0x5654578fd182 0x5654578fd4fd 0x5654578fd3a6 0x5654578d4723 0x5654578d43cc 0x7fb8b1292bf7
^C
```
The process is killed after this without any external interruption.
Could this be a memory issue. I also tried this same experiment on machine with over 20GB RAM available, but the results were similar.

## To reproduce
**Python package Requirements:**
```text
torch
transformers
onnx
onnxruntime
onnxruntime-tools
```

**Run command:**
```
python convert_graph_to_onnx.py --framework pt --model setu4993/LaBSE --quantize path_to_model/labse_bert_onnx/labse_bert.onnx --pipeline sentiment-analysis --opset 11
```
File: [`convert_graph_to_onnx.py`](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_graph_to_onnx.py)





Comments:
Indeed, this looks like a memory error!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.