Converting PyTorch to ONNX model doubles file size for Deberta v3. Not case of renaming.
Author: jordiclive
Pull request: no
Labels: name: bug

### System Info

```shell
- OS Platform and Distribution: Amazon Linux 2
- ONNX version 1.11.0 
- Python version: 3.8
- Transformers 4.17
```


### Who can help?

@LysandreJik, @patil-suraj

### Describe the bug
When I try and convert `microsoft/deberta-v3-large` to onnx format, the file size doubles to 1.8Gb while the pytorch file size is 800Mb. 
There was an issue [here](https://github.com/onnx/onnx/issues/3278#issuecomment-781948998) that was resolved by removing the duplication of weights. However, trying this code snippet in the deberta case only reduced the file size from 1.8Gb to 1.6Gb.


### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction


```
from collections import OrderedDict
from typing import Mapping
from pathlib import Path
from transformers.onnx import export
from transformers.onnx import OnnxConfig
from transformers import AutoTokenizer, AutoModel, AutoConfig

onnx_path = Path("deberta.onnx")

class DebertaConfig(OnnxConfig):
    @property
    def inputs(self) -> Mapping[str, Mapping[int, str]]:
        return OrderedDict(
            [
                ("input_ids", {0: "batch", 1: "sequence"}),
                ("attention_mask", {0: "batch", 1: "sequence"}),
                ("token_type_ids", {0: "batch", 1: "sequence"}),
            ]
        )

config = AutoConfig.from_pretrained("microsoft/deberta-v3-large")

base_model = AutoModel.from_pretrained("microsoft/deberta-v3-large")
tokenizer = AutoTokenizer.from_pretrained("microsoft/deberta-v3-large")
onnx_config = DebertaConfig(config)
onnx_inputs, onnx_outputs = export(tokenizer, base_model, onnx_config, 15, onnx_path)
```

### Expected behavior

```shell
Expect the model.onnx file to be ~800Mb like the pytorch file.
```


Comments:
Maybe @lewtun and @michaelbenayoun have insights

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Reopen

Just to mention that @michaelbenayoun has built a tool in [Optimum](https://github.com/huggingface/optimum/blob/main/optimum/onnx/graph_transformations.py) to remove duplicate weights. In this case, it reduces the .onnx size from 1.9GB to 1.7GB. 
```python
from optimum.onnx.graph_transformations import remove_duplicate_weights

new_model = remove_duplicate_weights(model)
```

And `optimum.onnx` might be the place where further enhancement shall take place. 

Hi @jordiclive,
It might be due to some tensor creation ops (either numpy or pytorch) that end up being stored in the ONNX graph. 
The state dict does not have to contain those so that would explain the difference, but that's just a guess I this point.

I'm not sure if this is a Transformers issue anymore. The same happens with Roberta.

I think the code-snippet I posted does the optimum tool reduction.

Just in case anyone comes across this, a large part of the size increase may have nothing to do with ONNX.  The deberta-v3-* appears to be trained with mixed precision, but from_pretrained automatically loads with float32 tensors.  You may be able to shave 800mb off by loading the model as:

```
model = AutoModel.from_pretrained('microsoft/deberta-v3-large', torch_dtype=torch.float16)
```