T5PreTrainedModel fails to export to ONNX
Author: borisfom
Pull request: no

### System Info

Version: 4.32.1
Python: 3.10


### Who can help?

@sanchit-gandhi 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

1. Run the repro script : 
```
import torch

from transformers import AutoTokenizer, T5ForConditionalGeneration

def input_example(self, max_batch=1, max_dim=64, seq_len=16):
    sample = next(self.parameters())
    input_ids = torch.randint(low=0, high=max_dim, size=(max_batch, seq_len), device=sample.device)
    labels = torch.randint(low=0, high=max_dim, size=(max_batch, seq_len), device=sample.device)
    attention_mask = torch.randint(low=0, high=1, size=(max_batch, seq_len), device=sample.device)
    return tuple([input_ids, attention_mask, labels])

model = T5ForConditionalGeneration.from_pretrained("google/byt5-small")

torch.onnx.export(model,
                  input_example(model),
                  "t5.onnx",
                  verbose=True,
                  opset_version=16,
                  do_constant_folding=True,
                  )
```

python t5_repro.py

2. Observe result: 
....
/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py:1655: UserWarning: The exported ONNX model failed ONNX shape inference.The model will not be executable by the ONNX Runt\
ime.If this is unintended and you believe there is a bug,please report an issue at https://github.com/pytorch/pytorch/issues.Error reported by strict ONNX shape inference: [ShapeIn\
ferenceError] (op_type:Where, node name: /encoder/block.0/layer.0/SelfAttention/Where): Y has inconsistent type tensor(float) (Triggered internally at /opt/pytorch/pytorch/torch/cs\
rc/jit/serialization/export.cpp:1410.)

Resulting ONNX does fail in ORT if attempted.


### Expected behavior

Working ONNX file should be generated.


Comments:
This patch fixes this case. I do not submit it as a PR though, as it's a WAR only and float type pollution should be fixed at the source.

[tr.patch](https://github.com/huggingface/transformers/files/12480124/tr.patch)


Hi @borisfom 

The support of ONNX is now moved to [Optimum](https://github.com/huggingface/optimum). There is a doc [here](https://huggingface.co/blog/convert-transformers-to-onnx).

On `transformers` side, we would be open to a PR from the community contributor if a root cause has to be applied directly to a modeling code in `transformers`. 🤗 Thank you for your comprehension.

@ydshieh would `transformers` side be open to a PR with this patch to directly address the issue faced?

Hey @mgrafu - is the patch on the `transformers` side? The `transformers` codebase is assumed to run independently of ONNX. So if this is an ONNX related issue, [Optimum](https://github.com/huggingface/optimum) is the best place for a patch. Thanks!

Yes the patch is on transformers side. The problem is that some arguments to these functions are coming in as float that should have been long integers. Torch is doing conversion on the fly, while ONNX exporter exports as they are, and that code fails later in ORT because ONNX does not have implicit conversion mechanism. The issue should be fixed at transformers code level - and probably a proper fix should fix those floats at the source. I am just not as familiar with the transformers source to do that kind of fix.


> Torch is doing conversion on the fly, while ONNX exporter exports as they are, and that code fails later in ORT because ONNX does not have implicit conversion mechanism

This hints to me that the `transformers` code is fine (since it's working with `torch`, and that's what we support) but the Optimum code needs to be updated to do the implicit conversion (since ONNX doesn't have this mechanism). But I see that there are a lot of `long` conversions already in the current code, so it might be that additional changes are required to make the implementation more robust. If you have a PR that showcases the changes, it might be easier to comment!

We can definitely take a look if you open a PR :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.