added support for exporting of T5 models to onnx with past_key_values.
Author: Ki6an
Pull request: yes

# What does this PR do?

>by applying this fix I was able to create **[fastT5](https://pypi.org/project/fastt5/)**  library. which increases the T5 model inference speed up to 5x.  for more details check out my [GitHub](https://github.com/Ki6an/fastT5) repo.

addressing [this ](https://github.com/huggingface/transformers/issues/10645)issue and [this ](https://github.com/huggingface/transformers/pull/9733)PR

while exporting T5 decoder model to onnx with `past_key_values` 
was getting this error.
```python
/usr/local/lib/python3.7/dist-packages/transformers/models/t5/modeling_t5.py in forward(self, hidden_states, mask, key_value_states, position_bias, past_key_value, layer_head_mask, query_length, use_cache, output_attentions)
    497                 position_bias = position_bias + mask  # (batch_size, n_heads, seq_length, key_length)
    498 
--> 499         scores += position_bias
    500         attn_weights = F.softmax(scores.float(), dim=-1).type_as(
    501             scores

RuntimeError: output with shape [5, 8, 1, 2] doesn't match the broadcast shape [5, 8, 2, 2]
```
the reason is while `torch-jit-tracing`    the `seq_lenth` is converted to type `<class 'torch.Tensor'>` in this line [424](https://github.com/huggingface/transformers/blob/26a33cfd8c2d6923f41ab98683f33172e8948ff3/src/transformers/models/t5/modeling_t5.py#L424) `  batch_size, seq_length = hidden_states.shape[:2]` 

next,  tracing throws the following warning at line [494](https://github.com/huggingface/transformers/blob/26a33cfd8c2d6923f41ab98683f33172e8948ff3/src/transformers/models/t5/modeling_t5.py#L494) 

```python
/usr/local/lib/python3.7/dist-packages/transformers/models/t5/modeling_t5.py:494: 
TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!

  position_bias = position_bias[:, :, -seq_length:, :]
```
so it keeps `position_bias` as constant and we get the error at line [499](https://github.com/huggingface/transformers/blob/26a33cfd8c2d6923f41ab98683f33172e8948ff3/src/transformers/models/t5/modeling_t5.py#L499). because of the shape mismatch of `positon_bais` and `scores`.



to solve this issue, we can create a variable `int_seq_length` that will stay as `<class 'int'>`  throughout the whole process. & we will use this variable in line   `position_bias = position_bias[:, :, -int_seq_length:, :]`.

now, tracing no longer throws the warning of `position_bias` being constant and we won't get the error: shape mismatch of `positon_bais` and `scores`. 
by following this simple fix I was able to export t5 to onnx as shown in this  [notebook ](https://colab.research.google.com/drive/1Q5GSqOOrhO-7NQLpZPZ1C7YrkAot3TJg?usp=sharing). & also was able to create ['fastT5'](https://github.com/Ki6an/fastT5) repo :)

t5: @patrickvonplaten, @patil-suraj



Comments:
@patrickvonplaten, @patil-suraj any updates on this PR or [10645](https://github.com/huggingface/transformers/issues/10645)  issue?

@mfuntowicz or @Narsil do you have 2min to give your feedback on this maybe? :-)

+1 for this

+1, this is needed for fastT5

Sorry, I failed to see the first mention.

Yes this is needed for T5. It's a relatively small change, so probably worth it.


@Ki6an thanks for the notebook !

Just a note for everyone reading this, dynamic sizes that are **too** general might affect performance (for instance at runtime, batch_size=1 can be enforced for `encoder_input_ids`. This can lead to some performance gains using `onnxruntime`.
Enforcing batch_size = num_beams *can* also lead to improvements. 

hey, @patrickvonplaten, @patil-suraj, @mfuntowicz  could you please have another look at this PR.

If this enables ONNX, I'm totally fine with the PR, but I'm no expert in ONNX at all...

I leave it no @Narsil to merge the PR if it looks good to him