replace torch.triu with onnx compatible code
Author: HenryDashwood
Pull request: yes

<!-- This line specifies which issue to close after the pull request is merged. -->
Fixes #5075 

There was a [draft pull request](https://github.com/huggingface/transformers/pull/6334) to this effect a few months ago but the author withdrew it. I'm not sure why. It resolves the _torch.triu_ issue with ONNX. It gives the same output in my tests and runs at the same speed. Entirely possible that I've missed something though!


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6929?src=pr&el=h1) Report
> Merging [#6929](https://codecov.io/gh/huggingface/transformers/pull/6929?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/207ed8cb78ceb4980e40c89f867b06202e660395?el=desc) will **decrease** coverage by `0.58%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6929/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6929?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6929      +/-   ##
==========================================
- Coverage   80.60%   80.02%   -0.59%     
==========================================
  Files         161      161              
  Lines       30119    30122       +3     
==========================================
- Hits        24278    24105     -173     
- Misses       5841     6017     +176     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6929?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/6929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iYXJ0LnB5) | `93.93% <100.00%> (+0.03%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_electra.py](https://codecov.io/gh/huggingface/transformers/pull/6929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9lbGVjdHJhLnB5) | `25.13% <0.00%> (-73.83%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_mobilebert.py](https://codecov.io/gh/huggingface/transformers/pull/6929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9tb2JpbGViZXJ0LnB5) | `24.55% <0.00%> (-72.36%)` | :arrow_down: |
| [src/transformers/modeling\_t5.py](https://codecov.io/gh/huggingface/transformers/pull/6929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190NS5weQ==) | `76.70% <0.00%> (-6.07%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `87.50% <0.00%> (-0.56%)` | :arrow_down: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `82.41% <0.00%> (-0.26%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `86.97% <0.00%> (+1.95%)` | :arrow_up: |
| [src/transformers/data/data\_collator.py](https://codecov.io/gh/huggingface/transformers/pull/6929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL2RhdGFfY29sbGF0b3IucHk=) | `91.90% <0.00%> (+5.26%)` | :arrow_up: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.71% <0.00%> (+5.26%)` | :arrow_up: |
| [src/transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/6929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19yb2JlcnRhLnB5) | `97.08% <0.00%> (+19.34%)` | :arrow_up: |
| ... and [2 more](https://codecov.io/gh/huggingface/transformers/pull/6929/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6929?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6929?src=pr&el=footer). Last update [207ed8c...82d9234](https://codecov.io/gh/huggingface/transformers/pull/6929?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


This is great, want to check in your tests?

I'm not quite sure what the most appropriate way of including them would be to be honest!

Currently I have a little script that looks like this 
```python
import torch
from transformers.modeling_bart import fill_with_neg_inf


def test_upper_right_triangle(torch_device):
    tgt_len = 512
    causal_mask_dtype = torch.float32

    causal_mask_short = torch.triu(
        fill_with_neg_inf(torch.zeros(tgt_len, tgt_len)),
        1).to(dtype=torch.float32, device=torch_device)

    tmp = fill_with_neg_inf(torch.zeros(tgt_len, tgt_len))
    mask = torch.arange(tmp.size(-1))
    tmp.masked_fill_(mask < (mask + 1).view(tmp.size(-1), 1), 0)
    causal_mask_long = tmp.to(dtype=causal_mask_dtype, device=torch_device)

    assert torch.all(torch.eq(causal_mask_short, causal_mask_long))


if __name__ == "__main__":
    test_upper_right_triangle('cpu')
```
as well as the fact that when I run the ```convert``` function, the output gives the same predictions at the same speed.

Since this this would be testing one version of the code against another possible version, as opposed to some external ground truth or expected value,  it feels a bit self referential?

you're right, LGTM @LysandreJik !