[Longformer] Major Refactor
Author: patrickvonplaten
Pull request: yes

## Longformer Refactor

This PR does a major refactoring of Longformer. Mainly, the Roberta abstraction is removed and compositionally is chosen instead. This has the following advantages:

- It's easier now to implement a `cross_attention_layer`
- The code is more readable and the logic stays in this file only
- A bug was corrected regarding the attention mask. @ibeltagy - maybe you can check this as well. Previously, if **no** `attention_mask` was inserted, the padding function that became before `super.forward()` in `LongformerModel` was not used, **but** if instead an `attention_mask = torch.tensor([1, ..., 1])` (attend to all tokens was passed, the padding function was applied and could lead to different outputs as when no `attention_mask` is passed. This should not be the case. `model(input_ids)` and `model(input_ids, attention_mask=torch.ones(input_ids.shape))` should always yield the same result. Removing the `super.forward()` abstraction makes the code much cleaner here so that a `attention_mask = torch.ones(input_ids.shape)` can be calculated before calling the longformer encoder. **IMPORTANT** Since in almost all tasks longformer somehow passes either a `global_attention_mask` or `attention_mask` to `LongformerModel`, this bug did not really become visible before.
- We don't have to "inject" a `self-attention layer` into another model anymore, which I did not like very much.
- Unnecessary code can be removed (head_mask, prev cross-attention layer inputs that do not work yet), ...

**Additionally**:
- Variable names are made more explicit and dead code (If statements that would have never occurred) was removed and code is simplified.
- The forward function of the self-attention layer is broken up into multiple helper functions. The advantage here is that quite some memory should be saved before `attention_probs` go out of scope after they are not used anymore and thus the memory bottleneck should be reduced. 
- All longformer models are added to the tests (@sgugger) and a couple more tests are added.

Next step is to add cross attention layers to longformer.

**Review**

I made sure that besides the bug with `attention_mask = None` vs `attention_mask = torch.ones(...)` all outputs stay the same.
Would be great if @thomwolf  @LysandreJik @sgugger @sshleifer @ibeltagy can do a quick review.


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5219?src=pr&el=h1) Report
> Merging [#5219](https://codecov.io/gh/huggingface/transformers/pull/5219?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/9a473f1e43221348334b9e7f95bb45770b7ef268&el=desc) will **decrease** coverage by `0.81%`.
> The diff coverage is `92.60%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5219/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5219?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5219      +/-   ##
==========================================
- Coverage   77.85%   77.04%   -0.82%     
==========================================
  Files         138      138              
  Lines       24314    24409      +95     
==========================================
- Hits        18930    18806     -124     
- Misses       5384     5603     +219     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5219?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_longformer.py](https://codecov.io/gh/huggingface/transformers/pull/5219/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19sb25nZm9ybWVyLnB5) | `91.66% <92.60%> (-1.45%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_mobilebert.py](https://codecov.io/gh/huggingface/transformers/pull/5219/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9tb2JpbGViZXJ0LnB5) | `23.62% <0.00%> (-73.11%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/5219/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9iZXJ0LnB5) | `73.37% <0.00%> (-25.00%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5219/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.39% <0.00%> (-0.15%)` | :arrow_down: |
| [src/transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5219/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19vcGVuYWkucHk=) | `81.09% <0.00%> (+1.37%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/5219/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9kaXN0aWxiZXJ0LnB5) | `98.76% <0.00%> (+32.51%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5219/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9vcGVuYWkucHk=) | `94.98% <0.00%> (+74.19%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5219?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5219?src=pr&el=footer). Last update [9a473f1...90d2aa6](https://codecov.io/gh/huggingface/transformers/pull/5219?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


@sshleifer and @ibeltagy - thanks a lot for your comments -> cleaned up the comments and some function naming.

All slow and normal tests pass on GPU => good to merge.