Output global_attentions in Longformer models
Author: gui11aume
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #7514

[From @patrickvonplaten]:

This PR introduces a new structure for the output attentions in Longformer. There are two types of attentions in Longformer: local attention outputs and global attention outputs. Previously, if global attention was used the `attentions` were set to the global attentions and the local attentions were discarded. This is suboptimal as one has no access to the local attentions in this case. The better design IMO is to have both `attentions` and `global_attentions` in Longformer (similar to `encoder_attentions`, `decoder_attentions` in Seq2Seq and `attentions`, `ngram_attentions` in ProphetNet).

Also, the PR switches from tuple indexing to using `ModelOutput` kwargs in the `test_attention_output` function which IMO we should bit by bit do for all tests from now on. In PT Longformer, the `is_global_attn_index` tensor is now only calculated once instead for each layer which slightly speeds up computation.

Awesome job @gui11aume! Especially for the docstring -> the description of `global_attentions` and `attentions` is impeccable.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests? 


## Who can review?

@patrickvonplaten

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

Comments:
We are actually having a longer internal discussion about the general handling of different attentions - this might still take a couple of days to be decided.

> Cool, wonderful that you added so many tests!
> 
> @patrickvonplaten, you say:
> 
> > if global attention was used the attentions were set to the global attentions and the local attentions were discarded
> 
> In this case, wouldn't the best model output be one where there are both `local_attentions`, `global_attentions` **as well as** `attentions` that are kept simply for backwards compatibility?

The previous design led to errors as shown here: https://github.com/huggingface/transformers/issues/5646 -> so I think it's fine to break backwards compatibility here. `local_attentions` would arguably be a better name than `attentions`, but for consistency with other models and for a standard case where `global_attention_mask=None`, so that `local_attentions` == (all) `attentions`, I would prefer to keep the name `attentions` here. 

> 
> Other than that and the docstrings, LGTM!

Docstrings will be corrected!



Works for me!

@gui11aume great work again - understanding longformer's attention is not straightforward and your doc string was spot-on! Looking forward to your next contribution ;-) Hope you're fine with the small changes I made

@patrickvonplaten @lalitpagaria this merge PR breaks longformer training with gradient checkpointing True
please fix as i am unable to train models with latest models. 


error comes on this line 
https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_longformer.py#L1072

model expect 2 to 6 positional arguments, but 7 where giving. 

> @patrickvonplaten @lalitpagaria this merge PR breaks longformer training with gradient checkpointing True
> please fix as i am unable to train models with latest models.
> 
> error comes on this line
> https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_longformer.py#L1072
> 
> model expect 2 to 6 positional arguments, but 7 where giving.

Hey @manishiitg,

Thanks a lot for message! This did indeed break gradient checkpointing for longformer - sorry! This PR fixes it: https://github.com/huggingface/transformers/pull/8415