[Longformer] Multiple choice for longformer
Author: patrickvonplaten
Pull request: yes

## Description:

- This PR adds Multiple Choice for Longformer as in: #4644 (Sorry for not telling you earlier @patil-suraj). @ibeltagy 

- The documentation is updated for all models using MultipleChoice since their `input_ids` have 3 dimensions. It is done by use of `{}.format()` for `INPUTS_DOCSTRING`. @LysandreJik 

- Adds a couple of models that were missing to the respective `models_page` @LysandreJik 

@ibeltagy Regarding global attention - I think we probably should automatically add global attention here since the multiple inputs are flattened across the dimension `num_choices` and then should attend each other. Maybe add global attention always on the first token of each input of the dim `num_choices`?

- Global attention should probably still be implemented. Waiting for @ibeltagy answer.



Comments:
As mentioned in the paper, they used global  attention on all answer candidates for WikiHop, so maybe we can use global attention on all choice tokens ?

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4645?src=pr&el=h1) Report
> Merging [#4645](https://codecov.io/gh/huggingface/transformers/pull/4645?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/e444648a302dc8520beec96356a4bf500944355c&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4645/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4645?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4645      +/-   ##
==========================================
+ Coverage   77.42%   77.44%   +0.01%     
==========================================
  Files         128      128              
  Lines       21017    21046      +29     
==========================================
+ Hits        16273    16299      +26     
- Misses       4744     4747       +3     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4645?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/4645/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9fX2luaXRfXy5weQ==) | `99.13% <ø> (ø)` | |
| [src/transformers/modeling\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/4645/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19hdXRvLnB5) | `73.80% <ø> (ø)` | |
| [src/transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/4645/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iZXJ0LnB5) | `86.96% <100.00%> (ø)` | |
| [src/transformers/modeling\_longformer.py](https://codecov.io/gh/huggingface/transformers/pull/4645/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19sb25nZm9ybWVyLnB5) | `97.05% <100.00%> (+0.22%)` | :arrow_up: |
| [src/transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/4645/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19yb2JlcnRhLnB5) | `95.71% <100.00%> (ø)` | |
| [src/transformers/modeling\_tf\_albert.py](https://codecov.io/gh/huggingface/transformers/pull/4645/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9hbGJlcnQucHk=) | `78.74% <100.00%> (ø)` | |
| [src/transformers/modeling\_tf\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/4645/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9iZXJ0LnB5) | `96.54% <100.00%> (ø)` | |
| [src/transformers/modeling\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/4645/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ194bG5ldC5weQ==) | `75.73% <100.00%> (ø)` | |
| [src/transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/4645/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19vcGVuYWkucHk=) | `80.41% <0.00%> (-1.38%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4645/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.66% <0.00%> (+0.16%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4645?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4645?src=pr&el=footer). Last update [e444648...826c1b0](https://codecov.io/gh/huggingface/transformers/pull/4645?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Looks like multiple choice models are used for datasets with short input like swag (question + multiple choices), and datasets with long context like Wikihop (question + multiple choices + a long context). For the second use case, as @patil-suraj mentioned, we need global attention on all tokens of the question and the choices. 

For the first use case, I am not sure how to handle it. Maybe global attention everywhere, but in this case, it is equivalent to n^2 attention. Do you know of a dataset of the first use case with inputs longer than swag?

@ibeltagy If we have such multiple use-cases then I think it would be better if we leave this to the user. Not entirely sure though.

@patil-suraj, we can leave it to the user, or we can just do as you suggested earlier, put global attention on the question and all choices, which should work. 

@patrickvonplaten, what do you think?


Yeah I think ideally we would leave it to the user with a `global_attention_mask` input argument that is automatically set when `None`. We could actually have this for all forward functions...I'll think a bit about it tomorrow!

We had some internal discussion and decided to change / extend the API of `Longformer` slightly. 

We will have two "mask" arguments for every `forward()` function in Longformer:  `attention_mask` (as usual composed of 0's and 1's only) and a `global_attention_mask` also composed of zeros and ones (0 => local attention; 1=> global attention). If `global_attention` is not defined by the user we create if necessary `LongformerForQuestionAnswering` and maybe `LongformerForMultipleChoice`.

We will keep the inner workings the same (merge `attention_mask` with `global_attention_mask`) but make sure the user has a more intuitive API, since people always think of masks as boolean tensors in this library. 
Is that ok for you @ibeltagy ? 

I will merge this PR and open a new one with the proposed changes.