[GPT2] Add lm_attention_mask as a optional argument
Author: thomasw21
Pull request: yes

# What does this PR do?

We provide a new optional argument in order to pass down a mask that should be applied on attention layers. As the naming conflicts with `attention_mask` (which is use to determine which ones are pad values see https://huggingface.co/transformers/v3.1.0/glossary.html#attention-mask).

Context: we want to be able to obtain a prefixlm like behaviour: Full attention mask on a prefix, and auto regressive masking on suffix.

Some changes that were unrelated to the feature introduction:
 - We've also factorised code from `prepare_inputs_for_generation` into GPT2PretrainedModel as this was duplicated.
 
What this PR doesn't do:
 - Essentially `attention_mask` can we added into `lm_attention_mask` (ie you can put 0's for all pad values and such). However we won't remove `attention_mask` for backward compatibility.
 
TODO:
 - add tests
 
 Example of codes:
 
 ```python
 model = GPT2LMHeadModel.from_pretrained("<MODEL_NAME>")
 model.generate(random_inputs) # autoregressive
 
 lm_attention_mask = torch.ones(1, 1 , random_inputs.shape[1], random_inputs.shape[1], dtype=torch.bool, device=random_inputs.device)
 model.generate(random_inputs, lm_attention_mask=lm_attention_mask) # prefixlm
 
 lm_attention_mask = torch.triu(torch.ones(1, 1 , random_inputs.shape[1], random_inputs.shape[1], dtype=torch.bool, device=random_inputs.device))
 model.generate(random_inputs, lm_attention_mask=lm_attention_mask) # fancy prefixlm
 ```
 
## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

cc @patrickvonplaten @LysandreJik @sgugger 


Comments:
Thanks for the comments! I don't like `causal_mask` as the whole point is to pass a non causal mask, nor `language_model_mask` because all models here are language models, and doesn't says much about this variable ... 

Some suggestions:
 - attention_layer_mask 
 - key_query_mask
 - others ?

> because all models here are language models

I disagree. Sequence classification, token classification, question answering models are not language models. I understand this new argument will only be added to causal LMs and some seq2seq LMs maybe? So only language models compared to the other models in the library.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Awaiting some comments from @patrickvonplaten .

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

un-stale

This week is the week haha - sorry to be so incredibly slow with this

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.