[XLNet] Fix mems behavior
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #7584 

XLNet is arguably not the cleanent implementation with a lot of config parameters flying around that all interact with each other in an overly complicated way: `use_cache`, `mem_len`, and `reuse_len`. Due to backward compatibility we cannot really get rid of those. I'd love to just completely get rid of `use_cache` and `reuse_len`, but this would require to update all configs which is not possible...

At first, this PR removes the `use_cache` and replaces it with `use_mems_eval` => `use_mems_eval` decides whether the mems should be used in evaluation mode, which defaults to `True` so that the arguably most important model `XLNetLMHeadModel` keeps full backward compatibility at inference. `use_cache` is a confusing name IMO because it does not correspond to the `use_cache` we know from GPT2 (we had a longer discussion on this internally on Slack).

The issue #7584 shows that if there is one training batch that is smaller than the other batches in the train data, the training breaks. Also as can be read upon in the issue linked below, the authors also don't use the memory mechanism for fine-tuning => therefore we add another param `use_mems_train` which defaults to `False` so that training works as a default. 
If for some special reason the user wants to use the memory mechanism during fine-tuning, he/she has to make sure that the batch_size of all training batches is the same.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments:
> The new arguments are perfectly acceptable, but I think it would be nice to keep backwards compatibility. It should be easy enough by checking if there's a `use_cache` value in the configuration and setting it to the value of `use_mems`.

Yes, true! The problem is that `use_cache` is always in `configuration_utils.py` so we can't really add something that checks whether `use_cache` is in the config (It's always there) and sets `use_mems` to the same value (It'd always do it)...

Thinking more about it `use_cache` should maybe not even be in `configuration_utils.py` but just in all "LMHead" configs?! I could move `use_cache` to all individual config files. I think we could do this without breaking changes -> Wdyt? @LysandreJik @sgugger ?

I have mixed feelings about this, on one hand it makes sense to remove that argument which is very specific to a few models from the main `configuration_utils`, but on the other I fear the associated breaking changes.

I think your last proposal would be great @patrickvonplaten, moving `use_cache` to the appropriate configuration files. This way we could have this PR with no breaking changes, and a more robust `use_cache`!

If it is not possible to do the move without breaking change, then let's forget this and fix the mems behavior with the small breaking change for version 4. Would still like to prevent this, if possible.

## Update 

@LysandreJik @sgugger - `use_cache` is only used in the following modeling files (TF & PT):
```
- modeling_t5.py
- modeling_bart.py (+ inherited)
- modeling_fstm.py
- modeling_prophetnet.py
- modeling_gpt2.py
- modeling_openai.py
- modeling_xlnet.py
- modeling_ctrl.py
```

Therefore, we can move `use_cache` to the respective configuration files and delete it from the general `configuration_utils.py` file.
I cannot really think of a use case where this would lead to breaking changes. If *e.g.* a saved BERT config includes a `use_cache`, this parameter will still be set to the config: https://github.com/huggingface/transformers/blob/18c8cf000bed04ec03470270ec2fbd9d49cce5c4/src/transformers/configuration_utils.py#L235 and still remain unused as before => I don't think this is a problem. A lot of old configs have unused config parameters that are then just ignored... For all models that make use of `use_cache`, the `use_cache` param is now "caught" and saved directly in the model's config class. 

=> this allows us to have 0 breaking changes for this PR and is also cleaner IMO. 

## Update

To have a bit more clarity for this PR.
This PR mainly solved the issue that XLNet cannot be trained at the moment. It does so by depreciating `use_cache` and replacing it by `use_mems_eval` and `use_mems_train`.
The PR keeps 100% backward compatibility for the PyTorch model.

The TF XLNet model was previously not kept up-to-date with the PT model. This PR notably forgot to update the TF model: https://github.com/huggingface/transformers/pull/5770 when the behavior of the PT model was changed. 
This PR brings the TF model up to date with the PT model, but does not keep 100% backward compatibility by completely removing the `use_cache` parameter from the models forward functions. I decided to not add depreciate the `use_cache` parameter because a) Before this PR `use_cache` was used incorrectly (as was corrected in #5770 but only for PT), thus b) bringing the TF model up to date with PT is breaking anyways and c) there are breaking changes for boolean inputs in the moment for TF. So in conclusion:

- for PT: No breaking changes -> `use_cache` is depreciated both in the config and in the model forward's
- for TF: One breaking change for TF that `use_cache` cannot be forwarded anymore to the models and has to be replaced by `use_mems`

At the same time this PR removes `use_cache` from `configuration_utils.py` which has no breaking changes except for models that don't use `use_cache` don't add an unused `use_cache` that defaults to `True` to their config anymore. But since none of those models is using `use_cache` this is hardly breaking.

@LysandreJik - is that ok for you?

Thank you for clarifying, I think that's fine. We should add that to the v4.0.0 as well. 

Thanks for taking care of it @patrickvonplaten