Fix use of mems in Transformer-XL text generation
Author: tommccoy1
Pull request: yes

In Transformer-XL, when ```mems``` is being used to save computation with the ```generate``` function, the inputs are not properly truncated, so that ```mems``` does not actually speed things up, and also seems to create inaccuracies in the output. I have attempted to fix this by changing Transformer-XL's ```prepare_inputs_for_generation``` function to make it more like that function as used in GPT-2.

See the issue at https://github.com/huggingface/transformers/issues/4752 for more details.


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4826?src=pr&el=h1) Report
> Merging [#4826](https://codecov.io/gh/huggingface/transformers/pull/4826?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/c58e6c129a153ca1a5021e5d7e642d00bf011e20&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4826/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4826?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4826   +/-   ##
=======================================
  Coverage   74.52%   74.53%           
=======================================
  Files         128      128           
  Lines       21497    21499    +2     
=======================================
+ Hits        16021    16024    +3     
+ Misses       5476     5475    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4826?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/4826/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190cmFuc2ZvX3hsLnB5) | `77.27% <100.00%> (+0.31%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4826/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.85% <0.00%> (-0.42%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4826/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.36% <0.00%> (ø)` | |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4826/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.27% <0.00%> (+0.12%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4826?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4826?src=pr&el=footer). Last update [c58e6c1...07c4126](https://codecov.io/gh/huggingface/transformers/pull/4826?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Hey @tommccoy1,

Thanks a lot for your PR! From some initial tests with this change, the results seem good! 
The bug you pointed out, might also be affecting `xlnet` actually...I will have to take a deeper look into both models to fully understand what's going on with `mems`. 

But also given the discussion on the original Transfo-XL repo, here: https://github.com/kimiyoung/transformer-xl/issues/49 suggests that you are 100% correct with your PR here.

It seems like you are passing all the tests. They will surely be one `RUN_SLOW` test that will fail, but this might also be due to prior incorrect assumptions regarding the `.generate()` function. I will check this PR next week :-) 

Also related:  #505

Hey, I'm taking a look at this atm - as expected by @patrickvonplaten, the slow test fails, but that's probably an issue with the slow test. I'll update it and merge the PR.

Hey, sorry for the delay - we've realized XLNet had a similar issue, and I'm opening up another PR to fix the slow tests to be consistent with this.

No worries re: the delay - thank you for looking over it & merging it!