Add cached past for language generation
Author: patrickvonplaten
Pull request: yes

add past input for gpt2 and ctrl for faster decoding for language generation.

1. add `prepare_inputs_for_generation` fn for gpt2 and ctrl 
2. add private `_do_output_past` fn for PretrainModel class to check whether model outputs past key-value states 
      - fn only covers cases for gpt2 and ctrl for the moment and needs to add 'xlnet' and 'transfo_xl' via `mem_len`
      - might be better to move `_do_output_past` to each individual LMHeadModel
3. rename `pasts` to `past`

can also add dummy tests for language generation

Comments:
That a great idea, @patrickvonplaten 

I'll let you finish this PR when you have time and ping me for review or questions.

True, I will implement this tomorrow!

tested for transfo_xl, gpt2, openai-gpt and xlnet in combination with PR #2289 

This looks great!
To pass the code quality test, you can use `make style`.
Please read this section of the (new) CONTRIBUTING guidelines: https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2292?src=pr&el=h1) Report
> Merging [#2292](https://codecov.io/gh/huggingface/transformers/pull/2292?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/aeef4823ab6099249679756182700e6800024c36?src=pr&el=desc) will **decrease** coverage by `0.17%`.
> The diff coverage is `8.51%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2292/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2292?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #2292      +/-   ##
==========================================
- Coverage   73.49%   73.32%   -0.18%     
==========================================
  Files          87       87              
  Lines       14793    14833      +40     
==========================================
+ Hits        10872    10876       +4     
- Misses       3921     3957      +36
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2292?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/2292/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ194bG5ldC5weQ==) | `72.9% <0%> (-0.37%)` | :arrow_down: |
| [src/transformers/modeling\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/2292/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19ncHQyLnB5) | `83.17% <16.66%> (-1.3%)` | :arrow_down: |
| [src/transformers/modeling\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/2292/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19jdHJsLnB5) | `94.22% <16.66%> (-2.13%)` | :arrow_down: |
| [src/transformers/modeling\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/2292/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190cmFuc2ZvX3hsLnB5) | `74.68% <20%> (-0.59%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2292/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `61.44% <3.84%> (-2.02%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2292?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2292?src=pr&el=footer). Last update [aeef482...fc84bd5](https://codecov.io/gh/huggingface/transformers/pull/2292?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Ok good for now, merging