Add generate() functionality to TF 2.0
Author: patrickvonplaten
Pull request: yes

I added the `_generate_no_beam_search` functionality for TF 2.0. It works for the following models:
'gpt2', 'openai', 'xlnet', 'xlm', 'ctrl'. Only for the model 'transfo-xl' it doesn't work because the 
lm_head is not implemented yet in TF 2.0 (added an issue here: #3064 ). 

Also I checked whether the pytorch 'distilgpt2' and TF 2.0 'distilgpt2' generate the same output (added one Integration test for this). Will add other integration tests in a future PR.

Setting only certain indices to values is much less straight forward in TF 2.0 than in pytorch, which is why I added more code for the TF 2.0 version. 

Would be very happy about some feedback @LysandreJik @thomwolf 

EDIT: There was also a bug in TFCTRL where tf.concat uses a pytorch argument 'dim' instead of 'axis'

## TODO
- [x] Discuss how to change the test_torch_tf_conversion.py() @LysandreJik @thomwolf 

## Future PR:
- [ ] Adapt all LMHead Integration Tests to greedy generate to be able to compare PT & TF
- [ ] Add generate() to TFTransfoXL (see Issue #3064 )

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3063?src=pr&el=h1) Report
> Merging [#3063](https://codecov.io/gh/huggingface/transformers/pull/3063?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/eec5ec807135ae61fa2266f3c7ad947cc207abda?src=pr&el=desc) will **increase** coverage by `0.22%`.
> The diff coverage is `90.1%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3063/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/3063?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3063      +/-   ##
==========================================
+ Coverage   77.59%   77.82%   +0.22%     
==========================================
  Files          98       98              
  Lines       16250    16422     +172     
==========================================
+ Hits        12610    12780     +170     
- Misses       3640     3642       +2
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3063?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/3063/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9fX2luaXRfXy5weQ==) | `98.91% <100%> (ø)` | :arrow_up: |
| [src/transformers/modeling\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/3063/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19jdHJsLnB5) | `98.22% <100%> (-0.02%)` | :arrow_down: |
| [src/transformers/modeling\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/3063/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ194bG5ldC5weQ==) | `75.77% <100%> (ø)` | :arrow_up: |
| [src/transformers/modeling\_tf\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/3063/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9ncHQyLnB5) | `96.14% <100%> (+1.47%)` | :arrow_up: |
| [src/transformers/modeling\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/3063/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19ncHQyLnB5) | `86.07% <100%> (-0.09%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/3063/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl94bG5ldC5weQ==) | `88.98% <100%> (+0.6%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/3063/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9jdHJsLnB5) | `99.57% <100%> (+1.74%)` | :arrow_up: |
| [src/transformers/modeling\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/3063/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190cmFuc2ZvX3hsLnB5) | `75.63% <100%> (ø)` | :arrow_up: |
| [src/transformers/modeling\_tf\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/3063/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl90cmFuc2ZvX3hsLnB5) | `91.13% <20%> (-1%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3063/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `91.56% <89.85%> (-1.22%)` | :arrow_down: |
| ... and [5 more](https://codecov.io/gh/huggingface/transformers/pull/3063/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3063?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3063?src=pr&el=footer). Last update [eec5ec8...b996a97](https://codecov.io/gh/huggingface/transformers/pull/3063?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


> This is cool, good job on the TensorFlow implementation! Regarding the use of the `past` and `mems`, I don't think they're actually implemented in the models though?

I think the tf models also have the `past` and `mems` functionality implemented (when looking into the tf modeling files the `past` and `mems` variables are used in the code.

Good to merge for me! 
Changed all `<tf_tensor>.shape` to `shape_list(<tf_tensor>)` to make function compatible with both eagermode and no eagermode after discussion with @thomwolf  and @jplu .

Will add additional Integration tests (so far only `tf_gpt2`) for other LMHead models and add beam_search once completed in torch version.