Clean Encoder-Decoder models with Bart/T5-like API and add generate possibility
Author: patrickvonplaten
Pull request: yes

Bert-Bert Encoder-Decoder models can now be used as is shown in the test cases: 
`tests/test_modeling_encoder_decoder.py`. 
Tests include:
- forward `input_ids` and `decoder_input_ids` for Bert-Bert
- backprop using masked language model loss for Bert-Bert
- backprop using "conventional" language model loss for Bert-Bert
- using the `generate()` fn with Bart-Bart
- saving and loading of Encoder-Decoder models.

Before merging a couple of things have to be agreed on as mentioned in the comments below.

UPDATE: 

This branch is IMO now fully functional for Bert-2-Bert models. 
I will finish the PR (clean the code, make a pretty docstring, etc...) once we agreed on the issues I mentioned further down. Would be very happy if you can review @thomwolf @LysandreJik @sshleifer @julien-c @yjernite 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3383?src=pr&el=h1) Report
> Merging [#3383](https://codecov.io/gh/huggingface/transformers/pull/3383?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/857ccdb259b7e46c60cf86c58b7ab038c63e4d4e&el=desc) will **increase** coverage by `0.29%`.
> The diff coverage is `88.46%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3383/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3383?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3383      +/-   ##
==========================================
+ Coverage   78.61%   78.90%   +0.29%     
==========================================
  Files         106      105       -1     
  Lines       17953    17973      +20     
==========================================
+ Hits        14114    14182      +68     
+ Misses       3839     3791      -48     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3383?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/3383/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iZXJ0LnB5) | `89.01% <75.00%> (+0.60%)` | :arrow_up: |
| [src/transformers/modeling\_encoder\_decoder.py](https://codecov.io/gh/huggingface/transformers/pull/3383/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19lbmNvZGVyX2RlY29kZXIucHk=) | `84.41% <92.30%> (+63.36%)` | :arrow_up: |
| [src/transformers/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/3383/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9fX2luaXRfXy5weQ==) | `98.98% <100.00%> (ø)` | |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3383/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.06% <0.00%> (+0.12%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3383/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.76% <0.00%> (+0.16%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3383?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3383?src=pr&el=footer). Last update [857ccdb...83f3d10](https://codecov.io/gh/huggingface/transformers/pull/3383?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


UPDATE:

I really liked the idea of adding a encoder_decoder config (@thomwolf ) and having for both the encoder_decoder config and model 2 `from_pretrained` fn:
1. The standard one which is used thanks to inheritence to `PretrainedConfig` and `PretrainedModel` 
2. a `from_encoder_decoder_pretrained` fn  (@sshleifer)

To understand how to use the encoder decoder class please confer to the added tests.

Code is cleaned: added type hints, cleaned the docstring and added a encoder-decoder model page.
Just need to resolve the issue with importing Bert's model tester. @sshleifer found a solution. If everybody is fine with it - I'll go for it :-) 

LGTM!

Ok Good to merge for me! If @sshleifer it's ok for you I will use your PR #4027 for the new test proposition.

Yes!