PegasusForConditionalGeneration (torch version)
Author: sshleifer
Pull request: yes

This PR adds, [pegasus](https://arxiv.org/abs/1912.08777), a SOTA summarization model ported from [tf1] (https://github.com/google-research/pegasus) in collaboration with @JingqingZ .

More info on the model can be found in `pegasus.rst` under Files changed.
Config: [here](https://s3.amazonaws.com/models.huggingface.co/bert/google/pegasus-xsum/config.json)

#### TODO This PR:
- [x] convert to bart state dict format
- [x] working sentencepiece to
- [x] integration test with good summary on xsum data. (Haven't checked parity).
- [x] beam_alpha -> length_penalty approximation.
- [x] check xsum rouge with length penalty 1. 24.34 vs 24.56 Rouge 2 in paper (very good, no bug). Gap likely from different length penalty.
- [x] convert other checkpoints besides xsum
- [x] tokenizer must know max_source_length (`tokenizer_config.json`)
- [x] `model_doc/pegasus.rst` (document known fp16 issue)
- [x] move all checkpoints to `google/pegasus/{dataset}/` 
- [ ] model_cards (S3)

#### Future PR(s):
- [ ] TF 2.0
- [ ] `tokenizer.add_tokens` doesn't work.
- [ ] support for finetuning pegasus-large  (WIP see `finetune_pegasus.sh`)
- [ ] potentially add pegasus's `length_normalization` logic if it helps metrics substantially (over equivalent length_penalty).
- [ ] faster tokenizer tests (with smaller sentencepiece model.)
- [ ] try to find a clean way to add the pegasus length penalty.
- [ ] pick checkpoint for summarization pipeline default -- probably cnndm.



#### Known FP16 Issue
fp16 generation doesn't work for most sequences. We have an activation that is 101,610 in both fp32 and fp16 (the limit is 65,504).
In `#pegasus-collab`, the authors responded that they never used fp16 during pretraining/finetuning.
Things I tried that didn't help: 
- never use `FusedLayerNorm`
- increase `layernorm_eps` to 1 (from 1e-5)

Things I haven't tried:
- change all softmaxes to dtype=torch.float32
- manually divide by 100 and finetune more with some loss that discourages large activations.


#### Implementation Choices

- I inherited from Bart with 0 change to bart, but added a new config/modeling file for namespace consistency/control.
- `PegasusTokenizer` inherits from `ReformerTokenizer` -- both just use a single `spiece.model`.
- added common test coverage for the tokenizer, not the model since it is 0 LOC.
- added integration tests for xsum.



###  Inference API

datasets will vary between checkpoints, but otherwise, I think these are almost correct front matter
```
---
language: en
datasets:
- xsum
tags:
- summarization
---
```
This doesn't seem to be helping since [xsum](https://huggingface.co/google/pegasus-xsum) still thinks its for mask filling.


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6340?src=pr&el=h1) Report
> Merging [#6340](https://codecov.io/gh/huggingface/transformers/pull/6340?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/f6cb0f806efecb64df40c946dacaad0adad33d53&el=desc) will **increase** coverage by `1.80%`.
> The diff coverage is `94.50%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6340/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6340?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6340      +/-   ##
==========================================
+ Coverage   77.51%   79.32%   +1.80%     
==========================================
  Files         150      153       +3     
  Lines       27789    27877      +88     
==========================================
+ Hits        21542    22113     +571     
+ Misses       6247     5764     -483     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6340?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_marian.py](https://codecov.io/gh/huggingface/transformers/pull/6340/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19tYXJpYW4ucHk=) | `90.00% <ø> (-0.91%)` | :arrow_down: |
| [src/transformers/configuration\_pegasus.py](https://codecov.io/gh/huggingface/transformers/pull/6340/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX3BlZ2FzdXMucHk=) | `90.90% <90.90%> (ø)` | |
| [src/transformers/tokenization\_pegasus.py](https://codecov.io/gh/huggingface/transformers/pull/6340/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fcGVnYXN1cy5weQ==) | `93.54% <93.54%> (ø)` | |
| [src/transformers/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/6340/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9fX2luaXRfXy5weQ==) | `99.27% <100.00%> (+0.01%)` | :arrow_up: |
| [src/transformers/configuration\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/6340/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX2F1dG8ucHk=) | `93.33% <100.00%> (+0.15%)` | :arrow_up: |
| [src/transformers/configuration\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/6340/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX2JhcnQucHk=) | `94.23% <100.00%> (+0.48%)` | :arrow_up: |
| [src/transformers/modeling\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/6340/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19hdXRvLnB5) | `78.61% <100.00%> (+14.65%)` | :arrow_up: |
| [src/transformers/modeling\_pegasus.py](https://codecov.io/gh/huggingface/transformers/pull/6340/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19wZWdhc3VzLnB5) | `100.00% <100.00%> (ø)` | |
| [src/transformers/tokenization\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/6340/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fYXV0by5weQ==) | `95.45% <100.00%> (-2.33%)` | :arrow_down: |
| [src/transformers/tokenization\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/6340/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25feGxtLnB5) | `16.26% <0.00%> (-66.67%)` | :arrow_down: |
| ... and [27 more](https://codecov.io/gh/huggingface/transformers/pull/6340/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6340?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6340?src=pr&el=footer). Last update [f6cb0f8...95e8544](https://codecov.io/gh/huggingface/transformers/pull/6340?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


I'm going to merge after 2 hours of docs work, then take another pass to document prepare_seq2seq_batch consistently when other tokenizers implement it.