Bart-CNN
Author: sshleifer
Pull request: yes

## Sources

- copy pastes code from generate but does not share very much in an effort to simplify. there is no big abstraction yet.
- also copy pastes some code from fairseq
- encoder_outputs and previous decoder attentions are cached.

## Differences with PretrainedModel.generate

these are all bc thats the way fairseq does it!
- BeamHypotheses(early_stopping=True)
- assumptions about various token_ids being present
- force decoder to start with EOS, then predict BOS
- decoder only considers the most recently generated token bc everything else is cached.
- prevent predictions of various special tokens at inopportune moments (all the -inf stuff)
- force eos if you hit max length
- max_length is about how many tokens you want to generate. Doesn't matter how many you have.
- min_len parameter to prevent short summaries
- no_ngram_repetition parameter (set to 3 in Bart-CNN) to prevent repetition.

## TODO
- [ ] docstrings
- [ ] Mystery: results are identical to fairseq 98.6% of the time, 1.4% of the time they differ by a few words.
- [ ] run rouge metrics, compare run time to fairseq.
- [ ] Resist pressure to make big seq2seq abstraction before there are more callers
- [ ] Deeper dive on the MaskedLM.tie_weights hack, what is right way to do it?


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3059?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@271344f`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `86.95%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3059/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/3059?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##             master    #3059   +/-   ##
=========================================
  Coverage          ?   76.59%           
=========================================
  Files             ?       98           
  Lines             ?    16219           
  Branches          ?        0           
=========================================
  Hits              ?    12423           
  Misses            ?     3796           
  Partials          ?        0
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3059?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3059/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `93.08% <ø> (ø)` | |
| [src/transformers/modeling\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/3059/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iYXJ0LnB5) | `93.05% <86.95%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3059?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3059?src=pr&el=footer). Last update [271344f...6e13b56](https://codecov.io/gh/huggingface/transformers/pull/3059?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


> ## Differences with PretrainedModel.generate
> these are all bc thats the way fairseq does it!
> 
> * BeamHypotheses(early_stopping=True)
I think we have that option as well
> * assumptions about various token_ids being present
> * force decoder to start with EOS, then predict BOS
That's weird, no?
> * decoder only considers the most recently generated token bc everything else is cached.
> * prevent predictions of various special tokens at inopportune moments (all the -inf stuff)
> * force eos if you hit max length
We had this in our code before as well - I deleted it because I think unfinished sentences (sentences that were finished because they hit `max_length`) should not be ended with an EOS.
> * max_length is about how many tokens you want to generate. Doesn't matter how many you have.
This makes sense since encoder-decoder models always start from 0 `input_ids` for the decoder model and only have `encoder_input_ids` where as the standard "only-decoder" models (GPT2) have `decoder_input_ids` and append their output to it