[WIP] Add BART for summarization training with CNN/DM using pytorch-lightning
Author: andr-ec
Pull request: yes

This pull request adds to the example for BART for summarization. I used the [example for NER](https://github.com/huggingface/transformers/tree/master/examples/ner) using pytorch-lightning as guidance. This example will train on CNN/DM and evaluate, and get decent results, though I haven't trained it on the full dataset just yet. I'm sure there are better defaults for the hyperparams but these seem to work. 
I based this PR on the code I wrote in this [colab](https://colab.research.google.com/drive/1C4jEf0fnLiz6Xdx4TDz1OoO4BRCjCx1m).

This would hopefully close https://github.com/huggingface/transformers/issues/3004

## TODO
- [x] Be able to train the model on a GPU.
- [x] remove unused args
- [x] add test step and save results.  

Happy to hear any feedback!

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3236?src=pr&el=h1) Report
> Merging [#3236](https://codecov.io/gh/huggingface/transformers/pull/3236?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/9d4a01905fad4f5eed2e6c1037dea9877711427a&el=desc) will **not change** coverage by `%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3236/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3236?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #3236   +/-   ##
=======================================
  Coverage   77.56%   77.56%           
=======================================
  Files         100      100           
  Lines       16970    16970           
=======================================
  Hits        13162    13162           
  Misses       3808     3808           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3236?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3236?src=pr&el=footer). Last update [9d4a019...9d4a019](https://codecov.io/gh/huggingface/transformers/pull/3236?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Nice! @yjernite might be interested!

I made those requested changes. And yes I'm planning to run finetuning this weekend and share results.  I only have access to a k80 so it'll take a while 🤷🏽‍♂️

This looks awesome. Let's coordinate with https://github.com/huggingface/transformers/pull/3290 as well to share whatever code is possible. 

@nateraw can you do a review of this PR as well?

@acarrera94 I will try to get this working this week. If you are in the pytorch-lightning open slack we can also chat a bit more about the design. 

@nateraw I've made all of those changes and it looks like #3290 has been merged, anything else that needs to change? Thanks!

It's blocked on me, I should be able to get to it tonight. 

New code looks great. Excited to try it out!

Thanks for sticking with it @acarrera I'm really impressed how concise this became. Next we can get some numbers. 

@acarrera94 `run_train.sh` is using 19GB on my system.
Does your system use less?
I am also seeing no memory savings from adding `--fp16`. 
Thanks!


@sshleifer I usually ran it using --max_seq_lengt=756. And that used less than 16gb of memory with a batch size of 4, so we might want to change that default.  And I haven’t tried it using --fp16. That comes from BaseTransformer right?