[examples] SummarizationDataset cleanup
Author: sshleifer
Pull request: yes

- factor out redundant tokenization logic
-  For both articles and summaries, batches are "trimmed" such that no columns are full of `pad_token_id`. 
- The max sizes are 1024 for source and 56 for target. This ensures that truncation never happens for summaries, and rarely happens for articles. These values are unchanged, but converted to command line arguments for ease of use.
- added small unittest (just for the dataset).
- Verified manually on GPU. Loss goes down, peak memory usage identical, speed improved by ~10\%.

Summary Statistics on summary lengths (in # tokens) for cnn_dm test data:
![image](https://user-images.githubusercontent.com/6045025/77860710-94b83500-71de-11ea-8d09-e952f3dd9e8c.png)




Comments:
@acarrera94 does this look OK?

This looks great! Something else we might want to figure out is the best configuration of max_seq_length and max_target_length. For example, the tesla V100 in google colab can for sure handle about max_seq_length=768 and max_target_length=56 with a batch size of 4. It can't handle max_seq_length=1028 with the same configuration, since it will run out of memory. 

Going to address by trimming batches so that they don't add extra padding. (EDIT: this is not enough, still need to truncate on both sides.)

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3451?src=pr&el=h1) Report
> Merging [#3451](https://codecov.io/gh/huggingface/transformers/pull/3451?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/b0ad06951708b782e45b02a4d092f6fcde68a9b9&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3451/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3451?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3451      +/-   ##
==========================================
+ Coverage   78.02%   78.03%   +0.01%     
==========================================
  Files         104      104              
  Lines       17709    17709              
==========================================
+ Hits        13817    13819       +2     
+ Misses       3892     3890       -2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3451?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3451/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `92.10% <0.00%> (+0.12%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3451/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.96% <0.00%> (+0.16%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3451?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3451?src=pr&el=footer). Last update [b0ad069...94a0baa](https://codecov.io/gh/huggingface/transformers/pull/3451?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Hard to unittest cause of the PL dependency but I verified that the script runs locally.

@sshleifer, in MT code it is common to group the batches together to minimize padding. Do you think this is worth implementing? One downside of this method is that is doesn't really work on TPU or architectures that expect fixed sizes. 

Yes, is something like [SortishSampler](https://github.com/fastai/fastai/blob/master/fastai/text/data.py#L99) the right idea?

It seems easy to implement if you only consider padding on the `source` side. Do you know of an intelligent `key_func` to sort examples that considers both sides?






lgtm