Add tests to Trainer
Author: sgugger
Pull request: yes

This PR moves the tests of the various `data_collator` in `test_data_collator.py` and adds tests of the Trainer on a simple regression problem. While testing, a few problems were uncovered:
- The number of epochs is documented as a float but used as an int, fixed the documentation.
- There was one more step done than specified by the argument `max_steps`.
- The evaluation loss was wrong whenever the evaluation dataset length is not a round multiple of the batch size.

Those three things are also fixed in the PR. With the regression infrastructure, we can add more tests (for custom data collator, optimizers, schedulers etc...) since each training is fast. Will do in follow-up PRs as this one was starting to be of a decent size already.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6605?src=pr&el=h1) Report
> Merging [#6605](https://codecov.io/gh/huggingface/transformers/pull/6605?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/fe0b85e77a6af041471657069bbb9c21a880cd5c?el=desc) will **increase** coverage by `0.08%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6605/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6605?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6605      +/-   ##
==========================================
+ Coverage   80.21%   80.30%   +0.08%     
==========================================
  Files         156      156              
  Lines       28178    28205      +27     
==========================================
+ Hits        22604    22650      +46     
+ Misses       5574     5555      -19     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6605?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/configuration\_reformer.py](https://codecov.io/gh/huggingface/transformers/pull/6605/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX3JlZm9ybWVyLnB5) | `100.00% <ø> (ø)` | |
| [src/transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/6605/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iZXJ0LnB5) | `88.42% <ø> (ø)` | |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6605/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `88.05% <ø> (ø)` | |
| [src/transformers/training\_args.py](https://codecov.io/gh/huggingface/transformers/pull/6605/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmluZ19hcmdzLnB5) | `91.26% <ø> (+10.67%)` | :arrow_up: |
| [src/transformers/configuration\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6605/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX3V0aWxzLnB5) | `95.97% <100.00%> (+0.02%)` | :arrow_up: |
| [src/transformers/data/data\_collator.py](https://codecov.io/gh/huggingface/transformers/pull/6605/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL2RhdGFfY29sbGF0b3IucHk=) | `90.90% <100.00%> (ø)` | |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6605/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.21% <100.00%> (-0.26%)` | :arrow_down: |
| [src/transformers/modeling\_albert.py](https://codecov.io/gh/huggingface/transformers/pull/6605/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19hbGJlcnQucHk=) | `83.50% <100.00%> (+0.13%)` | :arrow_up: |
| [src/transformers/modeling\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/6605/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19kaXN0aWxiZXJ0LnB5) | `97.84% <100.00%> (+0.02%)` | :arrow_up: |
| [src/transformers/modeling\_longformer.py](https://codecov.io/gh/huggingface/transformers/pull/6605/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19sb25nZm9ybWVyLnB5) | `92.02% <100.00%> (+0.07%)` | :arrow_up: |
| ... and [15 more](https://codecov.io/gh/huggingface/transformers/pull/6605/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6605?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6605?src=pr&el=footer). Last update [9a86321...3f89b2d](https://codecov.io/gh/huggingface/transformers/pull/6605?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Re. the eval loss, did you also run [test_trainer_distributed.py](https://github.com/huggingface/transformers/blob/master/tests/test_trainer_distributed.py) on a multi-gpu machine?

No, I don't have a multi-GPU machine setup. It does not seem like this test uses the eval_loss anywhere, it only computes a metric.

> No, I don't have a multi-GPU machine setup.

You can use the office machines! Not necessarily related to this PR but to keep in mind to run this test once in a while

> Not necessarily related to this PR but to keep in mind to run this test once in a while

It was indeed broken not due to this PR, fixed. Know how to run it periodically now :-)