New TensorFlow trainer version
Author: jplu
Pull request: yes

Hello,

This PR is a proposal for an updated version of the current TensorFlow trainer. This new trainer brings the following improvements:

- Uses the Keras methods `.compile()` + `.fit()` instead of the custom training loop. This change brings a better integration with TensorFlow and the different strategies that can be used for training a model. It takes also advantages of all the optimizations done by the Google team for a proper training.
- Uses the Keras methods `.evaluate()` and `predict()` instead of the custom evaluation loop. Same advantages than for the training part.
- Uses the Keras callbacks and metrics. We can takes advantages of the callback and metric features proposed by default when training/evaluate a model with Keras. Also one can create its own callbacks and metrics and use them for the training/evaluation.
- Big reduction in terms of line of codes which makes it easier to maintain.
- Create a new optimizer for gradient accumulation to move the logic inside this new optimizer than inside the trainer instead.

Of course this is still far to be finished and there is still work to do but you can easily see the direction I'm thinking of.

@LysandreJik @sgugger I will be happy to ear your comments.

@thomwolf @lhoestq Here I have created a file where I put all the Keras metrics, but we should definitely think a way to integrate such metrics directly inside `datasets` where they will be better suited.

Comments:
# 1. Input data

> The PyTorch side of the library (and all the PyTorch scripts) have datasets that eventually yield dictionaries containing all inputs of the model as well as the labels. In an ideal world, it would be great if we could leverage that format easily for TF as well (so that the same script can be used for PT and TF by just changing a few lines, especially when using the datasets library). I don't know if that's possible or not but one thing to explore more I believe.

Can you elaborate a bit more please? Do you mean that the input data given to the `.fit()` method should be a dictionary? If it is what you mean, it is already the case.

# 2. Optimizer and scheduler
> I like the new optimizer for gradient accumulation a lot. This feels like a very good design. Should we deprecate GradientAccumulator?

Yes, this should be deprecated because we won't use it anymore.

> But where are the schedulers? Is this something you intend to control via callbacks? If that's the case I didn't see one with a sensible default (the linear + warmup used in PT for instance).

The schedulers are directly inside the optimizer, if you look at the `create_optimizer` method you can see that the scheduler is first created and then given to the Adam optimizer as input for the `learning_rate` parameter. In the previous Trainer the scheduler was returned only for being used in the logging, the scheduling is done automatically internally in the `tf.keras.optimizers.Optimizer` class.

# 3. Callbacks
> Leveraging Keras callbacks is definitely a good idea. My only remark here is that is should be more customizable. On the PT side we have some default callbacks and the init takes a list of additional callbacks the user can add.

No worries, this will be added in the next push :)

# 4. Metrics
> I'm not in favor of adding a new file of metrics we will have to maintain forever. We should provide an adapter for datasets Metric object and rely on the datasets library (or existing Keras metrics if users prefer them) but they shouldn't be in the transformers library (on the PT we will soon deprecate the ones in the metric folder).

I fully agree, and this is why I asked @thomwolf and @lhoestq opinion's on this on the best way to integrate Keras metrics inside datasets :) For this will require, I think, a non negligeable amont of work in `datasets` that I would prefer to do not do alone.

> Can you elaborate a bit more please? Do you mean that the input data given to the `.fit()` method should be a dictionary? If it is what you mean, it is already the case.

If you look at the new example scripts (like [run_glue](https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_glue.py)) the datasets are immediately sent to `Trainer` with no platform-specific processing needed. It would be really cool if we could just replace `Trainer` by `TFTrainer` in that script and have it work the same way. I'm not sure if the easiest for that is to change the input of `training_step` or do some internal processing of the dataset inside `TFTrainer`.

> If you look at the new example scripts (like run_glue) the datasets are immediately sent to Trainer with no platform-specific processing needed. It would be really cool if we could just replace Trainer by TFTrainer in that script and have it work the same way. I'm not sure if the easiest for that is to change the input of training_step or do some internal processing of the dataset inside TFTrainer.

Hum I see. As a first glance, I would say it will requires much more changes than just replace Trainer by TFTrainer, at least all the metrics part won't be compliant, and the way we create the model is different (in TF we have to do that in a strategy).

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.