Repository: huggingface/transformers

Issue:
[finetune_trainer] --evaluate_during_training is no more
In `examples/seq2seq/builtin_trainer/` all scripts reference `--evaluate_during_training ` but it doesn't exist in pt trainer, but does exist in tf trainer:
```
grep -Ir evaluate_during
builtin_trainer/finetune.sh:    --do_train --do_eval --do_predict --evaluate_during_training \
builtin_trainer/train_distil_marian_enro.sh:    --do_train --do_eval --do_predict --evaluate_during_training\
builtin_trainer/finetune_tpu.sh:    --do_train --do_eval --evaluate_during_training \
builtin_trainer/train_distilbart_cnn.sh:    --do_train --do_eval --do_predict --evaluate_during_training \
builtin_trainer/train_distil_marian_enro_tpu.sh:    --do_train --do_eval --evaluate_during_training \
builtin_trainer/train_mbart_cc25_enro.sh:    --do_train --do_eval --do_predict --evaluate_during_training \
```

```
Traceback (most recent call last):
  File "finetune_trainer.py", line 310, in <module>
    main()
  File "finetune_trainer.py", line 118, in main
    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/transformers/hf_argparser.py", line 144, in parse_args_into_dataclasses
    raise ValueError(f"Some specified arguments are not used by the HfArgumentParser: {remaining_args}")
ValueError: Some specified arguments are not used by the HfArgumentParser: ['--evaluate_during_training']
```
Is this meant to be replaced by: `--evaluation_strategy` - this is the closest I found in `training_args.py`

If so which one? `steps` or `epoch`?

Also the help output is borked:

```
$ python finetune_trainer.py -h
...
 [--evaluation_strategy {EvaluationStrategy.NO,EvaluationStrategy.STEPS,EvaluationStrategy.EPOCH}]
```
probably this is not what what's intended, but 
```
 [--evaluation_strategy {no, steps, epochs}
```

But perhaps it's a bigger issue - I see `trainer.args.evaluate_during_training`:
```
src/transformers/integrations.py:        ) and (not trainer.args.do_eval or not trainer.args.evaluate_during_training):
```
and also `--evaluate_during_training` in many other files under `examples/`.

Thank you.

@sgugger, @patrickvonplaten 



Discussion:
Found the source of breakage: https://github.com/huggingface/transformers/pull/8604 - I guess that PR needs more work

Resolving patch:
diff --git a/src/transformers/integrations.py b/src/transformers/integrations.py
--- a/src/transformers/integrations.py
+++ b/src/transformers/integrations.py
@@ -2,6 +2,7 @@
 import math
 import os
 
+from .trainer_utils import EvaluationStrategy
 from .utils import logging
 
 
@@ -212,13 +213,13 @@ def _objective(trial, checkpoint_dir=None):
         # Check for `do_eval` and `eval_during_training` for schedulers that require intermediate reporting.
         if isinstance(
             kwargs["scheduler"], (ASHAScheduler, MedianStoppingRule, HyperBandForBOHB, PopulationBasedTraining)
-        ) and (not trainer.args.do_eval or not trainer.args.evaluate_during_training):
+        ) and (not trainer.args.do_eval or trainer.args.evaluation_strategy == EvaluationStrategy.NO):
             raise RuntimeError(
                 "You are using {cls} as a scheduler but you haven't enabled evaluation during training. "
                 "This means your trials will not report intermediate results to Ray Tune, and "
                 "can thus not be stopped early or used to exploit other trials parameters. "
                 "If this is what you want, do not use {cls}. If you would like to use {cls}, "
-                "make sure you pass `do_eval=True` and `evaluate_during_training=True` in the "
+                "make sure you pass `do_eval=True` and `evaluation_strategy='steps'` in the "
                 "Trainer `args`.".format(cls=type(kwargs["scheduler"]).__name__)
             )
 
diff --git a/src/transformers/trainer_tf.py b/src/transformers/trainer_tf.py
--- a/src/transformers/trainer_tf.py
+++ b/src/transformers/trainer_tf.py
@@ -19,7 +19,7 @@
 
 from .modeling_tf_utils import TFPreTrainedModel
 from .optimization_tf import GradientAccumulator, create_optimizer
-from .trainer_utils import PREFIX_CHECKPOINT_DIR, EvalPrediction, PredictionOutput, set_seed
+from .trainer_utils import PREFIX_CHECKPOINT_DIR, EvalPrediction, EvaluationStrategy, PredictionOutput, set_seed
 from .training_args_tf import TFTrainingArguments
 from .utils import logging
 
@@ -561,7 +561,7 @@ def train(self) -> None:
 
                     if (
                         self.args.eval_steps > 0
-                        and self.args.evaluate_during_training
+                        and self.args.evaluate_strategy == EvaluationStrategy.STEPS
                         and self.global_step % self.args.eval_steps == 0
                     ):
                         self.evaluate()
diff --git a/src/transformers/training_args_tf.py b/src/transformers/training_args_tf.py
--- a/src/transformers/training_args_tf.py
+++ b/src/transformers/training_args_tf.py
@@ -34,8 +34,12 @@ class TFTrainingArguments(TrainingArguments):
             Whether to run evaluation on the dev set or not.
         do_predict (:obj:`bool`, `optional`, defaults to :obj:`False`):
             Whether to run predictions on the test set or not.
-        evaluate_during_training (:obj:`bool`, `optional`, defaults to :obj:`False`):
-            Whether to run evaluation during training at each logging step or not.
+        evaluation_strategy (:obj:`str` or :class:`~transformers.trainer_utils.EvaluationStrategy`, `optional`, defaults to :obj:`"no"`):
+            The evaluation strategy to adopt during training. Possible values are:
+
+                * :obj:`"no"`: No evaluation is done during training.
+                * :obj:`"steps"`: Evaluation is done (and logged) every :obj:`eval_steps`.
+
         per_device_train_batch_size (:obj:`int`, `optional`, defaults to 8):
             The batch size per GPU/TPU core/CPU for training.
         per_device_eval_batch_size (:obj:`int`, `optional`, defaults to 8):


Test patch: