[traner] fix --lr_scheduler_type choices
Author: stas00
Pull request: yes

This PR fixes:
```
$ python ./run_clm.py -h | grep lr_scheduler_type
                  [--lr_scheduler_type {SchedulerType.LINEAR,SchedulerType.COSINE,SchedulerType.COSINE_WITH_RESTARTS,SchedulerType.POLYNOMIAL,SchedulerType.CONSTANT,SchedulerType.CONSTANT_WITH_WARMUP}]
```
to:
```
                  [--lr_scheduler_type {linear,cosine,cosine_with_restarts,polynomial,constant,constant_with_warmup}]
```

I'm not sure what the original intention was since the current suggestions do not work:
```
run_clm.py: error: argument --lr_scheduler_type: invalid SchedulerType value: 'SchedulerType.LINEAR'
```

I couldn't find any readily-available methods to do the same in the `enum` superclass: https://docs.python.org/3/library/enum.html

@sgugger 

Comments:
This fix works, but only for this particular enum. I'm wondering if it shouldn't be better to just change [this line](https://github.com/huggingface/transformers/blob/master/src/transformers/hf_argparser.py#L83) in `HFArgumentParser`? For instance
```bash
$ python examples/text-classification/run_glue.py -h | grep evaluation_strategy
```

returns
```
--evaluation_strategy {EvaluationStrategy.NO,EvaluationStrategy.STEPS,EvaluationStrategy.EPOCH}
```

which probably does not work as well. (Or if does, we want to display "no"/"steps"/"epoch" here.)

That's an excellent suggestion, @sgugger - thank you! Please have a look at this variation.

Weird, somehow it broke `finetune_trainer.py`:

```
examples/seq2seq/test_finetune_trainer.py:84: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
examples/seq2seq/test_finetune_trainer.py:76: in finetune_trainer_quick
    output_dir = self.run_trainer(1, "12", MBART_TINY, 1, distributed, deepspeed, extra_args_str)
examples/seq2seq/test_finetune_trainer.py:210: in run_trainer
    main()
examples/seq2seq/finetune_trainer.py:160: in main
    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
src/transformers/hf_argparser.py:150: in parse_args_into_dataclasses
    namespace, remaining_args = self.parse_known_args(args=args)
/usr/local/lib/python3.6/argparse.py:1773: in parse_known_args
    self.error(str(err))
/usr/local/lib/python3.6/argparse.py:2393: in error
    self.exit(2, _('%(prog)s: error: %(message)s\n') % args)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = HfArgumentParser(prog='finetune_trainer.py', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)
status = 2
message = "finetune_trainer.py: error: argument --evaluation_strategy: invalid choice: <EvaluationStrategy.STEPS: 'steps'> (choose from 'no', 'steps', 'epoch')\n"

```

The test is just passing `--evaluation_strategy steps`


Ok, tested locally and doing this does not work indeed (e.g. you can't launch any script with `--evaluation_strategy steps` or `--lr_scheduler_type linear`). To have it work, we have to change the type of the enum to `str`, but then the actual values of the dataclass are strings.

So there is no easy solution. I'm fine with leaving as is or also to remove the choices and expand the help to show the actual possible values, but it looks like it won't work as is.

Thank you for validating it.

How about moving my initial PR's `get_arg_names` to `ExplicitEnum` so any sub-class has access to it - perhaps a different name? And then we change just `meta["choices"]` in the arg parser to get these values?

No the initial PR doesn't work either (this is not caught by the tests since the test do not use `--lr_scheduler_type` in any of the example scripts). The field ends up being a `str` if you try on your side (and not a `SchedulerType` despite the cast in the post_init so then all tests comparing `self.args.lr_scheduler_type` to `SchedulerType.XXX` will fail.

Ah, my bad! I was testing on the `parse` method of `HfArgumentParser`, not `parse_into_dataclasses`. There is a way to make this work :-)

I'm all ears.



The easy fix is to force `kwargs["type"] = type(kwargs["choices"][0])` for the Enum subclasses, after your line `kwargs["choices"] = [x.value for x in field.type]`, and let the dataclass set them back to their proper enum types in the postinit (as is done right now).

I even have a function that will automagically do the casting back after the init, which is the following:
```
    for dtype in self.dataclass_types:
            keys = {f.name for f in dataclasses.fields(dtype) if f.init}
            keys_to_enum_types = {f.name: f.type for f in dataclasses.fields(dtype) if isinstance(f.type, type) and issubclass(f.type, Enum)}
            inputs = {k: v for k, v in vars(namespace).items() if k in keys}
            for k in keys:
                if k in keys_to_enum_types:
                    inputs[k] = keys_to_enum_types[k](inputs[k])
                delattr(namespace, k)
            obj = dtype(**inputs)
            outputs.append(obj)
```
in the `parse_into_dataclasses` method. The same would need to be done in the other special parse methods for consistency.

Please feel free to take over, rather than me being the middle person as you know what you're doing and I will learn from your work when it's done. Thank you!

Fabulous! 

This is not a user facing code, correct? There are 3 pretty big identical chunks of code which can be refactored then.

Not sure we actually need them since the dataclasses need to re-cast those args to the enum type anyway for when someone is using `TrainingArguments` not in CLI. I was hoping to remove the lines
```
self.evaluation_strategy = EvaluationStrategy(self.evaluation_strategy)
self.lr_scheduler_type = SchedulerType(self.lr_scheduler_type)
```
by using those four lines, but they are still necessary. So we'll probably just remove those three blocks of identical code (let's see what @LysandreJik thinks!)