[DeepSpeed] decouple `DeepSpeedConfigHF` from `Trainer`
Author: stas00
Pull request: yes

As requested in https://github.com/huggingface/transformers/issues/11954 this PR 

* uncouples `DeepSpeedConfigHF` from `Trainer` so one can activate `zero.Init()` in `modeling_utils.py` and several other places w/o needing to rely on the HF `Trainer`.
* adds a new `LoggingLevel` ctx manager to `testing_utils.py`
* adds a new test testing  `DeepSpeedConfigHF` decoupled from the `Trainer`.
* starts a new doc

* well, through the PR things got renamed too, see the final diff for all the changes.

The plan is to merge this, then move all of the Deepspeed integration code into its own `src/transformers/deepspeed.py` and the docs into `docs/source/main_classes/deepspeed.rst`, since the integration has now outgrown the Trainer alone.

Fixes: https://github.com/huggingface/transformers/issues/11954

@sgugger 

Comments:
Thank you for the great feedback and suggestions, Sylvain.

On to the next PR to move the code and docs.

@stas00 I have a question about this PR. My use case is HF trainer + DeepSpeed + hyperparameter search.

In **training_arg.py**: hf_deepspeed_config is first loaded from config file (e.g. zero3.config) as HfTrainerDeepSpeedConfig, and then is adjusted with TrainingArguments values. In particular, all "auto" in zero3.config would be resolved to actual integral or floating values.

            self.hf_deepspeed_config = HfTrainerDeepSpeedConfig(self.deepspeed)
            self.hf_deepspeed_config.trainer_config_process(self)

In **trainer.py -> _hp_search_setup()**: hf_deepspeed_config is reset by zero3.config to HfDeepSpeedConfig, different from HfTrainerDeepSpeedConfig, and thus "auto" values are remained unchanged as string type, not integral or floating type. 

            from transformers.deepspeed import HfDeepSpeedConfig
            self.args.hf_deepspeed_config = HfDeepSpeedConfig(self.args.deepspeed)

This didn't work for hyperparameter search because DS cannot be initialized with "auto" values. The error messages are attached below. I feel  _hp_search_setup() should do exact the same thing as in training_args.py to reset hf_deepspeed_config as HfTrainerDeepSpeedConfig and resolve all "auto" values, but I am not sure what was the reason you chose the other way in this PR.

```
  File "/home/meiyang/src/transformers_fork/src/transformers/integrations.py", line 164, in run_hp_search_optuna
    study.optimize(_objective, n_trials=n_trials, timeout=timeout, n_jobs=n_jobs)
  File "/home/meiyang/bin/miniconda3/envs/gptj/lib/python3.8/site-packages/optuna/study/study.py", line 400, in optimize
    _optimize(
  File "/home/meiyang/bin/miniconda3/envs/gptj/lib/python3.8/site-packages/optuna/study/_optimize.py", line 66, in _optimize
    _optimize_sequential(
  File "/home/meiyang/bin/miniconda3/envs/gptj/lib/python3.8/site-packages/optuna/study/_optimize.py", line 163, in _optimize_sequential
    trial = _run_trial(study, func, catch)
  File "/home/meiyang/bin/miniconda3/envs/gptj/lib/python3.8/site-packages/optuna/study/_optimize.py", line 264, in _run_trial
    raise func_err
  File "/home/meiyang/bin/miniconda3/envs/gptj/lib/python3.8/site-packages/optuna/study/_optimize.py", line 213, in _run_trial
    value_or_values = func(trial)
  File "/home/meiyang/src/transformers_fork/src/transformers/integrations.py", line 154, in _objective
    trainer.train(resume_from_checkpoint=checkpoint, trial=trial)
  File "/home/meiyang/src/transformers_fork/src/transformers/trainer.py", line 1155, in train
    self.model = self.call_model_init(trial)
  File "/home/meiyang/src/transformers_fork/src/transformers/trainer.py", line 1019, in call_model_init
    model = self.model_init()
  File "run_clm_local.py", line 289, in model_init
    pretrained_model = AutoModelForCausalLM.from_pretrained(
  File "/home/meiyang/src/transformers_fork/src/transformers/models/auto/auto_factory.py", line 447, in from_pretrained
    return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
  File "/home/meiyang/src/transformers_fork/src/transformers/modeling_utils.py", line 1488, in from_pretrained
    with deepspeed.zero.Init(config_dict_or_path=deepspeed_config()):
  File "/home/meiyang/src/deepspeed/deepspeed/runtime/zero/partition_parameters.py", line 461, in __init__
    _ds_config = DeepSpeedConfig(config_dict_or_path,
  File "/home/meiyang/src/deepspeed/deepspeed/runtime/config.py", line 873, in __init__
    self._configure_train_batch_size()
  File "/home/meiyang/src/deepspeed/deepspeed/runtime/config.py", line 1050, in _configure_train_batch_size
    self._batch_assertion()
  **File "/home/meiyang/src/deepspeed/deepspeed/runtime/config.py", line 986, in _batch_assertion
    train_batch > 0
TypeError: '>' not supported between instances of 'str' and 'int'**

```

Honestly, I have never used `_hp_search_setup()` and have no idea what it does, so it's very possible the DS integration doesn't support it at the moment as indicated by your report. 

Do you want to try and make it work and make a PR if you succeed? 

I'm not exactly sure what you mean by:

> but I am not sure what was the reason you chose the other way in this PR.

but perhaps it'd be much simpler for you to code what you think it should be and then we can look together at what you meant.

How does that sound?

Sure. I am working on it. After replacing to HFTrainerDeepSpeedConfig, I was able to pass DS initialization. But there are other issues for model init. Not sure if they’re DS related or not. Will continue looking tomorrow. 

Thank you for working on this, @dunalduck0 - I trust that you will figure it out.

While you work on this please log all the steps so that we can reproduce the process and we will need to create tests to verify the workings of this path based on your logs as we currently don't have any tests exercising this path with deepspeed. 

Hi stas00, I wanted to update this thread a bit. To my question on Mar 3rd, I think the change below is good enough to make sure DeepSpeed configuration is loaded properly.  . 
![image](https://user-images.githubusercontent.com/51274745/157412570-3e3a719d-9cdb-4245-84f1-058ddf69d9f1.png)

But the ultimate goal, to use hyperparameter search feature in Transformers with DeepSpeed, is still blocking (see [discussion thread](https://discuss.huggingface.co/t/using-hyperparameter-search-in-trainer/785/10) and [blog](https://huggingface.co/blog/ray-tune) to learn this feature). The difficulty is how to use an "used" DeepSpeed engine, as in the problems you addressed last month: https://github.com/microsoft/DeepSpeed/issues/1748

The difference in my case is I don't need to save/load_checkpoint. For hyperparameters search, both trainer and DeepSpeed engine need to be reused in multiple trial trainings, to discover the optimal settings. I was managed to run trial 1 (the 1st time the DeepSpeed engine is used in training) and then trial 2 failed inside of DeepSpeed with "index out of range" error. It looks like some partition went wrong, but I am not sure. I wonder if you have reached any good solution with DeepSpeed team on this type of issues?

Some improvements have been made recently in Deepspeed, but last I tried to re-use the engine it still didn't work in all situations. You're of course welcome to open and issue at Deepspeed and ask for a better engine reuse. If you don't ask other priorities will take over.

But I'm sure we can find a workaround with what we have.

Bottom line, I need to study how this feature works, write a simple example that deploys this feature, then add DS support, then turn it into a test.

Thank you for the links with the usage examples. 

I'm currently very busy with the BigScience 176B model training launch, but hopefully next week I should have some time to tinker with this. Unless of course, you or someone else beats me to it ;)

@dunalduck0, my apologies for taking forever to attend to this feature request. 

Please try this PR https://github.com/huggingface/transformers/pull/16740 and let me know if it addressed your need.

I only added a basic test, so if you're doing something specific that happens not to work please let me know and I will extend the test to include it.