try eval befor train gives ValueError with deepspeed Zero2
Author: Chandler-Bing
Pull request: no
Labels: name: trainer

name: DeepSpeed

### System Info

```
- `transformers` version: 4.39.0
- Platform: Linux-3.10.0-1127.19.1.el7.x86_64-x86_64-with-glibc2.31
- Python version: 3.10.13
- Huggingface_hub version: 0.20.2
- Safetensors version: 0.4.1
- Accelerate version: 0.26.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.1.2 (True)
- Tensorflow version (GPU?): 2.16.1 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

```

### Who can help?

@pacman100 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Hi @pacman100 , I want model to evaluate before my train process, so I add` trainer.evaluate()` before `trainer.train()`  as  [sgugger](https://discuss.huggingface.co/t/how-to-evaluate-before-first-training-step/18838/2?u=chandler-m) suggests.
however my run script is still `deepspeed run.py --args` ,  it triggered a `ValueError("ZeRO inference only makes sense with ZeRO Stage 3 - please adjust your config") `[deepspeed_init](https://github.com/huggingface/transformers/blob/v4.39.1/src/transformers/integrations/deepspeed.py#L397)


I tried to comment the code here [ _, _ = deepspeed_init] (https://github.com/huggingface/transformers/blob/v4.39.1/src/transformers/trainer.py#L3480), the evaluate works fine, but  in `trainer.train()`, it gives me
```
Traceback (most recent call last):
  File "/app/nfs_share_dir/5/boruipeng/finance_llm/pretraining.py", line 325, in <module>
    main()
  File "/app/nfs_share_dir/5/boruipeng/finance_llm/pretraining.py", line 297, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 1780, in train
    return inner_training_loop(
  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 2118, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs)
  File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 3045, in training_step
    self.accelerator.backward(loss)
  File "/opt/conda/lib/python3.10/site-packages/accelerate/accelerator.py", line 1958, in backward
    self.deepspeed_engine_wrapped.backward(loss, **kwargs)
AttributeError: 'NoneType' object has no attribute 'backward'
```
ops... seems not a feasible solution...

why would this happen? **how should I do if I want my model evaluate before train with Zero-2**?


Thank u

### Expected behavior

successfully eval before train

Comments:
Gentle ping @pacman100 

cc @SunMarc @muellerzr 

Hi @Chandler-Bing, we have added `eval_on_start` arg recently on the main branch of transformers. Could you try that ? 

> Hi @Chandler-Bing, we have added `eval_on_start` arg recently on the main branch of transformers. Could you try that ?

Hi @SunMarc , problem solved! things works as expected. hope release soon
thanks😄 


Wow, glad it worked !