Possible bug when using cosine lr scheduler with gradient accumulation
Author: hanyin88
Pull request: no
Labels: name: bug

### System Info

transforme 4.47
deepspeed

### Who can help?

@muellerzr @SunMarc

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

[UPDATE} Turns out the issue is that I need to explicitly set "gradient_accumulation_steps": "auto" in the deepspeed config. Initially in my config I set most things to "auto" but I did not set gradient_accumulation_steps explictly, assuming it will be treated the same way as auto, otherwise will report error. FYI, would appreciate any good resource on what other parameters should be set as "auto" in deepspeed config.... I have not found a good reference. Thanks!


### Reproduction
I am running a trianing script with deepspeed as 
```
deepspeed training.py
```

When using official trainer with gradient_accumulation_steps = 2 and lr_scheduler_type = cosine, I got following learning rate curve (training epoch = 1):

![image](https://github.com/user-attachments/assets/f86f250a-9567-49a9-96c1-272f10247771)

So seems the scheduler stepped more than one cycle. When digging through the source code, seems the issue is that in:
```
def _get_cosine_schedule_with_warmup_lr_lambda(
    current_step: int, *, num_warmup_steps: int, num_training_steps: int, num_cycles: float, min_lr_rate: float = 0.0
):
    if current_step < num_warmup_steps:
        return float(current_step) / float(max(1, num_warmup_steps))
    progress = float(current_step - num_warmup_steps) / float(max(1, num_training_steps - num_warmup_steps))
    factor = 0.5 * (1.0 + math.cos(math.pi * float(num_cycles) * 2.0 * progress))
    factor = factor * (1 - min_lr_rate) + min_lr_rate
    return max(0, factor)
```
num_training_steps is 
```
num_update_steps_per_epoch = len_dataloader // args.gradient_accumulation_steps
max_steps = math.ceil(args.num_train_epochs * num_update_steps_per_epoch)
self.create_optimizer_and_scheduler(num_training_steps=max_steps)
self.create_scheduler(num_training_steps=num_training_steps, optimizer=optimizer)
```


Therefore current_step is the global step without taking into account for gradient_accumulation, while num_training_steps did account for gradient_accumulation. As an example, if we train 1000 cases on one GPU for 1 epoch with a accumulation step of 2, the num_training_steps is 500, but current_step can go up to 1000.

Wondering if this is expected behavior or something that should be fixed?

Many thanks.


### Expected behavior

![image](https://github.com/user-attachments/assets/0199dec9-cebf-4779-91e2-800bdfaa6327)


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I think this might be related to the gradient accum fixes that are already in? @cc muellerzr @sunmarc

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

*loud cough*

please ignore me ccing the user whose name was cc, and actual cc @muellerzr @sunmarc

There is an update in the post =) 

> [UPDATE} Turns out the issue is that I need to explicitly set "gradient_accumulation_steps": "auto" in the deepspeed config. Initially in my config I set most things to "auto" but I did not set gradient_accumulation_steps explictly, assuming it will be treated the same way as auto, otherwise will report error. FYI, would appreciate any good resource on what other parameters should be set as "auto" in deepspeed config.... I have not found a good reference. Thanks!

If there is a mismatch in the config between `deepspeed_config` and `trainer_config`, it should raise an error actually. See `trainer_config_process` function . Could you share a reproducer @hanyin88, that will be very helpful for us !  

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.