4.47.1 Hugging Face Trainer loss accumulated by sum instead of mean
Author: jdf-prog
Pull request: no
Labels: name: bug

### System Info

Version: `transformers==4.47.1`

When I training model using this version of hugging face trainer, the loss is accumulated by sum instead of mean. Or it's better saying that the `tr_loss_step` did not divide by the global batch size. That means, the reported loss will scale proportionally with the global batch size. 

![image](https://github.com/user-attachments/assets/55c0ca9a-c4b0-400b-9059-9b4fc9686810)

When I change the transformer version to be 4.44.0, this problem is and everything works good.
![image](https://github.com/user-attachments/assets/144b01d5-e133-49cc-b5e5-bd5267deef2a)

My global batch size is set to 128 here. You can see from the above image that `tr_loss_step=original_loss / 128`, which is not the case when transformers version is 4.47.1, where  `tr_loss_step=original_loss`.





### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Any training with `transformers==4.47.1`

### Expected behavior

loss accumulated by sum instead of mean

Comments:
cc @SunMarc @muellerzr 

Can you try with the main branch of transformers, we fixed a couple of issue grad accumulation, so your issue might be solved. Otherwise, can you share a minimal reproducer ? Thanks ! 

> Can you try with the main branch of transformers, we fixed a couple of issue grad accumulation, so your issue might be solved. Otherwise, can you share a minimal reproducer ? Thanks !

A simple reproduce would be using llamafactory:
```
git clone https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory
pip install -e .
llamafactory-cli train examples/train_lora/llama3_lora_dpo.yaml
```
And test for `transformers==4.47.1` and `transformers<=4.45.2`, where the former should has higher loss than the later.

However, I cannot test `transformers==4.48.0` on the main branch since there seems a lot of change in the modeling_llama file. 

I also encountered the same problem in this version, when adjusting gradient_accumulation_steps, the loss scales accordingly

I also encountered this problem, but I found that for me, it doesn't happen with all the models. It happens with "facebook/opt-350m" and "facebook/opt-1.3b", but not with "TinyLlama/TinyLlama_v1.1". The difference may be that opt is in 16 bits while TinyLlama in 32 bits. Using a Titan RTX.

I updated to 4.48.0 and the issue seems to be corrected for opt, but not for other models like "projecte-aina/FLOR-760M".

I think the bug is going to be fixed after https://github.com/huggingface/transformers/pull/35651 merged.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.