Disable loss rounding in training stats log
Author: AnMakc
Pull request: yes

This PR disables unconditional rounding of the loss value to 4 digits in logged values. Effectively reverting part of https://github.com/huggingface/transformers/pull/9491.

Closes https://github.com/huggingface/transformers/issues/38032 (stale)

Motivation:
- Loss value may have very low absolute magnitude depending on model/data/objective function.
- No other values are rounded (e.g. eval/loss or grad_norm) which may result in confusion.
- Rounding is a display/formatting detail in this case and should not be done silently in non-UI code

### Examples of undesired behavior that would be fixed with this PR:
Notebook outputs: 
<img width="282" height="281" alt="Screenshot 2025-11-07 at 16 06 21" src="https://github.com/user-attachments/assets/adcd0245-d097-44d6-8582-8372f83634b8" />

Wandb loss chart:
<img width="405" height="241" alt="Screenshot 2025-11-07 at 15 42 45" src="https://github.com/user-attachments/assets/e15103a2-935e-4885-a61f-843e309b1c00" />



Comments:
cc @sunmarc, but maybe formatting using something like `%.4f` so that it always displays 4 significant figures with an optional exponent would preserve the clean rendering but still work in cases when the loss is very small?

@Rocketknight1 Any rounding at this point introduces inconsistency between validation and training loss. We could not apply formatting here as we log numeric value, not string.

The main point is that any formatting is a representation detail that should be handled by the display-related code and we should not make assumptions on how it works as in most cases it is some external handler (wandb, trackio, ipython NB widget, etc.).

@Rocketknight1 @SunMarc

Please, take a look

This use case didn't exist when this was originally coded. The goal was to avoid a pointless output like `loss=1.3982828288737373`

I wonder if using a scientific notation can address this use-case while not making it an ugly experience for all other use cases. `loss=2.5444e-4` for the example above. Same 4 decimals, except the exponent overcomes the issue of lost precision in the original.


> I wonder if using a scientific notation can address this use-case

At this point these are still numeric values so we can't apply formatting, see https://github.com/huggingface/transformers/pull/42104#issuecomment-3511663788

Apply at the point of print then, while saving the full precision in the log file?

> Apply at the point of print then, while saving the full precision in the log file?

There are multiple ways of how the values could be displayed - jupyter notebooks, reporting to wandb, trackio, etc. Each handles display rounding separately. See attached screenshots in the first message here. 

This PR makes this behaviour explicit and consistent across different type of values/metrics. If there is a benefit of rounding the metric in specific logs - it could be added in a separate PR.

Whatever works. To make it clear - I wasn't objecting to your PR, was just suggesting on possible further work to keep things neat for multiple use-cases since I was tagged as the originator of this "feature".

Sure, just clarified the intention. Thanks!

@SunMarc Could you merge please?


To not have regression for other users, maybe we can implement @Rocketknight1 suggestions in `PrinterCallback` class that we have in transformers @AnMakc ? It should be quite straightforward.

> maybe formatting using something like %.4f so that it always displays 4 significant figures with an optional exponent would preserve the clean rendering but still work in cases when the loss is very small?

Thanks a lot @stas00 for the suggestion 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42104). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@SunMarc 

Updated the PR to format metrics in console with 4 significant digits.
Python default is to use scientific notation for values <1e-4 which seems sane threshold.

Example output: 
`{'loss': '0.0002014', 'grad_norm': '0.0001294', 'learning_rate': '2.179e-07', 'epoch': '0.0003269'}`

Note - epoch formatted the same way for consistency.

that looks great, except `'epoch': '0.0003269'` which is not a sensible format when facing a human reader, IMHO.

Should it report the percentage of epoch progress? `0.03%` maybe?



Percentage would not work for second+ epoch.

why not? `135.03%` unless we don't have the knowledge about which epoch is running?

But please feel free to ignore my suggestions if the current maintainers are ok with your last proposal. They decide.

I think we can keep this for now. If ppl complains about that, we can still do follow-up pr for specific metrics