Add the learning rate (in exponential representation, like "9.197948717948718e-08") to the model training output
Author: artyomboyko
Pull request: no
Labels: name: Feature request

### Feature request

Good day. Can you add current learning rate to output of training process. 
Now, if we train model with Trainer we get those output (without leraning rate):

<img width="302" alt="image" src="https://github.com/huggingface/transformers/assets/17560478/239c3ce1-c7dc-4d79-9664-52ac4e3ea605">


I'm using this code to add information about the current learning rate value to the output:

```python
from transformers import Trainer, Seq2SeqTrainer
from typing import Any, Dict, List, Union

class MyTrainer(Seq2SeqTrainer):
    def log(self, logs: Dict[str, float]) -> None:
        logs["learning_rate"] = self._get_learning_rate()
        print(logs["learning_rate"])
        super().log(logs)
        

trainer = MyTrainer(
    args=training_args,
    model=model,
    train_dataset=common_voice["train"],
    eval_dataset=common_voice["test"],
    data_collator=data_collator,
    compute_metrics=compute_metrics,
    tokenizer=processor.feature_extractor,
)

trainer.train()
```

This causes the learning rate values to appear in the tabular output during model training, as here:

<img width="536" alt="image" src="https://github.com/huggingface/transformers/assets/17560478/defa1a31-15a5-4994-905e-7afdccee76de">


But the latest version of the library doesn't have enough field width to display this parameter correctly.

### Motivation

Currently, when using the library, the learning rate is not displayed. It can only be viewed on the hub. But if the library is used locally without internet access, there is no way to track the current learning rate during fine-tuning/training of the model.

### Your contribution

Gave a succinct example of the sample code in "Feature request" field that I use to output a learning rate value.

Comments:
Hey! I think the way you log it is nice, just needs the log to be in scientific format! 

@ArthurZucker I already try convert `float` to scientific notation by using:
```python
...
    logs["learning_rate"] = "{:.2e}".format(self._get_learning_rate())
...
```

It dosent work. Tensorboard `add_scalar()` din't support this.

@ArthurZucker Anyway, it would be great if this was implemented in the library itself. Then there would be no need to make "crutches" to output this information.

feel free to open a PR and ping @muellerzr ! 

@ArthurZucker Maybe you know someone on the HuggingFace team who could help solve the "Rate" field width issue in the output?

@muellerzr Testing in progress...)))

<img width="605" alt="image" src="https://github.com/huggingface/transformers/assets/17560478/478bb935-2474-4c91-96a6-5ae470fdf396">


@muellerzr 

Tested it 3 times.
First time - just installed the library.
Second time - restarted the kernel in notepad. Repeated the test.
Third time - restarted WSL2, to be more sure that the problem is reproduced after a reboot.

The result in all three cases:

<img width="560" alt="image" src="https://github.com/huggingface/transformers/assets/17560478/356d13d1-69f3-4b91-8415-96b5fbc1db9e">


The code used in the tests*:
```python
training_args = Seq2SeqTrainingArguments(
    output_dir = result_model_name,                       # измените имя репозитория по своему усмотрению
    per_device_train_batch_size=train_batch_size,
    gradient_accumulation_steps=accumulation_steps,       # увеличивается в 2x раза при каждом уменьшении размера батча в 2x раза  
    learning_rate=lr,
    lr_scheduler_type=lr_scheduler,                           # Планировщик для learning_rate
    warmup_steps=250,
    max_steps=training_steps_count,
    gradient_checkpointing=True,
    fp16=True,
    evaluation_strategy="steps",
    optim = "adamw_torch",
    per_device_eval_batch_size=eval_batch_size,        # Самый простой способ задать равным параметру per_device_train_batch_size
    predict_with_generate=True,
    generation_max_length=225,
    save_steps=500,
    eval_steps=500,
    logging_steps=25,
    report_to=["tensorboard"],
    load_best_model_at_end=True,
    metric_for_best_model=selected_metric,
    greater_is_better=False,
    push_to_hub=True,
    hub_private_repo=True
)


# class MyTrainer(Seq2SeqTrainer):
#     def log(self, logs: Dict[str, float]) -> None:
#         logs["learning_rate"] = self._get_learning_rate() * 1_000_000  # Умножение необходимо для исправления отображения. 
#         print(logs["learning_rate"])
#         super().log(logs)



# trainer = MyTrainer(
#     args=training_args,
#     model=model,
#     train_dataset=common_voice["train"],
#     eval_dataset=common_voice["test"],
#     data_collator=data_collator,
#     compute_metrics=compute_metrics,
#     tokenizer=processor.feature_extractor,
# )


trainer = Seq2SeqTrainer(
    args=training_args,
    model=model,
    train_dataset=common_voice["train"],
    eval_dataset=common_voice["test"],
    data_collator=data_collator,
    compute_metrics=compute_metrics,
    tokenizer=processor.feature_extractor,
)
```
(*) - Naturally, the commented lines were not executed.

@muellerzr The learning rate is duplicated in scientific notation and decimalization for some unknown reason.

I’m still actively working on it, but your excitement is appreciated :)

@muellerzr If additional testing is required. Tag me. Thanks again for your responsiveness.

@ArthurZucker @muellerzr Good afternoon. I think you might be interested to know. Thanks to the feature you are developing, I was able to fine-tune Whisper small to WER = 12.6%.

https://huggingface.co/artyomboyko/whisper-small-fine_tuned-ru-v2
https://huggingface.co/spaces/artyomboyko/Whisper-medium-ru-v2

@muellerzr Good day. Will this enhancement be implemented?