Save last/best model during training
Author: 2533245542
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.20.1
- Platform: Linux-5.8.0-45-generic-x86_64-with-glibc2.17
- Python version: 3.8.13
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.12.0+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

@sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

trainer_arg = TrainerArguments(
    save_strategy='steps'
    evaluation_strategy='steps'

    max_steps=50
    eval_steps=5
    save_steps=10
    save_total_limit=2

    load_best_model_at_end=true
)

### Expected behavior


Saving the best/last model in the trainer is confusing to me, even after reading these two posts, and it would be helpful if @sgugger , the expert of trainer can clarify me. 

https://stackoverflow.com/questions/62525680/save-only-best-weights-with-huggingface-transformers
https://discuss.huggingface.co/t/save-only-best-model-in-trainer/8442

A few questions

## Question 1
Below is the input to TrainerArgument. This trains the model for 50 steps, does evaluation every 5 steps (10 evaluation), and saves model every 10 steps (5 saves, for now)

But because save_total_limit is 2, only the 2 most recent models will be saved (this looks like what it should behave from plain text, but it seems to save the best and the most recent one in this case, as I read from the posts? Would like to hear some explanation about that. And what if the best model happens to be the last model, which 2 models will be saved then?)

```
save_strategy='steps'
evaluation_strategy='steps'

max_steps=50
eval_steps=5
save_steps=10
save_total_limit=2

load_best_model_at_end=true
```
## Question 1.1
Then what would happen if I replace the above with
load_best_model_at_end=false

Will it change any of the behavior regarding to saving the model?
## Question 1.2
And another question, what would happen when setting save_total_limit=1? Looks like it will just save the best model? Internally, during training, is it like checking if the current saved model is the best one, if so, do nothing, else replace it with the current best trained model?

## Question 2
The below is essentially saying in order to find the best model, there needs to be a score for each saved model, so the save_step has to be a multiple of eval_step, right?

<img width="870" alt="image" src="https://user-images.githubusercontent.com/28517073/190272027-4dba218d-3561-4c1f-be5a-9f6bcbf9eb82.png">
 

Comments:
`save_total_limit` will control the number of checkpoints being saved, so with `save_total_limit=2`:
- when `load_best_model_at_end=True`, you have the best model and the last model (unless the last model is the best model in which case you have the two last models)
- when `load_best_model_at_end=False`, you have the last two models

The only exception is when `save_total_limit=1` and `load_best_model_at_end=True` where we always keep the best model and the last model (to be able to resume training if something happens), so in this case there might be two models saved.

Question 2 is a paraphrase of the green block ;-)


> `save_total_limit` will control the number of checkpoints being saved, so with `save_total_limit=2`:
> 
>     * when `load_best_model_at_end=True`, you have the best model and the last model (unless the last model is the best model in which case you have the two last models)
> 
>     * when `load_best_model_at_end=False`, you have the last two models
> 
> 
> The only exception is when `save_total_limit=1` and `load_best_model_at_end=True` where we always keep the best model and the last model (to be able to resume training if something happens), so in this case there might be two models saved.
> 
> Question 2 is a paraphrase of the green block ;-)

Say I set load_best_model_at_end=True and save_total_limit=1. I then finished a model pretraining. 

Now my folder contains 2 saved models, `checkpoint-1000` and `checkpoint-900`. I know the 1000 one is the last running one, but is there a way to confirm whether the former or the later is the best one?



Trust what I just said? 😝 
The name of the best checkpoint is also saved in the trainer state normally. If you inspect it you should get confirmation.

in the newest version(now is 2023-02), just set load_best_model_at_end=True & metric_for_best_model=**(eg, accuracy) & greater_is_better=True(or False if eval loss metric)，then，the trainer will not delete the best model when rotate the checkpoint.

"Hello, do you have any idea of how to load the best model? For example, if I do:

python

peft_training_args = TrainingArguments(
    output_dir=output_dir,
    auto_find_batch_size=True,
    save_strategy="no",
    metric_for_best_model="loss",
    greater_is_better=False,
    save_total_limit=2,
    load_best_model_at_end=True,
    learning_rate=1e-2,
    logging_steps=1,
    max_steps=10
)

And after I save it like this:

peft_trainer.model.save_pretrained(peft_model_path)

Do I have the best model?"

thanks in advance.