examples/seq2seq/finetune_trainer.py: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`.
Author: forest1988
Pull request: no

## Environment info
     
- `transformers` version: 3.3.1
   - I did `pip install -e .` in the repository cloned from
https://github.com/huggingface/transformers/tree/ba4bbd92bcb55febbfa06aaa1551738388ec7eb0
- Platform: Linux
- Python version: 3.8.3 (anaconda3-2020.07)
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
 examples/seq2seq: @sshleifer

## Information

Model I am using (Bert, XLNet ...): Bart (facebook/bart-base)

The problem arises when using:
* [x] the official example scripts: (give details below)

The tasks I am working on is:
* [x] an official XSUM summarization task

## To reproduce

Running examples/seq2seq/finetune_trainer.py as below.

```sh
$ CUDA_VISIBLE_DEVICES=3 python finetune_trainer.py \
    --learning_rate=3e-5 \
    --fp16 \
    --do_train --do_eval --do_predict --evaluate_during_training \
    --predict_with_generate \
    --n_val 1000 \
    --model_name_or_path facebook/bart-base \
    --data_dir ********/xsum/ \
    --output_dir ******** \
    2>&1 | tee test.log
```

Then, I get an UserWarning  message:

```sh
0%|          | 0/76506 [00:00<?, ?it/s]/********/.pyenv/versions/anaconda3-2020.07/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:118: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  warnings.warn("Detected call of `lr_scheduler.step()` before `optimizer.step()`. "
```

## Expected behavior

It's an UserWarning, not an Error. It may be not critical, but I want to know why this is caused. 
I'm sorry if the "Bug Report" isn't the right choice for this issue.

I'm new to Transformers Trainer. I apologize that I can't distinguish whether this UserWarning belongs to examples/seq2seq/ or Trainer itself.

Thank you in advance.

Comments:
This happens in every trainer I've ever used and you should just ignore it. It's happening in pytorch so we cant control the type of error.

cc @sgugger who may have a different opinion.

This is a warning that appears all the time from PyTofh if you save your learning rate scheduler. You should file an issue on their side if you find it annoying.
Normally, the latest version of transformers should catch that warning from you though.

@sshleifer @sgugger 
Thank you for quickly answering my question!

I apologize that I misunderstood this UserWarning as to be caused by your codes.
Thanks to your kind explanations, I now understand that this is caused not by examples/seq2seq and transformers Trainer, but by PyTorch. 
I also understand that I will come across the same UserWarning all the time if I save the learning rate scheduler.

I'm relieved to hear that I can ignore it if I don't find it annoying.

Thanks again!