[Trainer/Deepspeed] handle get_last_lr() before first step()
Author: stas00
Pull request: yes

with deepspeed's fp16 and dynamic loss scale enabled the optimizer/scheduler steps may not run for the first few dozens steps while loss is overflowing, so `get_last_lr` will fail if called during that warm up stage, so this PR tries to catch that special warm situation and handle it by returning a fake LR=0, which is a good default because since there is no stepping it's effectively a 0.

I'm just not sure if I should warn - it ends up emitting like 20-30 of those: if the user picks `--logging_steps=`, e.g:

```
2021-02-23 12:53:06,798] [INFO] [stage2.py:1357:step] [deepscale] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 4294967296, reducing to 4294967296
[WARNING|trainer.py:1142] 2021-02-23 12:53:06,799 >> tried to get lr value before scheduler/optimizer started stepping, returning lr=0
{'loss': 11.0, 'learning_rate': 0, 'epoch': 0.0}                                                                                                                 
[2021-02-23 12:53:06,990] [INFO] [stage2.py:1357:step] [deepscale] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 4294967296, reducing to 2147483648.0
[WARNING|trainer.py:1142] 2021-02-23 12:53:06,992 >> tried to get lr value before scheduler/optimizer started stepping, returning lr=0
{'loss': 10.9922, 'learning_rate': 0, 'epoch': 0.0}                                                                                                              
                 
```

* [x] added a test too.

I first thought it should be handled by DeepSpeed https://github.com/microsoft/DeepSpeed/issues/782 but then realized that since pytorch optimizers won't be aware of this, we have to handle this in the trainer since we are the ones calling `get_last_lr()` sort of prematurely - (yet, we don't have a way to know that it's premature as we can't even called `lr_scheduler.step()` as it's being handled opaquely by DeepSpeed.

@sgugger 

Fixes: #https://github.com/huggingface/transformers/issues/10330#issuecomment-784457460

Comments:
Yes, that's a good idea. I'm just not clear on whether you suggest to make a function just for the deepspeed segment of the branch or wrap up the whole getting lr function?

Plus, the code needs the `trainer` object, so I'm not sure how to put it in utils. 


I propose to put it as a separate Trainer method instead. 
```
logs["learning_rate"] = self._get_learning_rate()
```

Please check if the proposed change in the next commit looks good to you - made it into a method and put it at the end of the file so it's out of the way.


> Yes, that's a good idea. I'm just not clear on whether you suggest to make a function just for the deepspeed segment of the branch or wrap up the whole getting lr function?

The whole thing, as you did.

> Plus, the code needs the trainer object, so I'm not sure how to put it in utils.

It doesn't need the whole Trainer, just the `lr_scheduler` and the `args` (to detect if deepspeed is activated).

Can someone explain to me why I am suscribed to this

On Tue, Feb 23, 2021, 7:43 PM Stas Bekman <notifications@github.com> wrote:

> Merged #10362 <https://github.com/huggingface/transformers/pull/10362>
> into master.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/pull/10362#event-4368294070>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AS5YU5U6O456ILN65VM72ELTARKSPANCNFSM4YDGTCNA>
> .
>


Hi @chrissyjsartt 

We won't know, since only you can do it. Perhaps you hit [Subscribe] by mistake?

But if you pay close attention to the email you received it tells you how to unsubscribe at the end of it:

> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/pull/10362#event-4368294070>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AS5YU5U6O456ILN65VM72ELTARKSPANCNFSM4YDGTCNA>