If run trainer._maybe_log_save_evaluate() twice continuously, it will appear “ZeroDivisionError: float division by zero”
Author: niuzaisheng
Pull request: no

## Environment info

- `transformers` version: 4.3.3
- Platform: Linux-4.15.0-139-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.0
- PyTorch version (GPU?): 1.8.0+cu111 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Irrelevant
- Using distributed or parallel set-up in script?: no

### Who can help
Library:
- trainer: @sgugger

## Information

In `transformers/trainer.py`, if I use the function `trainer._maybe_log_save_evaluate() `continuously, `self.state.global_step - self._globalstep_last_logged` will be zero, so raise `ZeroDivisionError` exception in line 1044:
`logs["loss"] = round(tr_loss_scalar / (self.state.global_step - self._globalstep_last_logged), 4)`

This situation occurs when an epoch is finished but `_maybe_log_save_evaluate()` is called twice in line 983 and line 989 , waiting for next epoch.


Comments:
@niuzaisheng The function starting with '_' is a hint that this is an internal function and is not for external use. Is there any particular reason you are using this and not any alternatives?

Looking into the function `_maybe_log_save_evaluate` should ideally not contain line 1225 `self._globalstep_last_logged = self.state.global_step`. I can open a PR and rectify this but not sure if this is a necessary change. 
@sgugger Please comment.

Because my training ended after the first epoch and raised `ZeroDivisionError` exception, I looked inside the source code. I think this is the reason why my training cannot go on.

@niuzaisheng Can you provide sample code and the full stacktrace?

Sorry, I can't give out all my training script. But this problem appeared coincidentally. If `should_log ` just right at the end of an epoch,  the func `_maybe_log_save_evaluate` will be called twice continuously.

<img width="1280" alt="截屏2021-03-23 下午7 30 29" src="https://user-images.githubusercontent.com/29062892/112139872-429dca00-8c0e-11eb-82f5-d6c20b65bd0e.png">

here is my stacktrace:
```
100%|█████████▉| 18028/18030 [5:21:43<00:01,  1.49it/s]
100%|█████████▉| 18029/18030 [5:21:43<00:00,  1.52it/s]
100%|██████████| 18030/18030 [5:21:44<00:00,  1.62it/s]{'loss': 1.3667, 'learning_rate': 0.0, 'epoch': 10.0, 'step': 18030}
Traceback (most recent call last):
  File "/home/XXXX/anaconda3/envs/allennlp/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/XXXX/anaconda3/envs/allennlp/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/XXXX/XXXX/XXXX/run_train.py", line 297, in <module>
    main()
  File "/home/XXXX/XXXX/XXXX/run_train.py", line 257, in main
    model_path=model_args.name_or_path if os.path.isdir(model_args.name_or_path) else None
  File "/home/XXXX/anaconda3/envs/allennlp/lib/python3.7/site-packages/transformers/trainer.py", line 989, in train
    self._maybe_log_save_evaluate(tr_loss, model, trial, epoch)
  File "/home/XXXX/anaconda3/envs/allennlp/lib/python3.7/site-packages/transformers/trainer.py", line 1044, in _maybe_log_save_evaluate
    logs["loss"] = round(tr_loss_scalar / (self.state.global_step - self._globalstep_last_logged), 4)
ZeroDivisionError: float division by zero

100%|██████████| 18030/18030 [5:21:54<00:00,  1.07s/it]
```
My `logging_steps` is set to 10 steps. And 18030 is just at the end of an epoch, Coincidentally.

So, at the first time in line 983 call `_maybe_log_save_evaluate()`, ` self._globalstep_last_logged ` will be set equal to  `self.state.global_step` by line 1052.
At second time in line 989 call `_maybe_log_save_evaluate()` , `logs["loss"] = round(tr_loss_scalar / (self.state.global_step - self._globalstep_last_logged), 4)` will raise ZeroDivisionError in line 1044.

I can avoid this problem by modifying `logging_steps` to other numbers.

Got it. This should be rectified. A simple : 
`if (step + 1) != steps_in_epoch:
     self._maybe_log_save_evaluate(tr_loss, model, trial, epoch)`
in line 1138 should work?


If you want to avoid two consecutive calls `_maybe_log_save_evaluate`, we can do this, but will it affect `should_evaluate` and `should_save` in `_maybe_log_save_evaluate` ? If `evaluation_strategy` is set to be `epoch`, will it affect ?

As its name indicates `_maybe_log_save_evaluate` does not log at each epoch, it depends on the value of the `self.control.should_log` variable which won't always be `True`. Since your log strategy is either `"steps"` or `"epoch"` it won't run the line
```
logs["loss"] = round(tr_loss_scalar / (self.state.global_step - self._globalstep_last_logged), 4)
```
twice in a row.

To debug further why you have a problem, we would need to know what training arguments you are using and how you launch your training, which you are not willing to provide.

I know why now. I have override the`trainer.log()` func, and didn't add  `self.control = self.callback_handler.on_log(self.args, self.state, self.control, logs)` at the end of the func. So `self.control.should_log` didn't set to False after log action.
Because my code was upgraded from the previous transformers 3.X version, this piece of code was not updated.

Thanks for your help!