User-defined callback can't use logging
Author: Zessay
Pull request: no

I define my own callback class and want to log some info every `print_step`. However, the `logger.info` statement doesn't work when training. If modifying `logger.info` to `print`, it prints the content correctly. But if I set `gradient_accumulation_steps` greater than `1`,  it will print `gradient_accumulation_steps` times which I don't want. I guess it may related to asynchronization, but I have no idea how to solve this problem. It would be my pleasure if anyone could help me. Thanks in adavance.

The following is my code

```python
import collections
import logging

from transformers.training_args import TrainingArguments
from transformers.trainer_callback import TrainerCallback, TrainerState, TrainerControl


logger = logging.getLogger(__name__)


class KubeLogCallback(TrainerCallback):
    def __init__(self, print_step: int = 100):
        self.print_step = print_step
        self.eval_step = 0
        
    def on_step_end(self, args: TrainingArguments, state: TrainerState, control: TrainerControl, **kwargs):
        if state.is_local_process_zero:
            if state.global_step != 0 and state.global_step % self.print_step == 0:
                step_log = f"Epoch: {state.epoch}/{state.num_train_epochs} |" \
                    + f" Steps: {state.global_step}/{state.max_steps} |"
                # print(step_log)
                logger.info(step_log)

    def on_substep_end(self, args: TrainingArguments, state: TrainerState, control: TrainerControl, **kwargs):
        self.on_step_end(args, state, control, **kwargs)

    def on_prediction_step(self, args: TrainingArguments, state: TrainerState, control: TrainerControl,
                           eval_dataloader=None, **kwargs):
        if eval_dataloader is not None:
            if state.is_local_process_zero and isinstance(eval_dataloader.dataset, collections.abc.Sized):
                self.eval_step += 1
                current_eval_step = self.eval_step % len(eval_dataloader)
                if current_eval_step != 0 and current_eval_step % 100 == 0:
                    step_log = f"Eval Steps: {current_eval_step}/{len(eval_dataloader)}"
                    logger.info(step_log)

```

Comments:
cc @sgugger 

I think this is because you are not using the logger from the transformers library but a new logger. If you replace `__name__` by `"transformers"` it should work.

> I think this is because you are not using the logger from the transformers library but a new logger. If you replace `__name__` by `"transformers"` it should work.

Thanks! I modify `__name__` to `transformers` and it can log the info correctly. But if I set `gradient_accumulation_steps` greater than `1`, it still log `gradient_accumulation_steps` times which looks redundancy. Is there any good way to slove it? Thanks for your reply again!

That is because of:
```py
    def on_substep_end(self, args: TrainingArguments, state: TrainerState, control: TrainerControl, **kwargs):
        self.on_step_end(args, state, control, **kwargs)
```

The `on_step_end` method is supposed to only be called every gradient accumulation steps of the training data, but your callback calls it at every subset.

> That is because of:
> 
> ```python
>     def on_substep_end(self, args: TrainingArguments, state: TrainerState, control: TrainerControl, **kwargs):
>         self.on_step_end(args, state, control, **kwargs)
> ```
> 
> The `on_step_end` method is supposed to only be called every gradient accumulation steps of the training data, but your callback calls it at every subset.

OK，thank you! :satisfied: