Cannot disable logging from trainer module
Author: alexf-a
Pull request: no

@sgugger @stas00 

- `transformers` version: 3.2.0
- Platform:
- Python version: 3.7.6
- PyTorch version (GPU?): 1.6.0, Tesla V100
- Tensorflow version (GPU?):
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Parallel

I am using the Hugging Face Trainer class for NER fine-tuning. Whenever I turn evaluation on with the `--do_eval` argument, my console gets overwhelmed with a printed dictionary that appears to be coming from evaluation that is happening inside of the Trainer.  

The dictionary has the keys:

` 'eval_loss', 'eval_accuracy_score', 'eval_precision', 'eval_recall', 'eval_f1', 'eval_class_report', 'eval_predictions' `

It's especially hard to read the console with NER predictions, because `eval_predictions`  is a list with each token receiving an IOB tag. 

I tried suppressing logging from transformers with this solution https://github.com/huggingface/transformers/issues/3050. I also tried disabling all logging below CRITICAL level. The problem persisted, and I noticed that the console output of the evaluation dictionary appeared to be coming from a print statement.

I tried suppressing all print statements from the Trainer's `train(...)` method, using this solution https://stackoverflow.com/questions/977840/redirecting-fortran-called-via-f2py-output-in-python/978264#978264. That worked, but now I have no logging of training at all :(. 



Comments:
Could you elaborate on the script you're using. `run_ner.py` does not report `eval_class_report` or `eval_predictions` and there are no print statements in it, nor are there in `Trainer.train` method.

Yes, it's a custom script. The script creates an `AutoModelForTokenClassification,` passes it to the `Trainer` and calls the `Trainer.train` method.

We are using WandB to plot a confusion matrix, so we define our own `compute_metrics` function which we also pass to the `Trainer` (sorry should have stated this earlier). `compute metrics` does return a dictionary with the keys `'predictions', 'class_report' and 'target'`. It looks like one that gets output from `Trainer` has the prefix `'eval_'` in front of each key produced by `compute_metrics`. 

I can't find anywhere in our script or our custom dependencies where we print or log this dictionary, and when I suppress console output from `Trainer` then the problem stops. 



Is it just a matter of changing the log level? `run_ner.py` sets it to `INFO` for the main process (it does it twice - once for the root logger and another for transformers' logger:

https://github.com/huggingface/transformers/blob/251eb70c979d74d3823e999236ff3621b07510a1/examples/token-classification/run_ner.py#L158-L168



Maybe your `Trainer` ends up with a `PrinterCallback` that prints all the logs. You can remove this with
```
form transformers.trainer_callback import PrinterCallback
trainer.remove_callback(PrinterCallback)
```

Fixed! I had to upgrade to 4.0. Ended up having to upgrade to import the PrinterCalback, but I believe the upgrade itself fixed the problem. 

Even better if the upgrade fixes the problem! There were printing statements in older versions indeed.

Same issue here - How do I stop all prints coming from trainer.predict() ?