Title: sync logging utils with transformers
Author: stas00 (CONTRIBUTOR)
State: closed
Created: 2020-09-11T19:46:13Z

sync the docs/code with the recent changes in transformers' `logging` utils:
1. change the default level to `WARNING`
2. add `DATASETS_VERBOSITY` env var
3. expand docs

Comments:
Also, some downloads and dataset processing can be quite long for large datasets like wikipedia/pg19/etc. We probably don't want to user to think that the library is hanging. Happy to reorganize logging between DEBUG/INFO/WARNING to make it less verbose by default though.

The problem is that `transformers` imports `datasets` and the latter starts logging on `import`: at least 3 info messages - apache beam/torch/tf available - so it injects noise whether one uses the library or not - i.e. no choice given to the user.

Would you be open for me to changing this PR, to keep the initial level at INFO, but to keep the `DATASETS_VERBOSITY` env var it introduces, to let the user control the verbosity?



> Also, some downloads and dataset processing can be quite long for large datasets like wikipedia/pg19/etc. We probably don't want to user to think that the library is hanging.

If you're referring to tqdm progress reports, it's not affected by changing the logging levels. It's not using logging.

> The problem is that `transformers` imports `datasets` and the latter starts logging on `import`: at least 3 info messages - apache beam/torch/tf available - so it injects noise whether one uses the library or not - i.e. no choice given to the user.
> 
> Would you be open for me to changing this PR, to keep the initial level at INFO, but to keep the `DATASETS_VERBOSITY` env var it introduces, to let the user control the verbosity?

For now we can do that, then I'll change some messages to warnings and set the default verbosity at warning as well at that point. Does it sound good to you ?

> If you're referring to tqdm progress reports, it's not affected by changing the logging levels. It's not using logging.

Actually we configured some progress bars to be disabled depending on the logging level ^^'


> For now we can do that, then I'll change some messages to warnings and set the default verbosity at warning as well at that point. Does it sound good to you ?

If it is logical then by all means. 

> > If you're referring to tqdm progress reports, it's not affected by changing the logging levels. It's not using logging.
> 
> Actually we configured some progress bars to be disabled depending on the logging level ^^'

This is very smart!

I reverted s/WARNINGS/INFO/.

Thank you!

Note that it’s the same in `transformers` @stas00, tdqm are also controlled by the logging level there.

> Note that it’s the same in `transformers` @stas00, tdqm are also controlled by the logging level there.

That's good to know, @thomwolf - thank you!

I see that it's controlled in `trainer.py`, but in `examples` it's not - since that's where I usually see the progressbars (and they are great!). But I suppose they aren't API, so `examples` can behave differently.

BTW, this is what I'm talking about:
```
python -c "import transformers"
2020-09-14 21:00:58.032658: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
PyTorch version 1.7.0.dev20200910 available.
TensorFlow version 2.3.0 available.
Apache Beam available.
```
why does the user need to see this? Especially, if they aren't even using `datasets` directly?

Yes you are right, we should re-think the logging level of various elements.
I also think that the `set_format` messages are confusing when they are the results of our internal operations (as mentioned [here](https://discuss.huggingface.co/t/pipeline-with-custom-dataset-tokenizer-when-to-save-load-manually/1084/7?u=thomwolf))

Actually I continued this PR in #635 to set the level to warning and update the logging level of some messages.

Let me know if it sounds good to you

Closing this one sice #635 got merged

Awesome! Thank you!

Any ideas how to eliminate this remaining log line from tensorflow (I know it's not `datasets` related, but perhaps you know).
```
python -c "import transformers"
2020-09-17 08:38:34.718410: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
```