[logging] unable to turn off tqdm logging
Author: stas00
Pull request: no

I'm writing a program in a notebook where I'm printing a results table for multiple models and I can't figure out how to turn off tqdm here as it downloads new models since its output breaks the table and adds a ton of unnecessary noise to the notebook's outputs.

So here is where the control is:

https://github.com/huggingface/transformers/blob/207594be81b8e5a8589c8b11c3b236924555d806/src/transformers/file_utils.py#L1890-L1898

Why does it turn off only if logging is `logging.NOTSET`?  `logging.NOTSET` is a special level. If it's `NOTSET` then the next handler's log level is used, so we end up falling back to the root's default log level which is WARNING. (detailed info: https://docs.python.org/3/library/logging.html#logging.Logger.setLevel)

In other words:

https://github.com/huggingface/transformers/blob/207594be81b8e5a8589c8b11c3b236924555d806/src/transformers/file_utils.py#L1897

will never be true.

So we need to choose a true level at which tqdm is enabled. If it's an info log level, then most likely something like:

`disable=bool(logging.get_verbosity() >= logging.WARNING)`?

Thank you

@LysandreJik, @sgugger 

-----------

a quick test for myself to check that tqdm can be turned off once we sort it out (assuming this model is not cached):

```
python -c "import transformers, logging; transformers.logging.set_verbosity_error(); \
transformers.AutoModel.from_pretrained('google/pegasus-pubmed')"
```
shouldn't log anything, but try `set_verbosity_info` first to ensure that the model isn't cached yet and kill it before it finishes.


Comments:
Hey @stas00, thanks for raising an issue! I think the `WARNING` level is probably too low, I'd expect to see some tqdm bars at this level. I think even in `ERROR` we would expect some tqdm bars?

I believe that the `datasets` team uses a specific logging command to turn tqdm bars off:
https://github.com/huggingface/datasets/blob/1aa09c9f5d7886ca1d3824ce5f9f3b82356b7fd2/src/datasets/utils/tqdm_utils.py#L71-L80

What do you think of having something similar? For many things, having progress bars disabled could mean waiting for a very very long time without any info about what's happening, so I feel like it's a bit orthogonal to the log level.

> For many things, having progress bars disabled could mean waiting for a very very long time without any info about what's happening, so I feel like it's a bit orthogonal to the log level.

a log is a log is a log is a log - a user should be able to control logging regardless of its function, IMHO, of course. 

for example, while in the console it's hard to know what's going on, in a notebook one can see exactly where the processing is, so it's much less of an issue.

> I believe that the datasets team uses a specific logging command to turn tqdm bars off:

It's totally fine with me if we do the same for `transformers`. Are you OK with re-using the same API as you linked to?

We can then document it in the logging doc.

Actually, I have just thought of a using a stream catcher

```
from transformers.testing_utils import CaptureStd
with CaptureStd():
    transformers.AutoModel.from_pretrained('google/pegasus-pubmed')"
```

but no, tqdm doesn't use normal std streams and thus can't be captured.

Yes, perfectly OK with re-using the same API!

Hello @stas00, have you started working on this? If not, I was wondering if I could attempt a PR!

yes, please, @jaketae - thank you!

> > For many things, having progress bars disabled could mean waiting for a very very long time without any info about what's happening, so I feel like it's a bit orthogonal to the log level.
> 
> a log is a log is a log is a log - a user should be able to control logging regardless of its function, IMHO, of course.
> 


Agree - our use case is we are load-testing a vector embeddings app that is using `tqdm` implicitly.  When we send in large documents it logs thousands of statements like this

![image](https://github.com/huggingface/transformers/assets/1972276/bdf45a19-07d6-4e24-9950-c7ccac3541b4)

`export TQDM_DISABLE=1`  doesn't work for me on 4.66.4