Fix `tqdm` zip bug
State: closed

This PR solves #5117, by wrapping the entire `zip` clause in tqdm.

For more information, please checkout this Stack Overflow thread:
https://stackoverflow.com/questions/41171191/tqdm-progressbar-and-zip-built-in-do-not-work-together

Comments:
@albertvillanova Thanks for your comment. What do you think about creating 2 `pbar` for each case? I see the `pbar_iterable` is initialized differently. Maybe `pbar` can also be initialized like that.

@albertvillanova Another solution I implemented is to change `pbar_iterable` and add the `zip` to it. I updated the PR with this solution. Let me know what you think.

_The documentation is not available anymore as the PR was closed or merged._

@albertvillanova Done :) Let me know what you think.

@albertvillanova Thanks :) I also don't see an easy way to test this. This was just a problem in the way `tqdm` was used. I'm not sure we should cover it in tests.

Hi, 

First of all, thanks for this PR. 
It's the first time I join a discussion on GitHUB on problem resolution in libraries such as transformers, so I hope I comply to the best practices for an efficient communication...

I am running `AutoTokenizer.from_pretrained` in a Google Colab notebook for using with BERT base. 
I am experiencing issue [5117](https://github.com/huggingface/datasets/issues/5117).

Each time I run my notebook, I do:

`! pip install transformers 
! pip install datasets 
! pip install huggingface_hub`

As I understand, the issue has been resolved and the solution merged to the released version of the code?
So I expect that the bug is resolved in my notebook, however this is not the case.

Do I get something wrong? 
Do I have to implement some change in the source code myself?

Thanks in advance for your help!

@Cochonaki Hi :) The problem was fixed but there wasn't a release since then. I believe a new release should come out in the upcoming weeks. Maybe someone from the core maintainers can answer that :)

cc: @albertvillanova 

Baby Haiti Coffee SE is born

NH watch

On Sun, Oct 23, 2022 at 02:39 Dudu Lasry ***@***.***> wrote:

> @Cochonaki <https://github.com/Cochonaki> Hi :) The problem was fixed but
> there wasn't a release since then. I believe a new release should come out
> in the upcoming weeks. Maybe someone from the core maintainers can answer
> that :)
>
> cc: @albertvillanova <https://github.com/albertvillanova>
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/pull/5120#issuecomment-1288024546>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAB4E2NCT7QO7W3PTQGDIKDWETMQ7ANCNFSM6AAAAAARGRBY2M>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>


Hi, @Cochonaki.

As @david1542 pointed out, we have not made a release since this bug was fixed. We will make one in the following weeks.

In the meantime, if you would like to incorporate the bug fix, you can install `datasets` from this repo main branch:
```shell
pip install git+https://github.com/huggingface/datasets#egg=datasets
```

Thanks a lot @albertvillanova and @david1542, it works now!
I am really thankful for your help, that encourages me to participate more in this community.
See you around!

Welcome!!! 🤗