Not all progress bars are showing up when they should for downloading dataset
Opened by garlandz-db (closed)

### Describe the bug

During downloading the rotten tomatoes dataset, not all progress bars are displayed properly. This might be related to [this ticket](https://github.com/huggingface/datasets/issues/5117) as it raised the same concern but its not clear if the fix solves this issue too.



ipywidgets
<img width="1243" alt="image" src="https://user-images.githubusercontent.com/110427462/224851138-13fee5b7-ab51-4883-b96f-1b9808782e3b.png">


tqdm
<img width="1251" alt="Screen Shot 2023-03-13 at 3 58 59 PM" src="https://user-images.githubusercontent.com/110427462/224851180-5feb7825-9250-4b1e-ad0c-f3172ac1eb78.png">

### Steps to reproduce the bug

1. Run this line
```
from datasets import load_dataset

rotten_tomatoes = load_dataset("rotten_tomatoes", split="train")
```

### Expected behavior

all progress bars for builder script, metadata, readme, training, validation, and test set


### Environment info

requirements.txt
```
aiofiles==22.1.0
aiohttp==3.8.4
aiosignal==1.3.1
aiosqlite==0.18.0
anyio==3.6.2
appnope==0.1.3
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
arrow==1.2.3
asttokens==2.2.1
async-generator==1.10
async-timeout==4.0.2
attrs==22.2.0
Babel==2.12.1
backcall==0.2.0
beautifulsoup4==4.11.2
bleach==6.0.0
brotlipy @ file:///Users/runner/miniforge3/conda-bld/brotlipy_1666764961872/work
certifi==2022.12.7
cffi @ file:///Users/runner/miniforge3/conda-bld/cffi_1671179414629/work
cfgv==3.3.1
charset-normalizer @ file:///home/conda/feedstock_root/build_artifacts/charset-normalizer_1661170624537/work
comm==0.1.2
conda==22.9.0
conda-package-handling @ file:///home/conda/feedstock_root/build_artifacts/conda-package-handling_1669907009957/work
conda_package_streaming @ file:///home/conda/feedstock_root/build_artifacts/conda-package-streaming_1669733752472/work
coverage==7.2.1
cryptography @ file:///Users/runner/miniforge3/conda-bld/cryptography_1669592251328/work
datasets==2.1.0
debugpy==1.6.6
decorator==5.1.1
defusedxml==0.7.1
dill==0.3.6
distlib==0.3.6
distro==1.4.0
entrypoints==0.4
exceptiongroup==1.1.0
executing==1.2.0
fastjsonschema==2.16.3
filelock==3.9.0
flaky==3.7.0
fqdn==1.5.1
frozenlist==1.3.3
fsspec==2023.3.0
huggingface-hub==0.10.1
identify==2.5.18
idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1663625384323/work
iniconfig==2.0.0
ipykernel==6.12.1
ipyparallel==8.4.1
ipython==7.32.0
ipython-genutils==0.2.0
ipywidgets==8.0.4
isoduration==20.11.0
jedi==0.18.2
Jinja2==3.1.2
json5==0.9.11
jsonpointer==2.3
jsonschema==4.17.3
jupyter-events==0.6.3
jupyter-ydoc==0.2.2
jupyter_client==8.0.3
jupyter_core==5.2.0
jupyter_server==2.4.0
jupyter_server_fileid==0.8.0
jupyter_server_terminals==0.4.4
jupyter_server_ydoc==0.6.1
jupyterlab==3.6.1
jupyterlab-pygments==0.2.2
jupyterlab-widgets==3.0.5
jupyterlab_server==2.20.0
libmambapy @ file:///Users/runner/miniforge3/conda-bld/mamba-split_1671598370072/work/libmambapy
mamba @ file:///Users/runner/miniforge3/conda-bld/mamba-split_1671598370072/work/mamba
MarkupSafe==2.1.2
matplotlib-inline==0.1.6
mistune==2.0.5
multidict==6.0.4
multiprocess==0.70.14
nbclassic==0.5.3
nbclient==0.7.2
nbconvert==7.2.9
nbformat==5.7.3
nest-asyncio==1.5.6
nodeenv==1.7.0
notebook==6.5.3
notebook_shim==0.2.2
numpy==1.24.2
outcome==1.2.0
packaging==23.0
pandas==1.5.3
pandocfilters==1.5.0
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
platformdirs==3.0.0
plotly==5.13.1
pluggy==1.0.0
pre-commit==3.1.0
prometheus-client==0.16.0
prompt-toolkit==3.0.38
psutil==5.9.4
ptyprocess==0.7.0
pure-eval==0.2.2
pyarrow==11.0.0
pycosat @ file:///Users/runner/miniforge3/conda-bld/pycosat_1666836580084/work
pycparser @ file:///home/conda/feedstock_root/build_artifacts/pycparser_1636257122734/work
Pygments==2.14.0
pyOpenSSL @ file:///home/conda/feedstock_root/build_artifacts/pyopenssl_1665350324128/work
pyrsistent==0.19.3
PySocks @ file:///home/conda/feedstock_root/build_artifacts/pysocks_1661604839144/work
pytest==7.2.1
pytest-asyncio==0.20.3
pytest-cov==4.0.0
pytest-timeout==2.1.0
python-dateutil==2.8.2
python-json-logger==2.0.7
pytz==2022.7.1
PyYAML==6.0
pyzmq==25.0.0
requests @ file:///home/conda/feedstock_root/build_artifacts/requests_1661872987712/work
responses==0.18.0
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
ruamel-yaml-conda @ file:///Users/runner/miniforge3/conda-bld/ruamel_yaml_1666819760545/work
Send2Trash==1.8.0
simplegeneric==0.8.1
six==1.16.0
sniffio==1.3.0
sortedcontainers==2.4.0
soupsieve==2.4
stack-data==0.6.2
tenacity==8.2.2
terminado==0.17.1
tinycss2==1.2.1
tomli==2.0.1
toolz @ file:///home/conda/feedstock_root/build_artifacts/toolz_1657485559105/work
tornado==6.2
tqdm==4.64.1
traitlets==5.8.1
trio==0.22.0
typing_extensions==4.5.0
uri-template==1.2.0
urllib3 @ file:///home/conda/feedstock_root/build_artifacts/urllib3_1669259737463/work
virtualenv==20.19.0
wcwidth==0.2.6
webcolors==1.12
webencodings==0.5.1
websocket-client==1.5.1
widgetsnbextension==4.0.5
xxhash==3.2.0
y-py==0.5.9
yarl==1.8.2
ypy-websocket==0.8.2
zstandard==0.19.0
```

Comments:
Hi! 

By default, tqdm has `leave=True` to "keep all traces of the progress bar upon the termination of iteration". However, we use `leave=False` in some places (as of recently), which removes the bar once the iteration is over.

I feel like our TQDM bars are noisy, so I think we should always set `leave=False` and also use the `delay` parameter to display progress bars only for tasks that take time (e.g., more than 3s). What do you think about this? Do you find these bars useful (after the dataset generation is over)?


Hi sorry for the late update. I think the problem still exists despite the `leave` flag

<img width="1105" alt="image" src="https://user-images.githubusercontent.com/110427462/226501615-5b02fb02-fd5f-4eda-b1f7-a7ed6570892d.png">


```
Package                  Version
------------------------ ---------
aiofiles                 22.1.0
aiohttp                  3.8.4
aiosignal                1.3.1
aiosqlite                0.18.0
anyio                    3.6.2
appnope                  0.1.3
argon2-cffi              21.3.0
argon2-cffi-bindings     21.2.0
arrow                    1.2.3
asttokens                2.2.1
async-generator          1.10
async-timeout            4.0.2
attrs                    22.2.0
Babel                    2.12.1
backcall                 0.2.0
beautifulsoup4           4.11.2
bleach                   6.0.0
brotlipy                 0.7.0
certifi                  2022.12.7
cffi                     1.15.1
cfgv                     3.3.1
charset-normalizer       2.1.1
comm                     0.1.2
conda                    22.9.0
conda-package-handling   2.0.2
conda_package_streaming  0.7.0
coverage                 7.2.1
cryptography             38.0.4
datasets                 2.8.0
debugpy                  1.6.6
decorator                5.1.1
defusedxml               0.7.1
dill                     0.3.6
distlib                  0.3.6
distro                   1.4.0
entrypoints              0.4
exceptiongroup           1.1.0
executing                1.2.0
fastjsonschema           2.16.3
filelock                 3.9.0
flaky                    3.7.0
fqdn                     1.5.1
frozenlist               1.3.3
fsspec                   2023.3.0
huggingface-hub          0.10.1
identify                 2.5.18
idna                     3.4
iniconfig                2.0.0
ipykernel                6.12.1
ipyparallel              8.4.1
ipython                  7.32.0
ipython-genutils         0.2.0
ipywidgets               8.0.4
isoduration              20.11.0
jedi                     0.18.2
Jinja2                   3.1.2
json5                    0.9.11
jsonpointer              2.3
jsonschema               4.17.3
jupyter_client           8.0.3
jupyter_core             5.2.0
jupyter-events           0.6.3
jupyter_server           2.4.0
jupyter_server_fileid    0.8.0
jupyter_server_terminals 0.4.4
jupyter_server_ydoc      0.6.1
jupyter-ydoc             0.2.2
jupyterlab               3.6.1
jupyterlab-pygments      0.2.2
jupyterlab_server        2.20.0
jupyterlab-widgets       3.0.5
libmambapy               1.1.0
mamba                    1.1.0
MarkupSafe               2.1.2
matplotlib-inline        0.1.6
mistune                  2.0.5
multidict                6.0.4
multiprocess             0.70.14
nbclassic                0.5.3
nbclient                 0.7.2
nbconvert                7.2.9
nbformat                 5.7.3
nest-asyncio             1.5.6
nodeenv                  1.7.0
notebook                 6.5.3
notebook_shim            0.2.2
numpy                    1.24.2
outcome                  1.2.0
packaging                23.0
pandas                   1.5.3
pandocfilters            1.5.0
parso                    0.8.3
pexpect                  4.8.0
pickleshare              0.7.5
pip                      22.3.1
platformdirs             3.0.0
plotly                   5.13.1
pluggy                   1.0.0
pre-commit               3.1.0
prometheus-client        0.16.0
prompt-toolkit           3.0.38
psutil                   5.9.4
ptyprocess               0.7.0
pure-eval                0.2.2
pyarrow                  11.0.0
pycosat                  0.6.4
pycparser                2.21
Pygments                 2.14.0
pyOpenSSL                22.1.0
pyrsistent               0.19.3
PySocks                  1.7.1
pytest                   7.2.1
pytest-asyncio           0.20.3
pytest-cov               4.0.0
pytest-timeout           2.1.0
python-dateutil          2.8.2
python-json-logger       2.0.7
pytz                     2022.7.1
PyYAML                   6.0
pyzmq                    25.0.0
requests                 2.28.1
responses                0.18.0
rfc3339-validator        0.1.4
rfc3986-validator        0.1.1
ruamel-yaml-conda        0.15.80
Send2Trash               1.8.0
setuptools               65.6.3
simplegeneric            0.8.1
six                      1.16.0
sniffio                  1.3.0
sortedcontainers         2.4.0
soupsieve                2.4
stack-data               0.6.2
tenacity                 8.2.2
terminado                0.17.1
tinycss2                 1.2.1
tomli                    2.0.1
toolz                    0.12.0
tornado                  6.2
tqdm                     4.65.0
traitlets                5.8.1
trio                     0.22.0
typing_extensions        4.5.0
uri-template             1.2.0
urllib3                  1.26.13
virtualenv               20.19.0
wcwidth                  0.2.6
webcolors                1.12
webencodings             0.5.1
websocket-client         1.5.1
wheel                    0.38.4
widgetsnbextension       4.0.5
xxhash                   3.2.0
y-py                     0.5.9
yarl                     1.8.2
ypy-websocket            0.8.2
zstandard                0.19.0
```

Any idea why this is happening? I debugged this to know the tqdm.pbar value is not being updated properly and its not the kernel not sending the comm messages to the IProgress bar