Repository: iterative/dvc

Issue:
ui: remove multi threaded prefix before each progress bar
When you run even a simple `dvc pull` for a single dvc file for a project that has only one data file it still shows the `Multi threaded` prefix before the progress bar.

```
git clone https://github.com/iterative/example-get-started
cd example-get-started
dvc pull
```

outputs some like:

```
Multi-Threaded:
 17%|█▋        |data/prepared/train.ts...
```

Commands like `dvc pull`, `dvc push`, etc are affected. I would expect to see something meaningful instead: `Downloading files (multiple threads)` or even w/o multiple threads.

Sometimes this `Multi-Threaded:` is blinking and replacing the progress bar itself which makes experience even worse.

ui: remove multi threaded prefix before each progress bar
When you run even a simple `dvc pull` for a single dvc file for a project that has only one data file it still shows the `Multi threaded` prefix before the progress bar.

```
git clone https://github.com/iterative/example-get-started
cd example-get-started
dvc pull
```

outputs some like:

```
Multi-Threaded:
 17%|█▋        |data/prepared/train.ts...
```

Commands like `dvc pull`, `dvc push`, etc are affected. I would expect to see something meaningful instead: `Downloading files (multiple threads)` or even w/o multiple threads.

Sometimes this `Multi-Threaded:` is blinking and replacing the progress bar itself which makes experience even worse.

Remove multi-threaded prefix before progress bar(#2686)
Fix #2686 

Also change the import to follow PEP8 import rules

* [x] Have you followed the guidelines in our
      [Contributing document](https://dvc.org/doc/user-guide/contributing/core)?

* [x] Does your PR affect documented changes or does it add new functionality
      that should be documented? If yes, have you created a PR for
      [dvc.org](https://github.com/iterative/dvc.org) documenting it or at
      least opened an issue for it? If so, please add a link to it.

-----



Discussion:
@shcheklein you just want different static message right? or want some conditional message?
I would like to work on this, I am curious what is causing this Multi-Threaded message to come though?
@harsh8398 it's better to have a conditional message - that describes what is happening depending on the command.

We def should remove the blinking behavior.

@kurianbenoy feel free! but please coordinate with @harsh8398 if needed and please ask an advice with @casperdcl  - an expert in TQDM :)
https://github.com/iterative/dvc/blob/a48481b01fd607b36f9ea5b643af24c938254f60/dvc/progress.py#L23

but @shcheklein
> Sometimes this `Multi-Threaded:` is blinking and replacing the progress bar itself which makes experience even worse.

shouldn't happen - is this still a thing after #2546?
@casperdcl I think it is still happening (on `dvc pull`). But I'll check again on the latest dev version. 
I'm referring to specifically the blinking and overwriting. Shouldn't happen.

Anyway there has to at least be a temporary blank line (could replace `"Multi-Threaded:"` with an empty string) as that's by far the most painless way of letting `tqdm` know about being in multi-threaded mode.
Didn't get the last thing. Why tqdm can't be in multi-threaded mode without stating that with a message?
https://github.com/iterative/dvc/blob/877e38551eca7b8d88155b4ae97dd1237c99b0d0/dvc/progress.py#L20-L21

I think the main idea is https://github.com/tqdm/tqdm/blob/91494ccf363e5eeb4745db7dbab70ef73c4a6eb0/tqdm/std.py#L1240 but due to https://github.com/iterative/dvc/blob/877e38551eca7b8d88155b4ae97dd1237c99b0d0/dvc/progress.py#L56 it might not be an issue any more.

Ah wait yes of course - it we assume all threads have `leave=False` then it's safe to remove the message. Otherwise they may wind up printing out of order.
So @casperdcl will it be possible to write conditional messages for each command as @shcheklein said?
I think missing conditional messages/verbosity is unrelated to the "remove unhelpful multi-threading message" issue. The first is a long term goal involving carefully relooking at each commamnd. The latter could probably be fixed by just removing the fake bar altogether...
@shcheklein you just want different static message right? or want some conditional message?
I would like to work on this, I am curious what is causing this Multi-Threaded message to come though?
@harsh8398 it's better to have a conditional message - that describes what is happening depending on the command.

We def should remove the blinking behavior.

@kurianbenoy feel free! but please coordinate with @harsh8398 if needed and please ask an advice with @casperdcl  - an expert in TQDM :)
https://github.com/iterative/dvc/blob/a48481b01fd607b36f9ea5b643af24c938254f60/dvc/progress.py#L23

but @shcheklein
> Sometimes this `Multi-Threaded:` is blinking and replacing the progress bar itself which makes experience even worse.

shouldn't happen - is this still a thing after #2546?
@casperdcl I think it is still happening (on `dvc pull`). But I'll check again on the latest dev version. 
I'm referring to specifically the blinking and overwriting. Shouldn't happen.

Anyway there has to at least be a temporary blank line (could replace `"Multi-Threaded:"` with an empty string) as that's by far the most painless way of letting `tqdm` know about being in multi-threaded mode.
Didn't get the last thing. Why tqdm can't be in multi-threaded mode without stating that with a message?
https://github.com/iterative/dvc/blob/877e38551eca7b8d88155b4ae97dd1237c99b0d0/dvc/progress.py#L20-L21

I think the main idea is https://github.com/tqdm/tqdm/blob/91494ccf363e5eeb4745db7dbab70ef73c4a6eb0/tqdm/std.py#L1240 but due to https://github.com/iterative/dvc/blob/877e38551eca7b8d88155b4ae97dd1237c99b0d0/dvc/progress.py#L56 it might not be an issue any more.

Ah wait yes of course - it we assume all threads have `leave=False` then it's safe to remove the message. Otherwise they may wind up printing out of order.
So @casperdcl will it be possible to write conditional messages for each command as @shcheklein said?
I think missing conditional messages/verbosity is unrelated to the "remove unhelpful multi-threading message" issue. The first is a long term goal involving carefully relooking at each commamnd. The latter could probably be fixed by just removing the fake bar altogether...


Resolving patch:
diff --git a/dvc/progress.py b/dvc/progress.py
--- a/dvc/progress.py
+++ b/dvc/progress.py
@@ -3,35 +3,12 @@
 import logging
 import sys
 from tqdm import tqdm
-from concurrent.futures import ThreadPoolExecutor
 from funcy import merge
 from dvc.utils import env2bool
 
 logger = logging.getLogger(__name__)
 
 
-class TqdmThreadPoolExecutor(ThreadPoolExecutor):
-    """
-    Ensure worker progressbars are cleared away properly.
-    """
-
-    def __enter__(self):
-        """
-        Creates a blank initial dummy progress bar if needed so that workers
-        are forced to create "nested" bars.
-        """
-        blank_bar = Tqdm(bar_format="Multi-Threaded:", leave=False)
-        if blank_bar.pos > 0:
-            # already nested - don't need a placeholder bar
-            blank_bar.close()
-        self.bar = blank_bar
-        return super(TqdmThreadPoolExecutor, self).__enter__()
-
-    def __exit__(self, *a, **k):
-        super(TqdmThreadPoolExecutor, self).__exit__(*a, **k)
-        self.bar.close()
-
-
 class Tqdm(tqdm):
     """
     maximum-compatibility tqdm-based progressbars
diff --git a/dvc/remote/local.py b/dvc/remote/local.py
--- a/dvc/remote/local.py
+++ b/dvc/remote/local.py
@@ -4,6 +4,7 @@
 import errno
 import logging
 from functools import partial
+from concurrent.futures import ThreadPoolExecutor
 
 from shortuuid import uuid
 
@@ -29,7 +30,7 @@
 )
 from dvc.config import Config
 from dvc.exceptions import DvcException, DownloadError, UploadError
-from dvc.progress import Tqdm, TqdmThreadPoolExecutor
+from dvc.progress import Tqdm
 
 from dvc.path_info import PathInfo
 
@@ -359,7 +360,7 @@ def _process(
             return 0
 
         if jobs > 1:
-            with TqdmThreadPoolExecutor(max_workers=jobs) as executor:
+            with ThreadPoolExecutor(max_workers=jobs) as executor:
                 fails = sum(executor.map(func, *plans))
         else:
             fails = sum(map(func, *plans))


Test patch: