internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: iterative/dvc

Issue:
brancher: dvcignore does not work as expected for subrepos when using brancher
As noted by @efiop in #3883, in CleanTree when we do
```py
    @cached_property
    def dvcignore(self):
        return DvcIgnoreFilter(self.tree)
```
`DvcIgnoreFilter` will use `WorkingTree.tree_root` or `GitTree.tree_root` as the root DVC repo directory.

For DVC subrepos, if the tree is a GitTree from brancher, this causes dvcignore to use the host/parent repo root `.dvcignore` rather than the subrepo root `.dvcignore`, since `GitTree.tree_root` will be the (parent/host level) git/scm root directory and not the DVC subrepo directory. For working trees this is not an issue, since the working tree root will be set to the correct `subrepo.root_dir`.

Some investigation should also be done to verify whether or not this issue (using host DVC repo root instead of subrepo root for brancher GitTrees) affects other places in DVC as well.


Discussion:

Resolving patch:
diff --git a/dvc/ignore.py b/dvc/ignore.py
--- a/dvc/ignore.py
+++ b/dvc/ignore.py
@@ -92,13 +92,14 @@ def is_dvc_repo(directory):
 
 
 class DvcIgnoreFilter:
-    def __init__(self, tree):
+    def __init__(self, tree, root_dir):
         self.tree = tree
+        self.root_dir = root_dir
         self.ignores = {
             DvcIgnoreDirs([".git", ".hg", ".dvc"]),
             DvcIgnoreRepo(),
         }
-        for root, dirs, files in self.tree.walk(self.tree.tree_root):
+        for root, dirs, files in self.tree.walk(self.root_dir):
             self._update(root)
             dirs[:], files[:] = self(root, dirs, files)
 
@@ -124,7 +125,7 @@ def __init__(self, tree, tree_root=None):
 
     @cached_property
     def dvcignore(self):
-        return DvcIgnoreFilter(self.tree)
+        return DvcIgnoreFilter(self.tree, self.tree_root)
 
     @property
     def tree_root(self):


Test patch:
diff --git a/tests/func/test_ignore.py b/tests/func/test_ignore.py
--- a/tests/func/test_ignore.py
+++ b/tests/func/test_ignore.py
@@ -11,6 +11,7 @@
     DvcIgnoreRepo,
 )
 from dvc.remote import LocalRemote
+from dvc.repo import Repo
 from dvc.scm.tree import WorkingTree
 from dvc.utils import relpath
 from dvc.utils.fs import get_mtime_and_size
@@ -151,3 +152,20 @@ def test_ignore_external(tmp_dir, scm, dvc, tmp_path_factory):
     remote = LocalRemote(dvc, {})
     result = {relpath(f, ext_dir) for f in remote.walk_files(ext_dir)}
     assert result == {"y.backup", "tmp"}
+
+
+def test_ignore_subrepo(tmp_dir, scm, dvc):
+    tmp_dir.gen({".dvcignore": "foo", "subdir": {"foo": "foo"}})
+    scm.add([".dvcignore"])
+    scm.commit("init parent dvcignore")
+
+    subrepo_dir = tmp_dir / "subdir"
+    assert not dvc.tree.exists(subrepo_dir / "foo")
+
+    with subrepo_dir.chdir():
+        subrepo = Repo.init(subdir=True)
+        scm.add(str(subrepo_dir / "foo"))
+        scm.commit("subrepo init")
+
+    for _ in subrepo.brancher(all_commits=True):
+        assert subrepo.tree.exists(subrepo_dir / "foo")
Repository: iterative/dvc

Issue:
"ERROR: unexpected error - 'GDriveRemote' object has no attribute '_credentials_location'"
**Please provide information about your setup**
DVC Version: 0.94.0
installation: pip
OS: Windows

When pushing to a gdrive that has the wrong credentials for the first time, the following error appears: 

```
ERROR: unexpected error - 'GDriveRemote' object has no attribute '_credentials_location'
```

As far as I can tell, this is because the GDriveRemote object's credentials_location() method hasn't instantiated _credential_location yet, and this error prevents the correct error message from appearing.

Best wishes,

Nick





Discussion:
Hi @nickdelgrosso ! Indeed, looks like that's a typo https://github.com/iterative/dvc/blob/master/dvc/remote/gdrive.py  and it should be `self.credentials_location` instead. Would you like to submit a quick fix PR for it? :slightly_smiling_face: 

Resolving patch:
diff --git a/dvc/remote/gdrive.py b/dvc/remote/gdrive.py
--- a/dvc/remote/gdrive.py
+++ b/dvc/remote/gdrive.py
@@ -340,7 +340,7 @@ def _gdrive_shared_drive_id(self, item_id):
                     "use the `gdrive_user_credentials_file` config\n"
                     "   option if you use multiple GDrive remotes with "
                     "different email accounts.\n\nDetails".format(
-                        item_id, self.path_info, self._credentials_location
+                        item_id, self.path_info, self.credentials_location
                     )
                 ) from exc
             raise


Test patch:
Repository: iterative/dvc

Issue:
tests: flaky ssh tests
https://travis-ci.com/github/iterative/dvc/jobs/340412779

```sh
tests/unit/remote/ssh/test_connection.py::test_hardlink 
[gw1] [ 80%] FAILED tests/unit/remote/ssh/test_connection.py::test_hardlink 
tests/unit/remote/ssh/test_connection.py::test_copy 
[gw1] [ 80%] FAILED tests/unit/remote/ssh/test_connection.py::test_copy 
```

I thought this was only failing on my machine, so I was ignoring it. But, looks like it occasionally happens on the CI. On my machine though this always appears.

Also, an important thing to note here is that, locally on my machine. it only fails when I'm running all of the tests, but when I rerun failed tests, it passes.




Discussion:
Might consider making ssh_server fixture's scope extend to `session`. That might help, but I'm not 100% sure.
Nope, didn't solve https://travis-ci.com/github/iterative/dvc/jobs/340941135

Resolving patch:
diff --git a/dvc/remote/ssh/connection.py b/dvc/remote/ssh/connection.py
--- a/dvc/remote/ssh/connection.py
+++ b/dvc/remote/ssh/connection.py
@@ -237,11 +237,15 @@ def execute(self, cmd):
             or channel.recv_ready()
             or channel.recv_stderr_ready()
         ):
-            import select
+            import selectors
+
+            selector = selectors.DefaultSelector()
+            selector.register(stdout.channel, selectors.EVENT_READ)
 
             got_chunk = False
-            readq, _, _ = select.select([stdout.channel], [], [], self.timeout)
-            for c in readq:
+            events = selector.select(self.timeout)
+            for key, _ in events:
+                c = key.fileobj
                 if c.recv_ready():
                     stdout_chunks.append(stdout.channel.recv(len(c.in_buffer)))
                     got_chunk = True


Test patch:
Repository: iterative/dvc

Issue:
tests: flaky ssh tests
https://travis-ci.com/github/iterative/dvc/jobs/340412779

```sh
tests/unit/remote/ssh/test_connection.py::test_hardlink 
[gw1] [ 80%] FAILED tests/unit/remote/ssh/test_connection.py::test_hardlink 
tests/unit/remote/ssh/test_connection.py::test_copy 
[gw1] [ 80%] FAILED tests/unit/remote/ssh/test_connection.py::test_copy 
```

I thought this was only failing on my machine, so I was ignoring it. But, looks like it occasionally happens on the CI. On my machine though this always appears.

Also, an important thing to note here is that, locally on my machine. it only fails when I'm running all of the tests, but when I rerun failed tests, it passes.




Discussion:
Might consider making ssh_server fixture's scope extend to `session`. That might help, but I'm not 100% sure.
Nope, didn't solve https://travis-ci.com/github/iterative/dvc/jobs/340941135
Now, it''s failing, because of course, `mock-ssh-server` uses `select.select()`, why not?

I'll try creating a fix upstream later when I am done with #3881.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -121,7 +121,7 @@ def run(self):
     "flake8-docstrings",
     "pydocstyle<4.0",
     "jaraco.windows==3.9.2",
-    "mock-ssh-server>=0.6.0",
+    "mock-ssh-server@git+https://github.com/iterative/mock-ssh-server.git",
     "moto==1.3.14.dev464",
     "rangehttpserver==1.2.0",
     "beautifulsoup4==4.4.0",


Test patch:
Repository: iterative/dvc

Issue:
plots: replace --show-json with --show-vega
Requested by @dmpetrov for cml. `--show-vega` should require a target and return a filled vega template. `--show-json` is not needed, let's delete it.


Discussion:
https://github.com/iterative/dvc/pull/3891#issuecomment-638345115

Resolving patch:
diff --git a/dvc/command/plots.py b/dvc/command/plots.py
--- a/dvc/command/plots.py
+++ b/dvc/command/plots.py
@@ -209,5 +209,5 @@ def _add_output_arguments(parser):
         "--show-vega",
         action="store_true",
         default=False,
-        help="Show output in VEGA format.",
+        help="Show output in Vega format.",
     )


Test patch:
Repository: iterative/dvc

Issue:
remove unused dependencies
From the quick looks of it:
https://github.com/iterative/dvc/blob/c70adebc75d50cce4351b19b9ecff1ce5b3c5cfc/setup.py#L63-L65


Discussion:
@skshetry Please feel free to submit a PR :slightly_smiling_face: These are leftovers from plots/metrics changes.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -60,9 +60,6 @@ def run(self):
     "grandalf==0.6",
     "distro>=1.3.0",
     "appdirs>=1.4.3",
-    "treelib>=1.5.5",
-    "inflect>=2.1.0,<4",
-    "humanize>=0.5.1",
     "PyYAML>=5.1.2,<5.4",  # Compatibility with awscli
     "ruamel.yaml>=0.16.1",
     "funcy>=1.14",


Test patch:
Repository: iterative/dvc

Issue:
snap: 'sudo' cannot be used with build provider 'lxd'
https://travis-ci.com/github/iterative/dvc/jobs/346490988#L495


Discussion:

Resolving patch:
diff --git a/dvc/updater.py b/dvc/updater.py
--- a/dvc/updater.py
+++ b/dvc/updater.py
@@ -48,7 +48,15 @@ def _with_lock(self, func, action):
             logger.debug(msg.format(self.lock.lockfile, action))
 
     def check(self):
-        if os.getenv("CI") or env2bool("DVC_TEST") or PKG == "snap":
+        if os.getenv("CI") or env2bool("DVC_TEST"):
+            return
+
+        if PKG == "snap":
+            # hardcoded transition message
+            version_major = version.parse(self.current).major
+            self.current = "v{}".format(version_major)
+            self.latest = "v{}".format(version_major + 1)
+            self._notify()
             return
 
         self._with_lock(self._check, "checking")
@@ -138,6 +146,17 @@ def _get_update_instructions(self):
             ),
             "conda": "Run `{yellow}conda{reset} update dvc`",
             "choco": "Run `{yellow}choco{reset} upgrade dvc`",
+            "snap": (
+                "To upgrade to the latest major release,\n"
+                "run `{yellow}snap{reset} refresh --channel=latest/beta`, or\n"
+                "to stay on the current major release track,\n"
+                "run `{yellow}snap{reset} refresh --channel={current}/stable`"
+                "\n\n"
+                "{red}WARNING{reset}: ignoring this message will result in\n"
+                "snap automatically performing an upgrade soon.\n"
+                "More information can be found at\n"
+                "{blue}https://github.com/iterative/dvc/issues/3872{reset}"
+            ),
             None: (
                 "Find the latest release at\n"
                 "{blue}https://github.com/iterative/dvc/releases/latest{reset}"


Test patch:
Repository: iterative/dvc

Issue:
`plots modify` is not preserving `meta`
I think, I implemented this. But, somehow it's not working. Also need to take a look if comments are preserved.


Discussion:

Resolving patch:
diff --git a/dvc/dvcfile.py b/dvc/dvcfile.py
--- a/dvc/dvcfile.py
+++ b/dvc/dvcfile.py
@@ -208,6 +208,8 @@ def _dump_pipeline_file(self, stage):
 
         if existing_entry:
             orig_stage_data = data["stages"][stage.name]
+            if "meta" in orig_stage_data:
+                stage_data[stage.name]["meta"] = orig_stage_data["meta"]
             apply_diff(stage_data[stage.name], orig_stage_data)
         else:
             data["stages"].update(stage_data)


Test patch:
diff --git a/tests/func/test_dvcfile.py b/tests/func/test_dvcfile.py
--- a/tests/func/test_dvcfile.py
+++ b/tests/func/test_dvcfile.py
@@ -290,3 +290,37 @@ def test_remove_stage_preserves_comment(tmp_dir, dvc, run_copy):
         "# This copies 'foo' text to 'foo' file."
         in (tmp_dir / PIPELINE_FILE).read_text()
     )
+
+
+def test_dvcfile_dump_preserves_meta(tmp_dir, dvc, run_copy):
+    tmp_dir.gen("foo", "foo")
+    stage = run_copy("foo", "bar", name="run_copy")
+    dvcfile = stage.dvcfile
+
+    data = dvcfile._load()[0]
+    metadata = {"name": "copy-file"}
+    data["stages"]["run_copy"]["meta"] = metadata
+    dump_yaml(dvcfile.path, data)
+
+    dvcfile.dump(stage, update_pipeline=True)
+    assert dvcfile._load()[0] == data
+    assert dvcfile._load()[0]["stages"]["run_copy"]["meta"] == metadata
+
+
+def test_dvcfile_dump_preserves_comments(tmp_dir, dvc):
+    text = textwrap.dedent(
+        """\
+        stages:
+          generate-foo:
+            cmd: echo foo > foo
+            # This copies 'foo' text to 'foo' file.
+            outs:
+            - foo"""
+    )
+    tmp_dir.gen("dvc.yaml", text)
+    stage = dvc.get_stage(name="generate-foo")
+    stage.outs[0].use_cache = False
+    dvcfile = stage.dvcfile
+
+    dvcfile.dump(stage, update_pipeline=True)
+    assert dvcfile._load()[1] == (text + ":\n\tcache: false\n".expandtabs())
Repository: iterative/dvc

Issue:
`plots modify --unset` should not convert to plots
```console
$ cat dvc.yaml
stages:
  plot-metrics:
    cmd: cp metrics-temp.csv metrics.csv
    deps:
    - metrics-temp.csv
    metrics: 
    - metrics.csv:
        cache: false
$ dvc plots modify -q metrics.csv --unset
$ cat dvc.yaml
stages:
  plot-metrics:
    cmd: cp metrics-temp.csv metrics.csv
    deps:
    - metrics-temp.csv
    plots:
    - metrics.csv:
        cache: false
```


Discussion:

Resolving patch:
diff --git a/dvc/repo/plots/__init__.py b/dvc/repo/plots/__init__.py
--- a/dvc/repo/plots/__init__.py
+++ b/dvc/repo/plots/__init__.py
@@ -9,6 +9,13 @@
 logger = logging.getLogger(__name__)
 
 
+class NotAPlotError(DvcException):
+    def __init__(self, out):
+        super().__init__(
+            f"'{out}' is not a plot. Use `dvc plots modify` to change that."
+        )
+
+
 class Plots:
     def __init__(self, repo):
         self.repo = repo
@@ -85,6 +92,8 @@ def modify(self, path, props=None, unset=None):
         from dvc.dvcfile import Dvcfile
 
         (out,) = self.repo.find_outs_by_path(path)
+        if not out.plot and unset is not None:
+            raise NotAPlotError(out)
 
         # This out will become a plot unless it is one already
         if not isinstance(out.plot, dict):
@@ -126,9 +135,7 @@ def _targets_to_outs(targets):
 
 def _plot_props(out):
     if not out.plot:
-        raise DvcException(
-            f"'{out}' is not a plot. Use `dvc plots modify` to change that."
-        )
+        raise NotAPlotError(out)
     if isinstance(out.plot, list):
         raise DvcException("Multiple plots per data file not supported.")
     if isinstance(out.plot, bool):


Test patch:
Repository: iterative/dvc

Issue:
~/.config/dvc created to full-world writeable
## Bug Report

### Please provide information about your setup

**Output of `dvc version`:**

```console
DVC version: 0.94.0
Python version: 3.8.3
Platform: Linux-5.6.15-arch1-1-x86_64-with-glibc2.2.5
Binary: False
Package: pip
Supported remotes: http, https
Repo: dvc, git
```

**Additional Information (if any):**

creating a new repo, setting a config, the ~/.config/dvc is created with 777 (rwx for all)
it should be 770 at most.



Discussion:
@bjonnh dvc creates your config file with default permissions that are determined by the umask you have setup in your environment. This is the same as what `git` does when it creates its config file.

The only sensitive thing that might be present in a dvc config is `password` for `ssh` remote, but we do not recommend using it. You should handle it by creating a ssh keyfile or using ssh keystore.
This is really weird because my umask is not that one, it is 022
But you are running os.umask(0) in daemon.py so it overrides the system umask as well (and sets it to 777) 
and also you are changing the umask when running os.umask(0o777 - mode) in dvc/utils/fs.py

You can test that on a fresh install in a docker
docker run -it --rm ubuntu:20.04 bash
```console
 apt-get update && apt-get install -y python virtualenv git
 virtualenv venv
 source venv/bin/activate
 pip install dvc
 mkdir test && cd test &&  git init . && dvc init
 ls -ld ~/.config/dvc 
```
and you get: drwxrwxrwx 2 root root 4096 Jun 11 15:33 /root/.config/dvc
Ah, got it. I missed that you were talking about ~/.config/dvc . Indeed, that is a bug on our side!
> But you are running os.umask(0) in daemon.py so it overrides the system umask as well (and sets it to 777)

Good catch! I think it is the one that is causing the issue. Let's remove it, I think it is a leftover.

> and also you are changing the umask when running os.umask(0o777 - mode) in dvc/utils/fs.py

We are only doing that when we get a `mode` that is set to anything and we set it back after. Let me double check...

Hm, I am not able to reproduce this locally though... Looking into it.
Yep, can confirm that removing `umask` call from daemon.py fixes the bug. @bjonnh Could you update the PR, please?

Resolving patch:
diff --git a/dvc/daemon.py b/dvc/daemon.py
--- a/dvc/daemon.py
+++ b/dvc/daemon.py
@@ -46,7 +46,6 @@ def _spawn_posix(cmd, env):
         os._exit(1)  # pylint: disable=protected-access
 
     os.setsid()
-    os.umask(0)
 
     try:
         pid = os.fork()


Test patch:
Repository: iterative/dvc

Issue:
plots show unable to handle multiple aruments
In my [test repository](https://github.com/pared/plot_test) I am unable to plot multiple targets.
running `dvc plots show` yields plot with only last plot present in `dvc.yaml`. Specifying,
like `dvc plot show roc.csv loss_over_time.json` yields error:
`ERROR: unexpected error - 'roc.csv'`


Discussion:

Resolving patch:
diff --git a/dvc/repo/plots/__init__.py b/dvc/repo/plots/__init__.py
--- a/dvc/repo/plots/__init__.py
+++ b/dvc/repo/plots/__init__.py
@@ -43,7 +43,9 @@ def collect(self, targets=None, revs=None):
             plots = _collect_plots(self.repo, targets, rev)
             for path_info, props in plots.items():
                 datafile = relpath(path_info, self.repo.root_dir)
-                data[rev] = {datafile: {"props": props}}
+                if rev not in data:
+                    data[rev] = {}
+                data[rev].update({datafile: {"props": props}})
 
                 # Load data from git or dvc cache
                 try:


Test patch:
diff --git a/tests/func/plots/test_plots.py b/tests/func/plots/test_plots.py
--- a/tests/func/plots/test_plots.py
+++ b/tests/func/plots/test_plots.py
@@ -585,3 +585,24 @@ def test_plots_modify_not_existing_template(dvc):
         dvc.plots.modify(
             "metric.json", props={"template": "not-existing-template.json"}
         )
+
+
+def test_multiple_plots(tmp_dir, scm, dvc, run_copy_metrics):
+    metric1 = [
+        OrderedDict([("first_val", 100), ("second_val", 100), ("val", 2)]),
+        OrderedDict([("first_val", 200), ("second_val", 300), ("val", 3)]),
+    ]
+    metric2 = [
+        OrderedDict([("first_val", 100), ("second_val", 100), ("val", 2)]),
+        OrderedDict([("first_val", 200), ("second_val", 300), ("val", 3)]),
+    ]
+    _write_csv(metric1, "metric_t1.csv")
+    _write_json(tmp_dir, metric2, "metric_t2.json")
+    run_copy_metrics(
+        "metric_t1.csv", "metric1.csv", plots_no_cache=["metric1.csv"]
+    )
+    run_copy_metrics(
+        "metric_t2.json", "metric2.json", plots_no_cache=["metric2.json"]
+    )
+
+    assert len(dvc.plots.show().keys()) == 2
Repository: iterative/dvc

Issue:
get: directory is checked-out inside pwd instead of it's own directory
## Bug Report

```console
$ cd "$(mktemp -d)"
$ dvc get https://github.com/schacon/cowsay
$ ls
beavis.zen.cow  daemon.cow             eyes.cow           kitty.cow       milk.cow       sheep.cow        stimpy.cow       turtle.cow
bong.cow        default.cow            flaming-sheep.cow  koala.cow       moofasa.cow    skeleton.cow     supermilker.cow  tux.cow
bud-frogs.cow   dragon-and-cow.cow     ghostbusters.cow   kosh.cow        moose.cow      small.cow        surgery.cow      udder.cow
bunny.cow       dragon.cow             head-in.cow        luke-koala.cow  mutilated.cow  sodomized.cow    telebears.cow    vader.cow
cheese.cow      elephant.cow           hellokitty.cow     mech-and-cow    ren.cow        squirrel.cow     three-eyes.cow   vader-koala.cow
cower.cow       elephant-in-snake.cow  kiss.cow           meow.cow        satanic.cow    stegosaurus.cow  turkey.cow       www.cow
$ rm -rf *cow
$ git checkout 0.94.0
$ dvc get https://github.com/schacon/cowsay
$ ls
cows
```
`git bisect` points me to #3883, so the regression is after `v1.0.0a7`.



Discussion:

Resolving patch:
diff --git a/dvc/repo/tree.py b/dvc/repo/tree.py
--- a/dvc/repo/tree.py
+++ b/dvc/repo/tree.py
@@ -404,10 +404,8 @@ def copytree(self, top, dest):
 
         for root, _, files in self.walk(top):
             root = PathInfo(root)
-            dest_dir = root.relative_to(top)
-            makedirs(dest_dir, exist_ok=True)
+            makedirs(dest, exist_ok=True)
             for fname in files:
                 src = root / fname
-                dest = dest_dir / fname
                 with self.open(src, mode="rb") as fobj:
-                    copy_fobj_to_file(fobj, dest)
+                    copy_fobj_to_file(fobj, dest / fname)


Test patch:
diff --git a/tests/func/test_get.py b/tests/func/test_get.py
--- a/tests/func/test_get.py
+++ b/tests/func/test_get.py
@@ -33,7 +33,9 @@ def test_get_repo_dir(tmp_dir, erepo_dir):
     trees_equal(erepo_dir / "dir", "dir_imported")
 
 
-def test_get_git_file(tmp_dir, erepo_dir):
+@pytest.mark.parametrize("repo_type", ["git_dir", "erepo_dir"])
+def test_get_git_file(request, tmp_dir, repo_type):
+    erepo_dir = request.getfixturevalue(repo_type)
     src = "some_file"
     dst = "some_file_imported"
 
@@ -45,7 +47,9 @@ def test_get_git_file(tmp_dir, erepo_dir):
     assert (tmp_dir / dst).read_text() == "hello"
 
 
-def test_get_git_dir(tmp_dir, erepo_dir):
+@pytest.mark.parametrize("repo_type", ["git_dir", "erepo_dir"])
+def test_get_git_dir(request, tmp_dir, repo_type):
+    erepo_dir = request.getfixturevalue(repo_type)
     src = "some_directory"
     dst = "some_directory_imported"
Repository: iterative/dvc

Issue:
get/import: progress bar is broken
We broke the progress bar for the get/import. It is now `Downloading 0/1 files` vs `Downloading 0/36.2M` before. Mean that we effectively don't show a progress bar for the case like:

```sh
dvc get https://github.com/iterative/dataset-registry get-started/data.xml -o data/data.xml
```


*Originally posted by @shcheklein in #3777*



Discussion:

Resolving patch:
diff --git a/dvc/remote/local.py b/dvc/remote/local.py
--- a/dvc/remote/local.py
+++ b/dvc/remote/local.py
@@ -626,8 +626,12 @@ def _process(
             func = pbar.wrap_fn(func)
             with ThreadPoolExecutor(max_workers=jobs) as executor:
                 if download:
-                    fails = sum(executor.map(func, *dir_plans))
-                    fails += sum(executor.map(func, *file_plans))
+                    from_infos, to_infos, names, _ = (
+                        d + f for d, f in zip(dir_plans, file_plans)
+                    )
+                    fails = sum(
+                        executor.map(func, from_infos, to_infos, names)
+                    )
                 else:
                     # for uploads, push files first, and any .dir files last
 


Test patch: