Repository: iterative/dvc

Issue:
migrate from travis to github actions

Discussion:
skshetry: ```
E       AssertionError: assert {'version.dvc...available'}}]} == {'dir.dvc': [...available'}}]}
E         Omitting 1 identical items, use -vv to show
E         Right contains 1 more item:
E         {'dir.dvc': [{'changed deps': {'dir (/tmp/pytest-of-runner/pytest-0/popen-gw2/erepo25)': 'update '
E                                                                                                  'available'}}]}
E         Full diff:
E           {
E         -  'dir.dvc': [{'changed deps': {'dir (/tmp/pytest-of-runner/pytest-0/popen-gw2/erepo25)': 'update '
E         -                                                                                          'available'}}],
E            'version.dvc': [{'changed deps': {'version (/tmp/pytest-of-runner/pytest-0/popen-gw2/erepo25)': 'update '
E                                                                                                            'available'}}],
E           }
```

This test fails from time to time. It's very hard to reproduce, and I have tried to debug on failure, but haven't had much luck.

Once when I was able to reproduce, I debugged through it and found that somehow the "dir.dvc"'s `Output::checksum` gets changed when calculating the checksum, so when it later compares, it does not find any differences at all.
But I couldn't figure out why/when that happened.


pared: @skshetry thanks for this info, that saved me a lot of time for sure :)
efiop: This is great :fire:

Oh, I see only windows 3.8 fails. Could create a p0 ticket and just skip 3.8 in this PR for now. Could  also mark it as "allow_failure" so we have a constant reminder :slightly_smiling_face: 
codecov[bot]: # [Codecov](https://codecov.io/gh/iterative/dvc/pull/4469?src=pr&el=h1) Report
> Merging [#4469](https://codecov.io/gh/iterative/dvc/pull/4469?src=pr&el=desc) into [master](https://codecov.io/gh/iterative/dvc/commit/226322d31253e65536703b36b542dfaabb4381ed?el=desc) will **increase** coverage by `0.58%`.
> The diff coverage is `95.83%`.

[![Impacted file tree graph](https://codecov.io/gh/iterative/dvc/pull/4469/graphs/tree.svg?width=650&height=150&src=pr&token=VOfa1EJWC4)](https://codecov.io/gh/iterative/dvc/pull/4469?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4469      +/-   ##
==========================================
+ Coverage   88.76%   89.34%   +0.58%     
==========================================
  Files         185      171      -14     
  Lines       13082    11755    -1327     
==========================================
- Hits        11612    10503    -1109     
+ Misses       1470     1252     -218     
```


| [Impacted Files](https://codecov.io/gh/iterative/dvc/pull/4469?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [dvc/tree/dvc.py](https://codecov.io/gh/iterative/dvc/pull/4469/diff?src=pr&el=tree#diff-ZHZjL3RyZWUvZHZjLnB5) | `95.85% <ø> (-0.05%)` | :arrow_down: |
| [dvc/tree/repo.py](https://codecov.io/gh/iterative/dvc/pull/4469/diff?src=pr&el=tree#diff-ZHZjL3RyZWUvcmVwby5weQ==) | `91.88% <83.33%> (-0.33%)` | :arrow_down: |
| [dvc/dvcfile.py](https://codecov.io/gh/iterative/dvc/pull/4469/diff?src=pr&el=tree#diff-ZHZjL2R2Y2ZpbGUucHk=) | `97.48% <100.00%> (ø)` | |
| [dvc/repo/checkout.py](https://codecov.io/gh/iterative/dvc/pull/4469/diff?src=pr&el=tree#diff-ZHZjL3JlcG8vY2hlY2tvdXQucHk=) | `97.87% <100.00%> (+0.09%)` | :arrow_up: |
| [dvc/repo/commit.py](https://codecov.io/gh/iterative/dvc/pull/4469/diff?src=pr&el=tree#diff-ZHZjL3JlcG8vY29tbWl0LnB5) | `93.33% <100.00%> (ø)` | |
| [dvc/repo/experiments/\_\_init\_\_.py](https://codecov.io/gh/iterative/dvc/pull/4469/diff?src=pr&el=tree#diff-ZHZjL3JlcG8vZXhwZXJpbWVudHMvX19pbml0X18ucHk=) | `87.74% <100.00%> (ø)` | |
| [dvc/repo/fetch.py](https://codecov.io/gh/iterative/dvc/pull/4469/diff?src=pr&el=tree#diff-ZHZjL3JlcG8vZmV0Y2gucHk=) | `92.68% <100.00%> (+0.37%)` | :arrow_up: |
| [dvc/repo/freeze.py](https://codecov.io/gh/iterative/dvc/pull/4469/diff?src=pr&el=tree#diff-ZHZjL3JlcG8vZnJlZXplLnB5) | `100.00% <100.00%> (ø)` | |
| [dvc/repo/pull.py](https://codecov.io/gh/iterative/dvc/pull/4469/diff?src=pr&el=tree#diff-ZHZjL3JlcG8vcHVsbC5weQ==) | `100.00% <100.00%> (ø)` | |
| [dvc/repo/reproduce.py](https://codecov.io/gh/iterative/dvc/pull/4469/diff?src=pr&el=tree#diff-ZHZjL3JlcG8vcmVwcm9kdWNlLnB5) | `95.00% <100.00%> (ø)` | |
| ... and [32 more](https://codecov.io/gh/iterative/dvc/pull/4469/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/iterative/dvc/pull/4469?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/iterative/dvc/pull/4469?src=pr&el=footer). Last update [2931d30...e024265](https://codecov.io/gh/iterative/dvc/pull/4469?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

efiop: @pared Is this WIP?
pared: @efiop yes

Patch:
diff --git a/.travis.yml b/.travis.yml
index 1dd401c1b..052574898 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -32,32 +32,6 @@ jobs:
     before_install: bash ./scripts/ci/install.sh
     install:
     script: ./scripts/ci/check_patch.sh
-  # test jobs
-  - name: "3.7 on Windows"
-    stage: test
-    os: windows           # Windows 10.0.17134 N/A Build 17134
-    language: shell       # 'language: python' is an error on Travis CI Windows
-    env:
-      - PYTHON_VERSION=3.7.5
-      - PATH=/c/Python37:/c/Python37/Scripts:$PATH
-  - name: "3.8 on Windows"
-    os: windows           # Windows 10.0.17134 N/A Build 17134
-    language: shell       # 'language: python' is an error on Travis CI Windows
-    env:
-      - PYTHON_VERSION=3.8.0
-      - PATH=/c/Python38:/c/Python38/Scripts:$PATH
-  - name: "3.6 on Linux"
-    language: python
-    python: 3.6
-  - name: "3.7 on Linux"
-    language: python
-    python: 3.7
-  - name: "3.8 on Linux"
-    language: python
-    python: 3.8
-  - name: "3.7 on Mac"
-    os: osx
-    osx_image: xcode11.3
   # build jobs
   - name: "Mac pkg"
     stage: build
diff --git a/dvc/api.py b/dvc/api.py
index 5bf1517af..bf79b682e 100644
--- a/dvc/api.py
+++ b/dvc/api.py
@@ -34,7 +34,8 @@ def get_url(path, repo=None, rev=None, remote=None):
             raise OutputNotFoundError(path, repo)
 
         cloud = metadata.repo.cloud
-        hash_info = _repo.repo_tree.get_hash(path_info)
+        with _repo.state:
+            hash_info = _repo.repo_tree.get_hash(path_info)
         return cloud.get_url_for(remote, checksum=hash_info.value)
 
 
diff --git a/dvc/config.py b/dvc/config.py
index 50bcfddc8..a01e39291 100644
--- a/dvc/config.py
+++ b/dvc/config.py
@@ -139,6 +139,7 @@ SCHEMA = {
         Optional("analytics", default=True): Bool,
         Optional("hardlink_lock", default=False): Bool,
         Optional("no_scm", default=False): Bool,
+        Optional("autostage", default=False): Bool,
         Optional("experiments", default=False): Bool,
         Optional("check_update", default=True): Bool,
     },
diff --git a/dvc/repo/experiments/__init__.py b/dvc/repo/experiments/__init__.py
index c57a24221..2a5e18f42 100644
--- a/dvc/repo/experiments/__init__.py
+++ b/dvc/repo/experiments/__init__.py
@@ -2,6 +2,7 @@ import logging
 import os
 import re
 import tempfile
+from collections import namedtuple
 from collections.abc import Mapping
 from concurrent.futures import (
     ProcessPoolExecutor,
@@ -72,14 +73,17 @@ class Experiments:
 
     EXPERIMENTS_DIR = "experiments"
     PACKED_ARGS_FILE = "repro.dat"
-    STASH_MSG_PREFIX = "dvc-exp-"
+    STASH_MSG_PREFIX = "dvc-exp:"
     STASH_EXPERIMENT_RE = re.compile(
-        r"(?:On \(.*\): )dvc-exp-(?P<baseline_rev>[0-9a-f]+)$"
+        r"(?:On \(.*\): )"
+        r"dvc-exp:(?P<baseline_rev>[0-9a-f]+)(:(?P<branch>.+))?$"
     )
     BRANCH_RE = re.compile(
         r"^(?P<baseline_rev>[a-f0-9]{7})-(?P<exp_sha>[a-f0-9]+)$"
     )
 
+    StashEntry = namedtuple("StashEntry", ["index", "baseline_rev", "branch"])
+
     def __init__(self, repo):
         from dvc.lock import make_lock
 
@@ -144,7 +148,9 @@ class Experiments:
         for i, entry in enumerate(self.stash_reflog):
             m = self.STASH_EXPERIMENT_RE.match(entry.message)
             if m:
-                revs[entry.newhexsha] = (i, m.group("baseline_rev"))
+                revs[entry.newhexsha] = self.StashEntry(
+                    i, m.group("baseline_rev"), m.group("branch")
+                )
         return revs
 
     def _init_clone(self):
@@ -184,30 +190,54 @@ class Experiments:
             branch.set_tracking_branch(ref)
         branch.checkout()
 
-    def _stash_exp(self, *args, params: Optional[dict] = None, **kwargs):
+    def _stash_exp(
+        self,
+        *args,
+        params: Optional[dict] = None,
+        branch: Optional[str] = None,
+        allow_unchanged: Optional[bool] = False,
+        apply_workspace: Optional[bool] = True,
+        **kwargs,
+    ):
         """Stash changes from the current (parent) workspace as an experiment.
 
         Args:
             params: Optional dictionary of parameter values to be used.
                 Values take priority over any parameters specified in the
                 user's workspace.
+            branch: Optional experiment branch name. If specified, the
+                experiment will be added to `branch` instead of creating
+                a new branch.
+            allow_unchanged: Force experiment reproduction even if params are
+                unchanged from the baseline.
+            apply_workspace: Apply changes from the user workspace to the
+                experiment workspace.
         """
-        rev = self.scm.get_rev()
-
-        # patch user's workspace into experiments clone
-        tmp = tempfile.NamedTemporaryFile(delete=False).name
-        try:
-            self.repo.scm.repo.git.diff(
-                patch=True, full_index=True, binary=True, output=tmp
-            )
-            if os.path.getsize(tmp):
-                logger.debug("Patching experiment workspace")
-                self.scm.repo.git.apply(tmp)
-            elif not params:
-                # experiment matches original baseline
-                raise UnchangedExperimentError(rev)
-        finally:
-            remove(tmp)
+        if branch:
+            rev = self.scm.resolve_rev(branch)
+        else:
+            rev = self.scm.get_rev()
+
+        if apply_workspace:
+            # patch user's workspace into experiments clone
+            #
+            # TODO: patching changes into an extended branch will require
+            # propagating changes down the full branch, since the workspace
+            # patch will be based on the original branch point, not the tip
+            # of the experiment branch
+            tmp = tempfile.NamedTemporaryFile(delete=False).name
+            try:
+                self.repo.scm.repo.git.diff(
+                    patch=True, full_index=True, binary=True, output=tmp
+                )
+                if os.path.getsize(tmp):
+                    logger.debug("Patching experiment workspace")
+                    self.scm.repo.git.apply(tmp)
+                elif not params and not allow_unchanged:
+                    # experiment matches original baseline
+                    raise UnchangedExperimentError(rev)
+            finally:
+                remove(tmp)
 
         # update experiment params from command line
         if params:
@@ -219,10 +249,15 @@ class Experiments:
         # save experiment as a stash commit w/message containing baseline rev
         # (stash commits are merge commits and do not contain a parent commit
         # SHA)
-        msg = f"{self.STASH_MSG_PREFIX}{rev}"
+        msg = self._stash_msg(rev, branch)
         self.scm.repo.git.stash("push", "-m", msg)
         return self.scm.resolve_rev("stash@{0}")
 
+    def _stash_msg(self, rev, branch=None):
+        if branch:
+            return f"{self.STASH_MSG_PREFIX}{rev}:{branch}"
+        return f"{self.STASH_MSG_PREFIX}{rev}"
+
     def _pack_args(self, *args, **kwargs):
         ExperimentExecutor.pack_repro_args(self.args_file, *args, **kwargs)
         self.scm.add(self.args_file)
@@ -252,19 +287,21 @@ class Experiments:
             with modify_data(path, tree=self.exp_dvc.tree) as data:
                 _update(data, params[params_fname])
 
-    def _commit(self, exp_hash, check_exists=True, branch=True):
+    def _commit(self, exp_hash, check_exists=True, create_branch=True):
         """Commit stages as an experiment and return the commit SHA."""
         if not self.scm.is_dirty():
             raise UnchangedExperimentError(self.scm.get_rev())
 
         rev = self.scm.get_rev()
         exp_name = f"{rev[:7]}-{exp_hash}"
-        if branch:
+        if create_branch:
             if check_exists and exp_name in self.scm.list_branches():
                 logger.debug("Using existing experiment branch '%s'", exp_name)
                 return self.scm.resolve_rev(exp_name)
             self.scm.checkout(exp_name, create_new=True)
-        logger.debug("Commit new experiment branch '%s'", exp_name)
+            logger.debug("Commit new experiment branch '%s'", exp_name)
+        else:
+            logger.debug("Commit to current experiment branch")
         self.scm.repo.git.add(A=True)
         self.scm.commit(f"Add experiment {exp_name}")
         return self.scm.get_rev()
@@ -302,7 +339,14 @@ class Experiments:
         Experiment will be reproduced and checked out into the user's
         workspace.
         """
-        rev = self.repo.scm.get_rev()
+        branch = kwargs.get("branch")
+        if branch:
+            rev = self.scm.resolve_rev(branch)
+            logger.debug(
+                "Using '%s' (tip of branch '%s') as baseline", rev, branch
+            )
+        else:
+            rev = self.repo.scm.get_rev()
         self._scm_checkout(rev)
         try:
             stash_rev = self._stash_exp(*args, **kwargs)
@@ -337,13 +381,12 @@ class Experiments:
         # commit) and baseline_rev is the baseline to compare output against.
         # The final experiment commit will be branched from baseline_rev.
         if revs is None:
-            to_run = {
-                rev: baseline_rev
-                for rev, (_, baseline_rev) in stash_revs.items()
-            }
+            to_run = dict(stash_revs)
         else:
             to_run = {
-                rev: stash_revs[rev][1] if rev in stash_revs else rev
+                rev: stash_revs[rev]
+                if rev in stash_revs
+                else self.StashEntry(None, rev, None)
                 for rev in revs
             }
 
@@ -355,12 +398,13 @@ class Experiments:
         # setup executors - unstash experiment, generate executor, upload
         # contents of (unstashed) exp workspace to the executor tree
         executors = {}
-        for rev, baseline_rev in to_run.items():
-            self._scm_checkout(baseline_rev)
+        for rev, item in to_run.items():
+            self._scm_checkout(item.baseline_rev)
             self.scm.repo.git.stash("apply", rev)
             repro_args, repro_kwargs = self._unpack_args()
             executor = LocalExecutor(
-                baseline_rev,
+                item.baseline_rev,
+                branch=item.branch,
                 repro_args=repro_args,
                 repro_kwargs=repro_kwargs,
                 dvc_dir=self.dvc_dir,
@@ -417,7 +461,10 @@ class Experiments:
                 exc = future.exception()
                 if exc is None:
                     exp_hash = future.result()
-                    self._scm_checkout(executor.baseline_rev)
+                    if executor.branch:
+                        self._scm_checkout(executor.branch)
+                    else:
+                        self._scm_checkout(executor.baseline_rev)
                     try:
                         self._collect_output(executor)
                     except DownloadError:
@@ -431,7 +478,8 @@ class Experiments:
                             remove(self.args_file)
 
                     try:
-                        exp_rev = self._commit(exp_hash)
+                        branch = not executor.branch
+                        exp_rev = self._commit(exp_hash, create_branch=branch)
                     except UnchangedExperimentError:
                         logger.debug(
                             "Experiment '%s' identical to baseline '%s'",
@@ -517,12 +565,12 @@ class Experiments:
 
         from dvc.repo.checkout import checkout as dvc_checkout
 
-        self._check_baseline(rev)
+        baseline_rev = self._check_baseline(rev)
         self._scm_checkout(rev)
 
         tmp = tempfile.NamedTemporaryFile(delete=False).name
         self.scm.repo.head.commit.diff(
-            "HEAD~1", patch=True, full_index=True, binary=True, output=tmp
+            baseline_rev, patch=True, full_index=True, binary=True, output=tmp
         )
 
         dirty = self.repo.scm.is_dirty()
@@ -549,11 +597,14 @@ class Experiments:
 
     def _check_baseline(self, exp_rev):
         baseline_sha = self.repo.scm.get_rev()
-        exp_commit = self.scm.repo.rev_parse(exp_rev)
-        parent = first(exp_commit.parents)
-        if parent is not None and parent.hexsha == baseline_sha:
-            return
-        raise BaselineMismatchError(parent, baseline_sha)
+        exp_baseline = self._get_baseline(exp_rev)
+        if exp_baseline is None:
+            # if we can't tell from branch name, fall back to parent commit
+            exp_commit = self.scm.repo.rev_parse(exp_rev)
+            exp_baseline = first(exp_commit.parents).hexsha
+        if exp_baseline == baseline_sha:
+            return exp_baseline
+        raise BaselineMismatchError(exp_baseline, baseline_sha)
 
     def _unstash_workspace(self):
         # Essentially we want `git stash pop` with `-X ours` merge strategy
@@ -581,6 +632,9 @@ class Experiments:
     @scm_locked
     def get_baseline(self, rev):
         """Return the baseline rev for an experiment rev."""
+        return self._get_baseline(rev)
+
+    def _get_baseline(self, rev):
         from git.exc import GitCommandError
 
         rev = self.scm.resolve_rev(rev)
@@ -588,16 +642,28 @@ class Experiments:
             name = self.scm.repo.git.name_rev(rev, name_only=True)
         except GitCommandError:
             return None
-        if not name:
-            return None
         if name in ("undefined", "stash"):
-            _, baseline = self.stash_revs.get(rev, (None, None))
-            return baseline
+            entry = self.stash_revs.get(rev)
+            if entry:
+                return entry.baseline_rev
+            return None
         m = self.BRANCH_RE.match(name)
         if m:
             return self.scm.resolve_rev(m.group("baseline_rev"))
         return None
 
+    def _get_branch_containing(self, rev):
+        from git.exc import GitCommandError
+
+        if self.scm.repo.head.is_detached:
+            self._checkout_default_branch()
+        try:
+            name = self.scm.repo.git.branch(contains=rev)
+            return name.rsplit("/")[-1].strip()
+        except GitCommandError:
+            pass
+        return None
+
     def checkout(self, *args, **kwargs):
         from dvc.repo.experiments.checkout import checkout
 
diff --git a/dvc/repo/experiments/executor.py b/dvc/repo/experiments/executor.py
index 21ce60142..725c01e5f 100644
--- a/dvc/repo/experiments/executor.py
+++ b/dvc/repo/experiments/executor.py
@@ -2,7 +2,7 @@ import logging
 import os
 import pickle
 from tempfile import TemporaryDirectory
-from typing import Iterable
+from typing import Iterable, Optional
 
 from funcy import cached_property
 
@@ -22,12 +22,16 @@ class ExperimentExecutor:
         baseline_rev: baseline revision that this experiment is derived from.
 
     Optional keyword args:
+        branch: Existing git branch for this experiment.
         repro_args: Args to be passed into reproduce.
         repro_kwargs: Keyword args to be passed into reproduce.
     """
 
-    def __init__(self, baseline_rev: str, **kwargs):
+    def __init__(
+        self, baseline_rev: str, branch: Optional[str] = None, **kwargs,
+    ):
         self.baseline_rev = baseline_rev
+        self.branch = branch
         self.repro_args = kwargs.pop("repro_args", [])
         self.repro_kwargs = kwargs.pop("repro_kwargs", {})
 
diff --git a/dvc/repo/experiments/show.py b/dvc/repo/experiments/show.py
index 6a2a6d50e..1175d4570 100644
--- a/dvc/repo/experiments/show.py
+++ b/dvc/repo/experiments/show.py
@@ -87,12 +87,12 @@ def show(
                 res[rev][exp_rev] = experiment
 
     # collect queued (not yet reproduced) experiments
-    for stash_rev, (_, baseline_rev) in repo.experiments.stash_revs.items():
-        if baseline_rev in revs:
+    for stash_rev, entry in repo.experiments.stash_revs.items():
+        if entry.baseline_rev in revs:
             with repo.experiments.chdir():
                 experiment = _collect_experiment(
                     repo.experiments.exp_dvc, stash_rev, stash=True
                 )
-            res[baseline_rev][stash_rev] = experiment
+            res[entry.baseline_rev][stash_rev] = experiment
 
     return res
diff --git a/dvc/repo/metrics/show.py b/dvc/repo/metrics/show.py
index 0913a5971..d0e7b07ae 100644
--- a/dvc/repo/metrics/show.py
+++ b/dvc/repo/metrics/show.py
@@ -11,28 +11,31 @@ logger = logging.getLogger(__name__)
 
 
 def _collect_metrics(repo, targets, recursive):
-    metrics = set()
 
+    if targets:
+        target_infos = [
+            PathInfo(os.path.abspath(target)) for target in targets
+        ]
+        tree = RepoTree(repo)
+
+        rec_files = []
+        if recursive:
+            for target_info in target_infos:
+                if tree.isdir(target_info):
+                    rec_files.extend(list(tree.walk_files(target_info)))
+
+        result = [t for t in target_infos if tree.isfile(t)]
+        result.extend(rec_files)
+
+        return result
+
+    metrics = set()
     for stage in repo.stages:
         for out in stage.outs:
             if not out.metric:
                 continue
-
             metrics.add(out.path_info)
-
-    if not targets:
-        return list(metrics)
-
-    target_infos = [PathInfo(os.path.abspath(target)) for target in targets]
-
-    def _filter(path_info):
-        for info in target_infos:
-            func = path_info.isin_or_eq if recursive else path_info.__eq__
-            if func(info):
-                return True
-        return False
-
-    return list(filter(_filter, metrics))
+    return list(metrics)
 
 
 def _extract_metrics(metrics, path, rev):
diff --git a/dvc/repo/scm_context.py b/dvc/repo/scm_context.py
index c0b0ffb5b..ec9585bb7 100644
--- a/dvc/repo/scm_context.py
+++ b/dvc/repo/scm_context.py
@@ -3,7 +3,11 @@ def scm_context(method):
         try:
             result = method(repo, *args, **kw)
             repo.scm.reset_ignores()
-            repo.scm.remind_to_track()
+            autostage = repo.config.get("core", {}).get("autostage", False)
+            if autostage:
+                repo.scm.track_changed_files()
+            else:
+                repo.scm.remind_to_track()
             repo.scm.reset_tracked_files()
             return result
         except Exception:
diff --git a/dvc/scm/base.py b/dvc/scm/base.py
index 1bde8f744..699c602e6 100644
--- a/dvc/scm/base.py
+++ b/dvc/scm/base.py
@@ -154,6 +154,11 @@ class Base:
         Method to remind user to track newly created files handled by scm
         """
 
+    def track_changed_files(self):
+        """
+        Method to stage files that have changed
+        """
+
     def track_file(self, path):
         """
         Method to add file to mechanism that will remind user
diff --git a/dvc/scm/git.py b/dvc/scm/git.py
index b777488ac..a3317218e 100644
--- a/dvc/scm/git.py
+++ b/dvc/scm/git.py
@@ -381,6 +381,12 @@ class Git(Base):
             "\tgit add {files}".format(files=files)
         )
 
+    def track_changed_files(self):
+        if not self.files_to_track:
+            return
+
+        self.add(self.files_to_track)
+
     def track_file(self, path):
         self.files_to_track.add(path)
 
diff --git a/dvc/tree/azure.py b/dvc/tree/azure.py
index 1aaa16b7a..3a14aac33 100644
--- a/dvc/tree/azure.py
+++ b/dvc/tree/azure.py
@@ -67,7 +67,10 @@ class AzureTree(BaseTree):
     @cached_property
     def blob_service(self):
         # pylint: disable=no-name-in-module
-        from azure.core.exceptions import ResourceNotFoundError
+        from azure.core.exceptions import (
+            HttpResponseError,
+            ResourceNotFoundError,
+        )
         from azure.storage.blob import BlobServiceClient
 
         logger.debug(f"URL {self.path_info}")
@@ -92,6 +95,10 @@ class AzureTree(BaseTree):
             container_client.get_container_properties()
         except ResourceNotFoundError:
             container_client.create_container()
+        except HttpResponseError as exc:
+            # client may not have account-level privileges
+            if exc.status_code != 403:
+                raise
 
         return blob_service
 
diff --git a/dvc/tree/gdrive.py b/dvc/tree/gdrive.py
index 12779a5c3..302a5ccda 100644
--- a/dvc/tree/gdrive.py
+++ b/dvc/tree/gdrive.py
@@ -366,10 +366,7 @@ class GDriveTree(BaseTree):
                 total=total,
                 disable=no_progress_bar,
             ) as wrapped:
-                # PyDrive doesn't like content property setting for empty files
-                # https://github.com/gsuitedevs/PyDrive/issues/121
-                if total:
-                    item.content = wrapped
+                item.content = wrapped
                 item.Upload()
         return item
 
@@ -381,9 +378,6 @@ class GDriveTree(BaseTree):
         # it does not create a file on the remote
         gdrive_file = self._drive.CreateFile(param)
 
-        gdrive_file.FetchMetadata(fields="fileSize")
-        size = int(gdrive_file["fileSize"])
-
         with Tqdm(
             desc=progress_desc,
             disable=no_progress_bar,
@@ -391,13 +385,7 @@ class GDriveTree(BaseTree):
             # explicit `bar_format` as `total` will be set by `update_to`
             bar_format=Tqdm.BAR_FMT_DEFAULT,
         ) as pbar:
-            if size:
-                gdrive_file.GetContentFile(to_file, callback=pbar.update_to)
-            else:
-                # PyDrive doesn't like downloading empty files
-                # https://github.com/iterative/dvc/issues/4286
-                with open(to_file, "w"):
-                    pass
+            gdrive_file.GetContentFile(to_file, callback=pbar.update_to)
 
     @contextmanager
     @_gdrive_retry
diff --git a/dvc/version.py b/dvc/version.py
index b3b458f85..5f74f0b12 100644
--- a/dvc/version.py
+++ b/dvc/version.py
@@ -6,7 +6,7 @@
 import os
 import subprocess
 
-_BASE_VERSION = "1.7.2"
+_BASE_VERSION = "1.7.4"
 
 
 def _generate_version(base_version):
diff --git a/setup.py b/setup.py
index ae7200af3..3611a3adb 100644
--- a/setup.py
+++ b/setup.py
@@ -88,7 +88,7 @@ install_requires = [
 # Extra dependencies for remote integrations
 
 gs = ["google-cloud-storage==1.19.0"]
-gdrive = ["pydrive2>=1.6.1", "six >= 1.13.0"]
+gdrive = ["pydrive2>=1.6.2", "six >= 1.13.0"]
 s3 = ["boto3>=1.9.201"]
 azure = ["azure-storage-blob>=12.0", "knack"]
 oss = ["oss2==2.6.1"]
@@ -107,6 +107,7 @@ tests_requirements = [
     "wheel>=0.31.1",
     # Test requirements:
     "pytest>=6.0.1",
+    "pytest-cov",
     "pytest-docker>=0.7.2",
     "pytest-timeout>=1.3.3",
     "pytest-cov>=2.6.1",


Test patch:
diff --git a/.github/workflows/tests.yaml b/.github/workflows/tests.yaml
new file mode 100644
index 000000000..2ac73bbb9
--- /dev/null
+++ b/.github/workflows/tests.yaml
@@ -0,0 +1,42 @@
+name: Tests build
+
+on: [push, pull_request]
+
+env:
+  DVC_TEST: "true"
+  HOMEBREW_NO_AUTO_UPDATE: 1
+  SHELL: /bin/bash
+
+jobs:
+  tests:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ubuntu-18.04, windows-2019, macos-10.15]
+        pyv: [3.6, 3.7, 3.8]
+        exclude:
+          - os: windows-2019
+            pyv: 3.8
+    steps:
+    - uses: actions/checkout@v2.3.2
+    - name: Set up Python
+      uses: actions/setup-python@v2.1.2
+      with:
+        python-version: ${{ matrix.pyv }}
+    - name: install
+      run: |
+        pip install ".[all,tests]"
+    - name: setup git
+      run: |
+        git config --global user.email "dvctester@example.com"
+        git config --global user.name "DVC Tester"
+    - name: run tests
+      run: python -m tests -n=2 --cov-report=xml
+    - name: upload coverage report
+      if: github.event.name == 'push'
+      uses: codecov/codecov-action@v1.0.13
+      with:
+        file: ./coverage.xml
+        fail_ci_if_error: true
+
+
diff --git a/tests/func/experiments/test_experiments.py b/tests/func/experiments/test_experiments.py
index 7ebf71a3f..b11a277f5 100644
--- a/tests/func/experiments/test_experiments.py
+++ b/tests/func/experiments/test_experiments.py
@@ -100,3 +100,39 @@ def test_get_baseline(tmp_dir, scm, dvc):
         stage.addressing, experiment=True, params=["foo=3"], queue=True
     )
     assert dvc.experiments.get_baseline("stash@{0}") == expected
+
+
+def test_extend_branch(tmp_dir, scm, dvc):
+    tmp_dir.gen("copy.py", COPY_SCRIPT)
+    tmp_dir.gen("params.yaml", "foo: 1")
+    stage = dvc.run(
+        cmd="python copy.py params.yaml metrics.yaml",
+        metrics_no_cache=["metrics.yaml"],
+        params=["foo"],
+        name="copy-file",
+    )
+    scm.add(["dvc.yaml", "dvc.lock", "copy.py", "params.yaml", "metrics.yaml"])
+    scm.commit("init")
+
+    dvc.reproduce(stage.addressing, experiment=True, params=["foo=2"])
+    exp_a = dvc.experiments.scm.get_rev()
+    exp_branch = dvc.experiments._get_branch_containing(exp_a)
+
+    dvc.reproduce(
+        stage.addressing,
+        experiment=True,
+        params=["foo=3"],
+        branch=exp_branch,
+        apply_workspace=False,
+    )
+    exp_b = dvc.experiments.scm.get_rev()
+
+    assert dvc.experiments._get_branch_containing(exp_b) == exp_branch
+
+    dvc.experiments.checkout(exp_a)
+    assert (tmp_dir / "params.yaml").read_text().strip() == "foo: 2"
+    assert (tmp_dir / "metrics.yaml").read_text().strip() == "foo: 2"
+
+    dvc.experiments.checkout(exp_b)
+    assert (tmp_dir / "params.yaml").read_text().strip() == "foo: 3"
+    assert (tmp_dir / "metrics.yaml").read_text().strip() == "foo: 3"
diff --git a/tests/func/metrics/test_diff.py b/tests/func/metrics/test_diff.py
index c7ea99bf3..8d00c2d81 100644
--- a/tests/func/metrics/test_diff.py
+++ b/tests/func/metrics/test_diff.py
@@ -195,3 +195,17 @@ def test_metrics_diff_cli(tmp_dir, scm, dvc, run_copy_metrics, caplog, capsys):
         "Path    Metric    Old      New      Change\n"
         "m.yaml  foo       1.23457  3.45679  2.22222"
     )
+
+
+def test_metrics_diff_non_metrics(tmp_dir, scm, dvc):
+    def _gen(val):
+        tmp_dir.scm_gen({"some_file.yaml": f"foo: {val}"}, commit=str(val))
+
+    _gen(1)
+    _gen(2)
+    _gen(3)
+
+    result = dvc.metrics.diff(targets=["some_file.yaml"], a_rev="HEAD~2")
+    assert result == {
+        "some_file.yaml": {"foo": {"old": 1, "new": 3, "diff": 2}}
+    }
diff --git a/tests/func/metrics/test_show.py b/tests/func/metrics/test_show.py
index 69414b4e9..bc93f4aef 100644
--- a/tests/func/metrics/test_show.py
+++ b/tests/func/metrics/test_show.py
@@ -110,3 +110,41 @@ def test_missing_cache(tmp_dir, dvc, run_copy_metrics):
     remove(stage.outs[0].cache_path)
 
     assert dvc.metrics.show() == {"": {"metrics.yaml": 1.1}}
+
+
+def test_show_non_metric(tmp_dir, dvc):
+    tmp_dir.gen("metrics.yaml", "foo: 1.1")
+
+    assert dvc.metrics.show(targets=["metrics.yaml"]) == {
+        "": {"metrics.yaml": {"foo": 1.1}}
+    }
+
+
+def test_show_non_metric_branch(tmp_dir, scm, dvc):
+    tmp_dir.scm_gen("metrics.yaml", "foo: 1.1", commit="init")
+    with tmp_dir.branch("branch", new=True):
+        tmp_dir.scm_gen("metrics.yaml", "foo: 2.2", commit="other")
+
+    assert dvc.metrics.show(targets=["metrics.yaml"], revs=["branch"]) == {
+        "workspace": {"metrics.yaml": {"foo": 1.1}},
+        "branch": {"metrics.yaml": {"foo": 2.2}},
+    }
+
+
+def test_non_metric_and_recurisve_show(tmp_dir, dvc, run_copy_metrics):
+    tmp_dir.gen(
+        {"metrics_t.yaml": "foo: 1.1", "metrics": {"metric1.yaml": "bar: 1.2"}}
+    )
+
+    metric2 = os.fspath(tmp_dir / "metrics" / "metric2.yaml")
+    run_copy_metrics("metrics_t.yaml", metric2, metrics=[metric2])
+
+    assert dvc.metrics.show(
+        targets=["metrics_t.yaml", "metrics"], recursive=True
+    ) == {
+        "": {
+            os.path.join("metrics", "metric1.yaml"): {"bar": 1.2},
+            os.path.join("metrics", "metric2.yaml"): {"foo": 1.1},
+            "metrics_t.yaml": {"foo": 1.1},
+        }
+    }
diff --git a/tests/func/test_add.py b/tests/func/test_add.py
index 9b3003f59..18d48a4d4 100644
--- a/tests/func/test_add.py
+++ b/tests/func/test_add.py
@@ -208,9 +208,14 @@ def test_add_file_in_dir(tmp_dir, dvc):
         ),
         (pytest.lazy_fixture("gs"), "md5", "8c7dd922ad47494fc02c388e12c00eac"),
         (
-            pytest.lazy_fixture("hdfs"),
-            "checksum",
-            "000002000000000000000000a86fe4d846edc1bf4c355cb6112f141e",
+            pytest.param(
+                pytest.lazy_fixture("hdfs"),
+                "checksum",
+                "000002000000000000000000a86fe4d846edc1bf4c355cb6112f141e",
+                marks=pytest.mark.xfail(
+                    reason="https://github.com/iterative/dvc/issues/4418"
+                ),
+            )
         ),
     ],
     indirect=["workspace"],
diff --git a/tests/func/test_api.py b/tests/func/test_api.py
index a736103a1..3f1579638 100644
--- a/tests/func/test_api.py
+++ b/tests/func/test_api.py
@@ -1,9 +1,10 @@
 import os
+import platform
 
 import pytest
 
 from dvc import api
-from dvc.exceptions import FileMissingError, OutputNotFoundError
+from dvc.exceptions import DvcException, FileMissingError, OutputNotFoundError
 from dvc.path_info import URLInfo
 from dvc.utils.fs import remove
 from tests.unit.tree.test_repo import make_subrepo
@@ -15,10 +16,16 @@ cloud_names = [
     "gdrive",
     "oss",
     "ssh",
-    "hdfs",
     "http",
 ]
-clouds = [pytest.lazy_fixture(cloud) for cloud in cloud_names]
+clouds = [pytest.lazy_fixture(cloud) for cloud in cloud_names] + [
+    pytest.param(
+        pytest.lazy_fixture("hdfs"),
+        marks=pytest.mark.xfail(
+            reason="https://github.com/iterative/dvc/issues/4418"
+        ),
+    )
+]
 all_clouds = [pytest.lazy_fixture("local_cloud")] + clouds
 
 # `lazy_fixture` is confusing pylint, pylint: disable=unused-argument
@@ -69,17 +76,26 @@ def test_open(tmp_dir, dvc, remote):
 @pytest.mark.parametrize(
     "cloud",
     [
-        pytest.lazy_fixture(cloud)
-        for cloud in [
-            "real_s3",  # NOTE: moto's s3 fails in some tests
-            "gs",
-            "azure",
-            "gdrive",
-            "oss",
-            "ssh",
-            "hdfs",
-            "http",
-        ]
+        *[
+            pytest.lazy_fixture(cloud)
+            for cloud in [
+                "real_s3",  # NOTE: moto's s3 fails in some tests
+                "gs",
+                "azure",
+                "gdrive",
+                "oss",
+                "hdfs",
+                "http",
+            ]
+        ],
+        pytest.param(
+            pytest.lazy_fixture("ssh"),
+            marks=pytest.mark.xfail(
+                raises=DvcException,
+                condition=platform.system() == "Windows",
+                reason="https://github.com/iterative/dvc/issues/4418",
+            ),
+        ),
     ],
 )
 def test_open_external(erepo_dir, cloud):
@@ -210,6 +226,9 @@ def test_get_url_granular(tmp_dir, dvc, s3):
         {"dir": {"foo": "foo", "bar": "bar", "nested": {"file": "file"}}}
     )
 
+    expected_url = URLInfo(s3.url) / "5f/c28ea78987408341668eba6525ebd1.dir"
+    assert api.get_url("dir") == expected_url
+
     expected_url = URLInfo(s3.url) / "ac/bd18db4cc2f85cedef654fccc4a4d8"
     assert api.get_url("dir/foo") == expected_url
 
diff --git a/tests/func/test_cache.py b/tests/func/test_cache.py
index a28f3a4f1..36e50ed22 100644
--- a/tests/func/test_cache.py
+++ b/tests/func/test_cache.py
@@ -1,4 +1,5 @@
 import os
+import platform
 import stat
 
 import configobj
@@ -163,6 +164,10 @@ class TestCmdCacheDir(TestDvc):
         config = configobj.ConfigObj(self.dvc.config.files["repo"])
         self.assertEqual(config["cache"]["dir"], dname.replace("\\", "/"))
 
+    @pytest.mark.skipif(
+        platform.system() == "Darwin",
+        reason="https://github.com/iterative/dvc/issues/4418",
+    )
     def test_relative_path(self):
         tmpdir = self.mkdtemp()
         dname = relpath(tmpdir)
diff --git a/tests/func/test_import_url.py b/tests/func/test_import_url.py
index 4af804deb..ae37bf21f 100644
--- a/tests/func/test_import_url.py
+++ b/tests/func/test_import_url.py
@@ -121,7 +121,12 @@ def test_import_url_with_no_exec(tmp_dir, dvc, erepo_dir):
         pytest.lazy_fixture("local_cloud"),
         pytest.lazy_fixture("s3"),
         pytest.lazy_fixture("gs"),
-        pytest.lazy_fixture("hdfs"),
+        pytest.param(
+            pytest.lazy_fixture("hdfs"),
+            marks=pytest.mark.xfail(
+                reason="https://github.com/iterative/dvc/issues/4418"
+            ),
+        ),
         pytest.param(
             pytest.lazy_fixture("ssh"),
             marks=pytest.mark.skipif(
diff --git a/tests/func/test_run_multistage.py b/tests/func/test_run_multistage.py
index e7d49a63d..01986372f 100644
--- a/tests/func/test_run_multistage.py
+++ b/tests/func/test_run_multistage.py
@@ -386,10 +386,15 @@ def test_run_overwrite_preserves_meta_and_comment(tmp_dir, dvc, run_copy):
             "37b51d194a7513e45b56f6524f2d51f2",
         ),
         (
-            pytest.lazy_fixture("hdfs"),
-            "checksum",
-            "0000020000000000000000003dba826b9be9c6a8e2f8310a770555c4",
-            "00000200000000000000000075433c81259d3c38e364b348af52e84d",
+            pytest.param(
+                pytest.lazy_fixture("hdfs"),
+                "checksum",
+                "0000020000000000000000003dba826b9be9c6a8e2f8310a770555c4",
+                "00000200000000000000000075433c81259d3c38e364b348af52e84d",
+                marks=pytest.mark.xfail(
+                    reason="https://github.com/iterative/dvc/issues/4418"
+                ),
+            )
         ),
     ],
     indirect=["workspace"],
diff --git a/tests/func/test_update.py b/tests/func/test_update.py
index 6a21e4967..129660b7c 100644
--- a/tests/func/test_update.py
+++ b/tests/func/test_update.py
@@ -6,7 +6,18 @@ from dvc.dvcfile import Dvcfile
 from tests.unit.tree.test_repo import make_subrepo
 
 
-@pytest.mark.parametrize("cached", [True, False])
+@pytest.mark.parametrize(
+    "cached",
+    [
+        True,
+        pytest.param(
+            False,
+            marks=pytest.mark.skip(
+                reason="https://github.com/iterative/dvc/issues/4418"
+            ),
+        ),
+    ],
+)
 def test_update_import(tmp_dir, dvc, erepo_dir, cached):
     gen = erepo_dir.dvc_gen if cached else erepo_dir.scm_gen
 
diff --git a/tests/remotes/__init__.py b/tests/remotes/__init__.py
index 4cd648f7e..e5e4debf0 100644
--- a/tests/remotes/__init__.py
+++ b/tests/remotes/__init__.py
@@ -40,8 +40,8 @@ def docker():
     import os
 
     # See https://travis-ci.community/t/docker-linux-containers-on-windows/301
-    if os.environ.get("TRAVIS") and os.name == "nt":
-        pytest.skip("disabled for Windows on Travis")
+    if os.environ.get("CI") and os.name == "nt":
+        pytest.skip("disabled for Windows on Github Actions")
 
     try:
         subprocess.check_output("docker ps", shell=True)
diff --git a/tests/unit/output/test_local.py b/tests/unit/output/test_local.py
index c2a2d07c5..236fe50ed 100644
--- a/tests/unit/output/test_local.py
+++ b/tests/unit/output/test_local.py
@@ -1,5 +1,7 @@
 import os
+import platform
 
+import pytest
 from mock import patch
 
 from dvc.cache.local import LocalCache
@@ -25,6 +27,10 @@ class TestLocalOutput(TestDvc):
                 o.save()
 
 
+@pytest.mark.skipif(
+    platform.system() == "Windows",
+    reason="https://github.com/iterative/dvc/issues/4418",
+)
 def test_str_workdir_outside_repo(erepo_dir):
     stage = Stage(erepo_dir.dvc)
     output = LocalOutput(stage, "path", cache=False)
diff --git a/tests/unit/scm/test_scm.py b/tests/unit/scm/test_scm.py
index 4fb590fc5..1121d4df5 100644
--- a/tests/unit/scm/test_scm.py
+++ b/tests/unit/scm/test_scm.py
@@ -17,6 +17,7 @@ class TestScmContext(TestCase):
         method = mock.Mock()
         wrapped = scm_context(method)
 
+        self.repo_mock.configure_mock(config={})
         wrapped(self.repo_mock)
 
         self.assertEqual(1, method.call_count)
@@ -25,6 +26,20 @@ class TestScmContext(TestCase):
 
         self.assertEqual(0, self.scm_mock.cleanup_ignores.call_count)
 
+    def test_should_check_autostage(self):
+        method = mock.Mock()
+        wrapped = scm_context(method)
+
+        config_autostage_attrs = {"config": {"core": {"autostage": True}}}
+        self.repo_mock.configure_mock(**config_autostage_attrs)
+        wrapped(self.repo_mock)
+
+        self.assertEqual(1, method.call_count)
+        self.assertEqual(1, self.scm_mock.reset_ignores.call_count)
+        self.assertEqual(1, self.scm_mock.track_changed_files.call_count)
+
+        self.assertEqual(0, self.scm_mock.cleanup_ignores.call_count)
+
     def test_should_throw_and_cleanup(self):
         method = mock.Mock(side_effect=Exception("some problem"))
         wrapped = scm_context(method)