Repository: iterative/dvc

Issue:
exp apply: safeguard against unclean workspaces
`dvc exp apply` will clear the workspace of untracked files and unstaged modifications. To prevent users losing work, it would be good to follow git's convention of preventing overwriting of untracked/modified files unless the user gives explicit permission.

* If `dvc exp apply` is executed on an unclean workspace, fail with a suggestion to stash work or specify a `--force` flag
* `dvc exp apply --force` acts like the current command does.


Discussion:
Related: https://github.com/iterative/dvc.org/issues/2397
This was the original behavior for `exp apply`, but after much discussion the conclusion was that a majority of typical DVC users would always be using `exp apply --force`. So the behavior was changed to force-apply by default (and the `exp apply --no-force` option was added for users who do not want the default behavior)
@pmrowla I hadn't noticed the `--no-force` argument. On reflection, switching between experiments would necessitate use of a `--force` flag if it existed, and I guess that use-case is considered the most common.

I will monitor how I use `exp apply` for now, and whether I can easily encorporate `--no-force` into my workflow.
@pmrowla Forcibly overwriting conflicts is intended as explained here and in https://github.com/iterative/dvc.org/issues/2397, but it seems like experiments are overwriting everything in the workspace, even untracked files. This seems to conflict with https://github.com/iterative/dvc.org/issues/2397#issuecomment-822887190.

Example:

```
$ dvc stage add -n mystage echo foo
Creating 'dvc.yaml'
Adding stage 'mystage' in 'dvc.yaml'

To track the changes with git, run:

    git add dvc.yaml

To enable auto staging, run:

        dvc config core.autostage true
$ dvc exp run
Running stage 'mystage':
> echo foo
foo
Generating lock file 'dvc.lock'
Updating lock file 'dvc.lock'

To track the changes with git, run:

    git add dvc.lock dvc.yaml

To enable auto staging, run:

        dvc config core.autostage true

Reproduced experiment(s): exp-396c5
Experiment results have been applied to your workspace.

To promote an experiment to a Git branch run:

        dvc exp branch <exp> <branch>

$ touch newfile
$ dvc exp apply exp-396c5
Changes for experiment 'exp-396c5' have been applied to your current workspace.
$ git status
On branch main
Untracked files:
  (use "git add <file>..." to include in what will be committed)
        dvc.lock
        dvc.yaml

nothing added to commit but untracked files present (use "git add" to track)
$ ls
dvc.lock dvc.yaml
```

Another user asked about this in discord: https://discord.com/channels/485586884165107732/485596304961962003/963800799220211782.

@pmrowla What do you think about bumping priority of this? It seems dangerous and potentially surprising to overwrite completely unrelated untracked files.
Unrelated untracked files being removed sounds like a bug. As for the priority, I guess I'm not sure if that should be opened as a separate issue from this one? It's unclear to me whether this issue is about the removing untracked files bug or if it's a higher level discussion about whether `--force` or `--no-force` should be the default behavior
I guess I hijacked this ticket from @mattlbeck. As mentioned in https://github.com/iterative/dvc/issues/6930#issuecomment-970712018 and other comments above, we already have discussed the default behavior and it usually makes sense to leave the current defaults.

So let's either prioritize this issue to stop dropping unrelated untracked files, or close this one and open a new issue.

Resolving patch:
diff --git a/dvc/repo/experiments/apply.py b/dvc/repo/experiments/apply.py
--- a/dvc/repo/experiments/apply.py
+++ b/dvc/repo/experiments/apply.py
@@ -1,5 +1,7 @@
 import logging
 import os
+from contextlib import contextmanager
+from typing import TYPE_CHECKING, Optional
 
 from dvc.repo import locked
 from dvc.repo.scm_context import scm_context
@@ -13,13 +15,18 @@
 )
 from .executor.base import BaseExecutor
 
+if TYPE_CHECKING:
+    from scmrepo import Git
+
+    from dvc.repo import Repo
+
 logger = logging.getLogger(__name__)
 
 
 @locked
 @scm_context
-def apply(repo, rev, force=True, **kwargs):
-    from scmrepo.exceptions import MergeConflictError
+def apply(repo: "Repo", rev: str, force: bool = True, **kwargs):
+    from scmrepo.exceptions import SCMError as _SCMError
 
     from dvc.repo.checkout import checkout as dvc_checkout
     from dvc.scm import GitMergeError, RevError, resolve_rev
@@ -38,37 +45,14 @@ def apply(repo, rev, force=True, **kwargs):
     ):
         raise InvalidExpRevError(exp_rev)
 
-    # Note that we don't use stash_workspace() here since we need finer control
-    # over the merge behavior when we unstash everything
-    if repo.scm.is_dirty(untracked_files=True):
-        logger.debug("Stashing workspace")
-        workspace = repo.scm.stash.push(include_untracked=True)
-    else:
-        workspace = None
-
-    from scmrepo.exceptions import SCMError as _SCMError
-
-    try:
-        repo.scm.merge(exp_rev, commit=False, squash=True)
-    except _SCMError as exc:
-        raise GitMergeError(str(exc), scm=repo.scm)
-
-    if workspace:
+    # NOTE: we don't use scmrepo's stash_workspace() here since we need
+    # finer control over the merge behavior when we unstash everything
+    with _apply_workspace(repo, rev, force):
         try:
-            repo.scm.stash.apply(workspace)
-        except MergeConflictError as exc:
-            # Applied experiment conflicts with user's workspace changes
-            if force:
-                # prefer applied experiment changes over prior stashed changes
-                repo.scm.checkout_index(ours=True)
-            else:
-                # revert applied changes and restore user's workspace
-                repo.scm.reset(hard=True)
-                repo.scm.stash.pop()
-                raise ApplyConflictError(rev) from exc
+            repo.scm.merge(exp_rev, commit=False, squash=True)
         except _SCMError as exc:
-            raise ApplyConflictError(rev) from exc
-        repo.scm.stash.drop()
+            raise GitMergeError(str(exc), scm=repo.scm)
+
     repo.scm.reset()
 
     if stash_rev:
@@ -84,3 +68,44 @@ def apply(repo, rev, force=True, **kwargs):
         "workspace.",
         rev,
     )
+
+
+@contextmanager
+def _apply_workspace(repo: "Repo", rev: str, force: bool):
+    from scmrepo.exceptions import MergeConflictError
+    from scmrepo.exceptions import SCMError as _SCMError
+
+    if repo.scm.is_dirty(untracked_files=True):
+        logger.debug("Stashing workspace")
+        stash_rev: Optional[str] = repo.scm.stash.push(include_untracked=True)
+    else:
+        stash_rev = None
+    try:
+        yield
+    except Exception:  # pylint: disable=broad-except
+        if stash_rev:
+            _clean_and_pop(repo.scm)
+        raise
+    if not stash_rev:
+        return
+
+    try:
+        repo.scm.reset()
+        repo.scm.stash.apply(stash_rev, skip_conflicts=force)
+        repo.scm.stash.drop()
+    except (MergeConflictError, _SCMError) as exc:
+        _clean_and_pop(repo.scm)
+        raise ApplyConflictError(rev) from exc
+    except Exception:  # pylint: disable=broad-except
+        _clean_and_pop(repo.scm)
+        raise
+
+
+def _clean_and_pop(scm: "Git"):
+    """Revert any changes and pop the last stash entry."""
+    scm.reset(hard=True)
+    if scm.is_dirty(untracked_files=True):
+        # drop any changes to untracked files before popping stash
+        scm.stash.push(include_untracked=True)
+        scm.stash.drop()
+    scm.stash.pop()


Test patch:
diff --git a/tests/func/experiments/test_experiments.py b/tests/func/experiments/test_experiments.py
--- a/tests/func/experiments/test_experiments.py
+++ b/tests/func/experiments/test_experiments.py
@@ -217,6 +217,25 @@ def test_apply(tmp_dir, scm, dvc, exp_stage, queue):
     )
 
 
+def test_apply_untracked(tmp_dir, scm, dvc, exp_stage):
+    from dvc.repo.experiments.base import ApplyConflictError
+
+    results = dvc.experiments.run(exp_stage.addressing, params=["foo=2"])
+    exp = first(results)
+    tmp_dir.gen("untracked", "untracked")
+    tmp_dir.gen("params.yaml", "conflict")
+
+    with pytest.raises(ApplyConflictError):
+        dvc.experiments.apply(exp, force=False)
+
+    assert (tmp_dir / "untracked").read_text() == "untracked"
+    assert (tmp_dir / "params.yaml").read_text() == "conflict"
+
+    dvc.experiments.apply(exp, force=True)
+    assert (tmp_dir / "untracked").read_text() == "untracked"
+    assert (tmp_dir / "params.yaml").read_text().strip() == "foo: 2"
+
+
 def test_get_baseline(tmp_dir, scm, dvc, exp_stage):
     from dvc.repo.experiments.base import EXPS_STASH