Repository: iterative/dvc

Issue:
```dvc exp run --queue```: deleting file 'params.yaml' for each command call
# Bug Report

 dvc exp run --queue: deleting file 'params.yaml' for each command call
<!--
## dvc exp run --queue: deleting file 'params.yaml' for each command call

Issue names must follow the pattern `command: description` where the command is the dvc command that you are trying to run. The description should describe the consequence of the bug. 

Example: `repro: doesn't detect input changes`
-->

## Description

```dvc exp run --queue``` deletes file 'params.yaml' for each trial to add a new experiment and the experiment don't put into queue. It works correctly only if command ```git add params.yaml``` was called before.


### Reproduce

Example files:
test.py
```
import numpy as np
import json
import yaml

params = yaml.safe_load(open('params.yaml'))["test"]

precision = np.random.random()
recall = params['value']
accuracy = np.random.random()

rows = {'precision': precision,
        'recall': recall,
        'accuracy': accuracy}

with open(params['metrics_path'], 'w') as outfile:
    json.dump(rows, outfile)

fpr = 10*np.random.random((1,10)).tolist()
tpr = 10*np.random.random((1,10)).tolist()

with open('plot.json', 'w') as outfile2:
    json.dump(
      {
        "roc": [ {"fpr": f, "tpr": t} for f, t in zip(fpr, tpr) ]
      }, 
      outfile2
      )
```

params.yaml
```
test:
  metrics_path: scores.json
  value: 2
```

dvc.yaml
```
stages:
  test:
    cmd: python test.py
    deps:
    - test.py
    params:
    - test.metrics_path
    - test.value
    metrics:
    - scores.json:
        cache: false
    plots:
    - plot.json:
        cache: false
        x: fpr
        y: tpr
```

1. ```dvc exp run --queue -S test.value=101```

<!--
Step list of how to reproduce the bug
-->

<!--
Example:

1. dvc init
2. Copy dataset.zip to the directory
3. dvc add dataset.zip
4. dvc run -d dataset.zip -o model ./train.sh
5. modify dataset.zip
6. dvc repro
-->

### Expected

New experiments adds to queue without 'params.yaml' deleting.

<!--
A clear and concise description of what you expect to happen.
-->

### Environment information

Ubuntu 20.04

<!--
This is required to ensure that we can reproduce the bug.
-->

**Output of `dvc doctor`:**

```console
$ dvc doctor

DVC version: 2.10.2 (pip)
---------------------------------
Platform: Python 3.9.7 on Linux-5.13.0-40-generic-x86_64-with-glibc2.31
Supports:
        webhdfs (fsspec = 2022.3.0),
        http (aiohttp = 3.8.1, aiohttp-retry = 2.4.6),
        https (aiohttp = 3.8.1, aiohttp-retry = 2.4.6)
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/nvme0n1p6
Caches: local
Remotes: None
Workspace directory: ext4 on /dev/nvme0n1p6
Repo: dvc, git
```

**Additional Information (if any):**
<!--
Please check https://github.com/iterative/dvc/wiki/Debugging-DVC on ways to gather more information regarding the issue.

If applicable, please also provide a `--verbose` output of the command, eg: `dvc add --verbose`.
If the issue is regarding the performance, please attach the profiling information and the benchmark comparisons.
-->

dvc.api.open: Will ignore remote option if a default remote is set in config.
# Bug Report

## dvc.api.open: Will ignore remote option if a default remote is set in config.

## Description

In some cases, when passing the remote parameter. The remote specification will get lost during the `dvc pull` execution and will be overwritten by the default remote from the dvc config.

### Reproduce

1. Create new git Repo
2. Init dvc
3. Add two different non local remotes (perferably s3)
4. Make one of the remotes the default
5. Push the same data to both remotes
6. Push git Repo
7. From somewhere outside a dvc project run:
```python
with dvc.api.open(path="myfile", remote='non-default-remote') 
```
8. Track the actually used remote with a Debugger. Or make sure both remotes require different Authentification and just provide the non-defaut-remote authentification during the api call. You will run into Permission errors.

### Expected

If the remote argument is passed this remote should allways be used, when opening files.

### Environment information

**Output of `dvc doctor`:**

```console
DVC version: 2.9.3 (pip)
---------------------------------
Platform: Python 3.8.12 on Linux-5.10.47-linuxkit-x86_64-with-glibc2.2.5
Supports:
        hdfs (fsspec = 2022.1.0, pyarrow = 1.0.1),
        webhdfs (fsspec = 2022.1.0),
        http (aiohttp = 3.8.1, aiohttp-retry = 2.4.6),
        https (aiohttp = 3.8.1, aiohttp-retry = 2.4.6),
        s3 (s3fs = 2022.1.0, boto3 = 1.20.24)
root@112f43f4f86b:/workspaces/centerline-identification-job# 
```



Discussion:
probably related to https://github.com/iterative/dvc/issues/6930 (untracked files being removed on post-run apply)
Only occurs if `--set-param` modifies an untracked params file and `--temp` or `--queue` flag is passed. 

The untracked params file gets added `scm.add` on [`_update_params`](https://github.com/iterative/dvc/blob/dbfe88510ff1185ba83c6b7d635275232f6ba086/dvc/repo/experiments/queue/base.py#L332) and stashed to [`stash_rev`](https://github.com/iterative/dvc/blob/dbfe88510ff1185ba83c6b7d635275232f6ba086/dvc/repo/experiments/queue/base.py#L406).

When using `WorkspaceExecutor`, [`init_git`](https://github.com/iterative/dvc/blob/d0eda1d5385ba6a78247528327563b536a561b78/dvc/repo/experiments/executor/local.py#L166) applies a `merge` on the current git repo, "restoring" the params file in the user workspace.

When using `TmpDirExecutor`, the `merge` is applied on the temp dir git repo, so the params file is never "restored" in the user workspace.

It seems like the behavior is expected then. However, maybe we can clarify in the docs that experiment files should be tracked by either Git or DVC?
I stumbled over this today and found it at least "unexpected" and did not know how to fix it, until I came here.
Yes, I think I was incorrect to say it is expected behavior. In fact, I ran into it myself and was confused recently (see #8256). Should we be applying the merge on top of both the tmp dir and the workspace @pmrowla?
Applying the merge on the workspace isn't the correct behavior, that would put `-S` modified params into the workspace and not the original params file.

I think the fix for this should just be to use `stash_workspace(include_untracked=True)` here
https://github.com/iterative/dvc/blob/55b16166c852a72e8dd47d4e32090063d27497e4/dvc/repo/experiments/queue/base.py#L320

When we leave that context it is supposed to restore the workspace to the state before any DVC exp related changes/operations were done. `include_untracked` was omitted because we aren't supposed to be touching untracked files at all in exps, but the bug is we actually can modify untracked files in the params file case.
Initial discussion:
https://discord.com/channels/485586884165107732/485596304961962003/933674011982983219

@PietrassykFP has been setting up the credentials via environmental variables, so if the remote gets resolved properly, it should not raise issues. I created a local test aiming to verify if remote gets resolved properly  and it seem to be working fine:

```
def test_open_from_proper_remote(tmp_dir, erepo_dir, make_tmp_dir):
    default_remote = make_tmp_dir("default_storage")
    other_remote = make_tmp_dir("other_storage")

    with erepo_dir.chdir():
        with erepo_dir.dvc.config.edit() as conf:
            conf["remote"]["default"] = {"url": str(default_remote)}
            conf["core"]["remote"] = "default"
            conf["remote"]["other"] = {"url": str(other_remote)}

        erepo_dir.dvc_gen({"foo": "foo content"}, commit="create file")
        erepo_dir.dvc.push(remote="other")

        (erepo_dir / "foo").unlink()
        shutil.rmtree((erepo_dir / ".dvc" / "cache"))

    assert not (
        default_remote / "6d" / "bda444875c24ec1bbdb433456be11f"
    ).is_file()
    assert (other_remote / "6d" / "bda444875c24ec1bbdb433456be11f").is_file()

    with api.open("foo", repo=os.fspath(erepo_dir)) as fd:
        result = fd.read()

    assert result == "foo content"
```

It would be good to reproduce the problem and create a test, though that needs some more investigation.

Resolving patch:
diff --git a/dvc/api/data.py b/dvc/api/data.py
--- a/dvc/api/data.py
+++ b/dvc/api/data.py
@@ -1,6 +1,6 @@
 from contextlib import _GeneratorContextManager as GCM
 from contextlib import contextmanager
-from typing import Optional
+from typing import Any, Dict, Optional
 
 from funcy import reraise
 
@@ -214,7 +214,11 @@ def open(  # noqa, pylint: disable=redefined-builtin
 
 
 def _open(path, repo=None, rev=None, remote=None, mode="r", encoding=None):
-    with Repo.open(repo, rev=rev, subrepos=True, uninitialized=True) as _repo:
+    repo_kwargs: Dict[str, Any] = {"subrepos": True, "uninitialized": True}
+    if remote:
+        repo_kwargs["config"] = {"core": {"remote": remote}}
+
+    with Repo.open(repo, rev=rev, **repo_kwargs) as _repo:
         with _wrap_exceptions(_repo, path):
             import os
             from typing import TYPE_CHECKING, Union
@@ -235,12 +239,6 @@ def _open(path, repo=None, rev=None, remote=None, mode="r", encoding=None):
                 fs_path = fs.from_os_path(path)
 
             try:
-                if remote:
-                    remote_odb = _repo.cloud.get_remote_odb(name=remote)
-                    oid = fs.info(fs_path)["dvc_info"]["md5"]
-                    fs = remote_odb.fs
-                    fs_path = remote_odb.oid_to_path(oid)
-
                 with fs.open(
                     fs_path,
                     mode=mode,
diff --git a/dvc/external_repo.py b/dvc/external_repo.py
--- a/dvc/external_repo.py
+++ b/dvc/external_repo.py
@@ -43,6 +43,7 @@ def external_repo(
 
     config = _get_remote_config(url) if os.path.isdir(url) else {}
     config.update(cache_config)
+    config.update(kwargs.pop("config", None) or {})
 
     main_root = "/"
     if for_write:
diff --git a/dvc/repo/experiments/run.py b/dvc/repo/experiments/run.py
--- a/dvc/repo/experiments/run.py
+++ b/dvc/repo/experiments/run.py
@@ -11,7 +11,7 @@
 
 
 @locked
-def run(  # noqa: C901
+def run(  # noqa: C901, PLR0912
     repo,
     targets: Optional[Iterable[str]] = None,
     params: Optional[Iterable[str]] = None,
@@ -36,6 +36,18 @@ def run(  # noqa: C901
         from dvc.utils.hydra import to_hydra_overrides
 
         path_overrides = to_path_overrides(params)
+
+        if tmp_dir or queue:
+            untracked = repo.scm.untracked_files()
+            for path in path_overrides:
+                if path in untracked:
+                    logger.debug(
+                        "'%s' is currently untracked but will be modified by DVC. "
+                        "Adding it to git.",
+                        path,
+                    )
+                    repo.scm.add([path])
+
         hydra_sweep = any(
             x.is_sweep_override()
             for param_file in path_overrides


Test patch:
diff --git a/tests/func/api/test_data.py b/tests/func/api/test_data.py
--- a/tests/func/api/test_data.py
+++ b/tests/func/api/test_data.py
@@ -193,10 +193,6 @@ def test_get_url_subrepos(tmp_dir, scm, local_cloud):
     assert api.get_url("subrepo/bar") == expected_url
 
 
-@pytest.mark.xfail(
-    raises=PathMissingError,
-    reason="https://github.com/iterative/dvc/issues/7341",
-)
 def test_open_from_remote(tmp_dir, erepo_dir, cloud, local_cloud):
     erepo_dir.add_remote(config=cloud.config, name="other")
     erepo_dir.add_remote(config=local_cloud.config, default=True)
diff --git a/tests/func/experiments/test_queue.py b/tests/func/experiments/test_queue.py
--- a/tests/func/experiments/test_queue.py
+++ b/tests/func/experiments/test_queue.py
@@ -92,3 +92,19 @@ def test_queue_remove_done(
     assert set(celery_queue.clear(success=True)) == set(success_tasks[:2])
 
     assert celery_queue.status() == []
+
+
+def test_queue_doesnt_remove_untracked_params_file(tmp_dir, dvc, scm):
+    """Regression test for https://github.com/iterative/dvc/issues/7842"""
+    tmp_dir.gen("params.yaml", "foo: 1")
+    stage = dvc.run(cmd="echo ${foo}", params=["foo"], name="echo-foo")
+    scm.add(
+        [
+            "dvc.yaml",
+            "dvc.lock",
+            ".gitignore",
+        ]
+    )
+    scm.commit("init")
+    dvc.experiments.run(stage.addressing, params=["foo=2"], queue=True)
+    assert (tmp_dir / "params.yaml").exists()