Repository: iterative/dvc

Issue:
`exp init`: clean up error messages
From #7130:

```
$ dvc -V
2.9.2
```

```
$ git init
$ dvc init
$ dvc exp init python train.py
ERROR: unexpected error - [Errno 2] No such file or directory: 'params.yaml'
```

* [x]  Why is it `unexpected` error? It should be like “ERROR: hyperparameters file ‘params.yaml’ is missing”.    ~~🤔  Why `dvc exp init` does not create it if it is required?~~

* [x]  Do we really need [Errno 2]? In what cases the code helps users? This makes the message longer and breaks human readability.


```
$ dvc exp run
ERROR: failed to reproduce 'dvc.yaml': [Errno 2] No such file or directory: '/Users/dmitry/src/test/myproj/data'
```

* [x]  full path. It is supposed to be relative. Why: 1) to make it short - the full path provides zero value in most of use cases; 2) it makes it hard to submit reports (some users are not comfortable to share their directories struture - See `'/Users/dmitry/src/test/` - bc it might contain confidential info like user and project names.

* [ ] ~~why `dvc exp init` checked `params.yaml` but didn’t check (or create) `data/` & `src/`?~~


```
$ mkdir data
$ dvc exp run
ERROR: failed to reproduce 'dvc.yaml': [Errno 2] No such file or directory: '/Users/dmitry/src/test/myproj/src'
```

* [x]  the same questions - [Errno 2], Full path.

* [ ]  ~~why it was not checked in `dvc init`? Or was not created?~~


```
$ mkdir src
$ dvc exp run
…
python: can't open file 'train.py': [Errno 2] No such file or directory
ERROR: failed to reproduce 'dvc.yaml': failed to run: python train.py, exited with 2
```

* [ ]   “exited with 2” Is it useful information? Where the exit code 2 belongs to? It might confuse users. Error code can be in debug errors not here.

* [x]  Another issue - ideally, we should have a reference to stage (a user might have multiple stages with the same or similar commands). The message might look like:    **ERROR: failed to execute stage ‘train’ from ‘dvc.yaml’**
`exp init`: more verbose output in non-interactive mode
Related to #7130. `dvc exp init -i` has verbose output to help guide the user. Non-interactive mode should have the same. A `-q` option can be added for experienced users to hide this output.

Current `dvc exp init -i` output:

```
$ dvc exp init -i python train.py
This command will guide you to set up a train stage in dvc.yaml.
See https://s.dvc.org/g/pipeline-files.

Enter the paths for dependencies and outputs of the command.
DVC assumes the following workspace structure:
├── data
├── metrics.json
├── models
├── params.yaml
├── plots
└── src

Path to a code file/directory [src, n to omit]:
'src' does not exist in the workspace. "exp run" may fail.
Path to a data file/directory [data, n to omit]:
'data' does not exist in the workspace. "exp run" may fail.
Path to a model file/directory [models, n to omit]:
Path to a parameters file [params.yaml, n to omit]:
Path to a metrics file [metrics.json, n to omit]:
Path to a plots file/directory [plots, n to omit]:
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
train:
  cmd: python train.py
  deps:
  - data
  - src
  params:
  - bar
  - foo
  outs:
  - models
  metrics:
  - metrics.json:
      cache: false
  plots:
  - plots:
      cache: false

Do you want to add the above contents to dvc.yaml? [Y/n]: Y

Created train stage in dvc.yaml. To run, use "dvc exp run".
See https://s.dvc.org/g/exp/run.
```

Current `dvc exp init` output:

```
$ dvc exp init python train.py
Created train stage in dvc.yaml. To run, use "dvc exp run".
See https://s.dvc.org/g/exp/run.
```

Proposed `dvc exp init` output:

```
$ dvc exp init python train.py
This command will set up a train stage in dvc.yaml.
See https://s.dvc.org/g/pipeline-files.

DVC assumes the following workspace structure:
├── data
├── metrics.json
├── models
├── params.yaml
├── plots
└── src

'src' does not exist in the workspace. "exp run" may fail.
'data' does not exist in the workspace. "exp run" may fail.
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
train:
  cmd: python train.py
  deps:
  - data
  - src
  params:
  - bar
  - foo
  outs:
  - models
  metrics:
  - metrics.json:
      cache: false
  plots:
  - plots:
      cache: false

Created train stage in dvc.yaml. To run, use "dvc exp run".
See https://s.dvc.org/g/exp/run.
```



Comments:
>  “exited with 2” Is it useful information? Where the exit code 2 belongs to? It might confuse users. Error code can be in debug errors not here.

I disagree with this, exit codes are important, well that's how you know it's succeeded or failed (or crashed).

>  Do we really need [Errno 2]? In what cases the code helps users? This makes the message longer and breaks human readability.
> $ dvc exp run
> ERROR: failed to reproduce 'dvc.yaml': [Errno 2] No such file or directory: '/Users/dmitry/src/test/myproj/data'

This is unrelated to `dvc exp init` and out of scope to be honest for this. This is part of a larger issue with DVC and happens in all of the commands. When I brought this a year ago for https://github.com/iterative/dvc/issues/5387, there were disagreements on how to handle these errors (cc @efiop).

From #7130:

> `dvc exp init` is not fully supported without interactive mode.
It looks like today the only supported flow is — interactive. The default `dvc init` has multiple assumptions about the project structure and it is not clear how a user should know about the assumptions.

Fair point @dmpetrov. Making the output consistent between them can make non-interactive mode more like `dvc exp init -i`.
```
'src' does not exist in the workspace. "exp run" may fail.
'data' does not exist in the workspace. "exp run" may fail.
```

I'm proposing creating these by default (plus `params.yaml`). Users need a single simple command to wrap their `python train.py` without manually creating all the structures.
There is a rule on https://clig.dev/#output - `Display output on success, but keep it brief.` It feels like these are not needed in the output if we follow the rule.

Two issues are here:

1. Do we need the content of dvc.yaml in the output?
```
train:
  cmd: python train.py
  deps:
  - data
```

2. Do we need links in the output? (Especially as a single line)
```
See https://s.dvc.org/g/pipeline-files.
...
See https://s.dvc.org/g/exp/run.
```
> There is a rule on https://clig.dev/#output - `Display output on success, but keep it brief.` It feels like these are not needed in the output if we follow the rule.

I think @skshetry would agree, which explains why there is so little output in non-interactive mode now. I suggested more verbose output since this command is tailored to users with no prior dvc knowledge.

> 1. Do we need the content of dvc.yaml in the output?

No. It can be an intro to dvc stage syntax, but it's not necessary. We also don't need to show it or prompt for confirmation in `dvc exp init -i`.

> Do we need links in the output? (Especially as a single line)

No. Again, I/we opted for more verbose output to guide users more. It's not output that is/needs to be machine readable, and I thought usability should trump style in this case, but obviously it can be removed.

---

Given those suggestions, the basic output can be:

```
$ dvc exp init python train.py
DVC assumes the following workspace structure:
├── data
├── metrics.json
├── models
├── params.yaml
├── plots
└── src

Created train stage in dvc.yaml. To run, use "dvc exp run".
```

Should we also make interactive mode more concise?

```
$ dvc exp init -i python train.py
DVC assumes the following workspace structure:
├── data
├── metrics.json
├── models
├── params.yaml
├── plots
└── src

Path to a code file/directory [src, n to omit]:
Path to a data file/directory [data, n to omit]:
Path to a model file/directory [models, n to omit]:
Path to a parameters file [params.yaml, n to omit]:
Path to a metrics file [metrics.json, n to omit]:
Path to a plots file/directory [plots, n to omit]:

Created train stage in dvc.yaml. To run, use "dvc exp run".
```
One thought on this: having quasi-unix style makes sense for dvc. In this case, I'm not so sure since the intended audience is less likely to be people familiar with that style.

Patch:
diff --git a/dvc/commands/experiments/init.py b/dvc/commands/experiments/init.py
--- a/dvc/commands/experiments/init.py
+++ b/dvc/commands/experiments/init.py
@@ -69,7 +69,6 @@ def run(self):
         )
 
         text = ui.rich_text.assemble(
-            "\n" if self.args.interactive else "",
             "Created ",
             (self.args.name, "bright_blue"),
             " stage in ",
diff --git a/dvc/repo/experiments/init.py b/dvc/repo/experiments/init.py
--- a/dvc/repo/experiments/init.py
+++ b/dvc/repo/experiments/init.py
@@ -15,7 +15,7 @@
     cast,
 )
 
-from funcy import compact, lremove
+from funcy import compact, lremove, lsplit
 from rich.rule import Rule
 from rich.syntax import Syntax
 
@@ -23,6 +23,7 @@
 from dvc.stage import PipelineStage
 from dvc.stage.serialize import to_pipeline_file
 from dvc.types import OptStr
+from dvc.utils import humanize
 from dvc.utils.serialize import dumps_yaml
 
 if TYPE_CHECKING:
@@ -79,18 +80,26 @@ def _disable_logging(highest_level=logging.CRITICAL):
         logging.disable(previous_level)
 
 
-def build_workspace_tree(workspace: Dict[str, str]) -> "Tree":
+def build_workspace_tree(workspace: Dict[str, str], label: str) -> "Tree":
     from rich.tree import Tree
 
-    tree = Tree(
-        "DVC assumes the following workspace structure:",
-        highlight=True,
-    )
+    tree = Tree(label, highlight=True)
     for value in sorted(workspace.values()):
         tree.add(f"[green]{value}[/green]")
     return tree
 
 
+def display_workspace_tree(
+    workspace: Dict[str, str], label: str, stderr: bool = False
+) -> None:
+    d = workspace.copy()
+    d.pop("cmd", None)
+
+    if d:
+        ui.write(build_workspace_tree(d, label), styled=True, stderr=stderr)
+    ui.write(styled=True, stderr=stderr)
+
+
 PIPELINE_FILE_LINK = "https://s.dvc.org/g/pipeline-files"
 
 
@@ -144,9 +153,9 @@ def init_interactive(
         "Enter the paths for dependencies and outputs of the command.",
         styled=True,
     )
-    if workspace:
-        ui.error_write(build_workspace_tree(workspace), styled=True)
-    ui.error_write(styled=True)
+
+    tree_label = "Using experiment project structure:"
+    display_workspace_tree(workspace, tree_label, stderr=True)
     ret.update(
         compact(
             _prompts(prompts, defaults, validator=validator, stream=stream)
@@ -192,13 +201,45 @@ def validate_prompts(
             raise InvalidResponse(msg_format.format(value, suffices[e.errno]))
     elif key in ("code", "data"):
         if not os.path.exists(value):
-            return value, (
-                f"[yellow]'{value}' does not exist in the workspace. "
-                '"exp run" may fail.[/]'
+            typ = "file" if is_file(value) else "directory"
+            return (
+                value,
+                f"[yellow]'{value}' does not exist, "
+                f"the {typ} will be created. ",
             )
     return value
 
 
+def is_file(path: str) -> bool:
+    _, ext = os.path.splitext(path)
+    return bool(ext)
+
+
+def init_deps(stage: PipelineStage) -> None:
+    from funcy import rpartial
+
+    from dvc.dependency import ParamsDependency
+    from dvc.fs.local import localfs
+
+    new_deps = [dep for dep in stage.deps if not dep.exists]
+    params, deps = lsplit(rpartial(isinstance, ParamsDependency), new_deps)
+
+    if new_deps:
+        paths = map("[green]{0}[/]".format, new_deps)
+        ui.write(f"Created {humanize.join(paths)}.", styled=True)
+
+    # always create a file for params, detect file/folder based on extension
+    # for other dependencies
+    dirs = [dep.fs_path for dep in deps if not is_file(dep.fs_path)]
+    files = [dep.fs_path for dep in deps + params if is_file(dep.fs_path)]
+    for path in dirs:
+        localfs.makedirs(path)
+    for path in files:
+        localfs.makedirs(localfs.path.parent(path), exist_ok=True)
+        with localfs.open(path, "w", encoding="utf-8"):
+            pass
+
+
 def init(
     repo: "Repo",
     name: str = "train",
@@ -278,19 +319,26 @@ def init(
 
     from dvc.ui.prompt import Confirm
 
-    if not interactive or Confirm.ask(
+    if interactive and not Confirm.ask(
         "Do you want to add the above contents to dvc.yaml?",
         console=ui.error_console,
         default=True,
         stream=stream,
     ):
-        with _disable_logging(), repo.scm_context(autostage=True, quiet=True):
-            stage.dump(update_lock=False)
-            stage.ignore_outs()
-            if params:
-                repo.scm_context.track_file(params)
-    else:
         raise DvcException("Aborting ...")
 
+    if interactive:
+        ui.error_write()  # add a newline after the prompts
+
+    with _disable_logging(), repo.scm_context(autostage=True, quiet=True):
+        stage.dump(update_lock=False)
+        stage.ignore_outs()
+        if not interactive:
+            label = "Using experiment project structure:"
+            display_workspace_tree(context, label)
+        init_deps(stage)
+        if params:
+            repo.scm_context.track_file(params)
+
     assert isinstance(stage, PipelineStage)
     return stage


Test patch:
diff --git a/tests/func/experiments/test_init.py b/tests/func/experiments/test_init.py
--- a/tests/func/experiments/test_init.py
+++ b/tests/func/experiments/test_init.py
@@ -44,6 +44,8 @@ def test_init_simple(tmp_dir, scm, dvc, capsys):
             }
         }
     }
+    assert (tmp_dir / "data").read_text() == "data"
+    assert (tmp_dir / "src").is_dir()
 
 
 @pytest.mark.parametrize("interactive", [True, False])
@@ -142,6 +144,8 @@ def test_init_interactive_when_no_path_prompts_need_to_be_asked(
             }
         }
     }
+    assert (tmp_dir / "src").is_dir()
+    assert (tmp_dir / "data").is_dir()
 
 
 def test_when_params_is_omitted_in_interactive_mode(tmp_dir, scm, dvc):
@@ -164,6 +168,8 @@ def test_when_params_is_omitted_in_interactive_mode(tmp_dir, scm, dvc):
         }
     }
     assert not (tmp_dir / "dvc.lock").exists()
+    assert (tmp_dir / "script.py").read_text() == ""
+    assert (tmp_dir / "data").is_dir()
     scm._reset()
     assert scm.is_tracked("dvc.yaml")
     assert not scm.is_tracked("params.yaml")
@@ -194,6 +200,8 @@ def test_init_interactive_params_validation(tmp_dir, dvc, capsys):
             }
         }
     }
+    assert (tmp_dir / "script.py").read_text() == ""
+    assert (tmp_dir / "data").is_dir()
 
     out, err = capsys.readouterr()
     assert (
@@ -205,19 +213,11 @@ def test_init_interactive_params_validation(tmp_dir, dvc, capsys):
         "Please retry with an existing parameters file.\n"
         "Path to a parameters file [params.yaml, n to omit]:"
     ) in err
-    assert not out
+    assert out == "Created script.py.\n"
 
 
 def test_init_with_no_defaults_interactive(tmp_dir, dvc):
-    inp = io.StringIO(
-        "python script.py\n"
-        "script.py\n"
-        "data\n"
-        "model\n"
-        "n\n"
-        "metric\n"
-        "n\n"
-    )
+    inp = io.StringIO("script.py\n" "data\n" "model\n" "n\n" "metric\n" "n\n")
     init(
         dvc,
         defaults={},
@@ -229,12 +229,14 @@ def test_init_with_no_defaults_interactive(tmp_dir, dvc):
         "stages": {
             "train": {
                 "cmd": "python script.py",
-                "deps": ["python script.py", "script.py"],
+                "deps": ["data", "script.py"],
                 "metrics": [{"metric": {"cache": False}}],
-                "outs": ["data"],
+                "outs": ["model"],
             }
         }
     }
+    assert (tmp_dir / "script.py").read_text() == ""
+    assert (tmp_dir / "data").is_dir()
 
 
 @pytest.mark.parametrize(
@@ -258,9 +260,7 @@ def test_init_with_no_defaults_interactive(tmp_dir, dvc):
     ],
     ids=["non-interactive", "interactive"],
 )
-def test_init_interactive_default(
-    tmp_dir, scm, dvc, interactive, overrides, inp, capsys
-):
+def test_init_default(tmp_dir, scm, dvc, interactive, overrides, inp, capsys):
     (tmp_dir / "params.yaml").dump({"foo": {"bar": 1}})
 
     init(
@@ -284,6 +284,8 @@ def test_init_interactive_default(
         }
     }
     assert not (tmp_dir / "dvc.lock").exists()
+    assert (tmp_dir / "script.py").read_text() == ""
+    assert (tmp_dir / "data").is_dir()
     scm._reset()
     assert scm.is_tracked("dvc.yaml")
     assert scm.is_tracked("params.yaml")
@@ -292,9 +294,11 @@ def test_init_interactive_default(
     out, err = capsys.readouterr()
 
     if interactive:
-        assert "'script.py' does not exist in the workspace." in err
-        assert "'data' does not exist in the workspace." in err
-    assert not out
+        assert "'script.py' does not exist, the file will be created." in err
+        assert "'data' does not exist, the directory will be created." in err
+    else:
+        assert "Using experiment project structure: " in out
+    assert "Created script.py and data" in out
 
 
 @pytest.mark.timeout(5, func_only=True)
@@ -365,6 +369,8 @@ def test_init_interactive_live(
         }
     }
     assert not (tmp_dir / "dvc.lock").exists()
+    assert (tmp_dir / "script.py").read_text() == ""
+    assert (tmp_dir / "data").is_dir()
     scm._reset()
     assert scm.is_tracked("dvc.yaml")
     assert scm.is_tracked("params.yaml")
@@ -372,10 +378,13 @@ def test_init_interactive_live(
     assert scm.is_ignored("models")
 
     out, err = capsys.readouterr()
+
     if interactive:
-        assert "'script.py' does not exist in the workspace." in err
-        assert "'data' does not exist in the workspace." in err
-    assert not out
+        assert "'script.py' does not exist, the file will be created." in err
+        assert "'data' does not exist, the directory will be created." in err
+    else:
+        assert "Using experiment project structure: " in out
+    assert "Created script.py and data" in out
 
 
 @pytest.mark.parametrize(
@@ -410,6 +419,8 @@ def test_init_with_type_live_and_models_plots_provided(
             }
         }
     }
+    assert (tmp_dir / "src").is_dir()
+    assert (tmp_dir / "data").is_dir()
 
 
 @pytest.mark.parametrize(
@@ -443,3 +454,5 @@ def test_init_with_type_default_and_live_provided(
             }
         }
     }
+    assert (tmp_dir / "src").is_dir()
+    assert (tmp_dir / "data").is_dir()