Repository: valohai/valohai-cli

Issue:
Allow passing a yaml path to the adhoc run commands
When starting an adhoc execution/task/pipeline, allow passing in a custom yaml path. 

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/valohai/valohai-cli/pull/195?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai) Report
> Merging [#195](https://codecov.io/gh/valohai/valohai-cli/pull/195?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai) (e550acc) into [master](https://codecov.io/gh/valohai/valohai-cli/commit/f24b166d68f2c5de1fc26c403c511964ae6ee870?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai) (f24b166) will **decrease** coverage by `0.08%`.
> The diff coverage is `90.00%`.

```diff
@@            Coverage Diff             @@
##           master     #195      +/-   ##
==========================================
- Coverage   85.76%   85.67%   -0.09%     
==========================================
  Files         110      110              
  Lines        3694     3727      +33     
  Branches      625      536      -89     
==========================================
+ Hits         3168     3193      +25     
- Misses        367      370       +3     
- Partials      159      164       +5     
```


| [Impacted Files](https://codecov.io/gh/valohai/valohai-cli/pull/195?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai) | Coverage Δ | |
|---|---|---|
| [valohai\_cli/commands/pipeline/run/run.py](https://codecov.io/gh/valohai/valohai-cli/pull/195/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai#diff-dmFsb2hhaV9jbGkvY29tbWFuZHMvcGlwZWxpbmUvcnVuL3J1bi5weQ==) | `87.50% <42.85%> (-10.18%)` | :arrow_down: |
| [tests/commands/execution/test\_run.py](https://codecov.io/gh/valohai/valohai-cli/pull/195/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai#diff-dGVzdHMvY29tbWFuZHMvZXhlY3V0aW9uL3Rlc3RfcnVuLnB5) | `100.00% <100.00%> (ø)` | |
| [tests/commands/pipeline/test\_run.py](https://codecov.io/gh/valohai/valohai-cli/pull/195/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai#diff-dGVzdHMvY29tbWFuZHMvcGlwZWxpbmUvdGVzdF9ydW4ucHk=) | `100.00% <100.00%> (ø)` | |
| [tests/commands/run\_test\_utils.py](https://codecov.io/gh/valohai/valohai-cli/pull/195/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai#diff-dGVzdHMvY29tbWFuZHMvcnVuX3Rlc3RfdXRpbHMucHk=) | `100.00% <100.00%> (ø)` | |
| [valohai\_cli/adhoc.py](https://codecov.io/gh/valohai/valohai-cli/pull/195/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai#diff-dmFsb2hhaV9jbGkvYWRob2MucHk=) | `82.53% <100.00%> (ø)` | |
| [...\_cli/commands/execution/run/dynamic\_run\_command.py](https://codecov.io/gh/valohai/valohai-cli/pull/195/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai#diff-dmFsb2hhaV9jbGkvY29tbWFuZHMvZXhlY3V0aW9uL3J1bi9keW5hbWljX3J1bl9jb21tYW5kLnB5) | `82.70% <100.00%> (+0.13%)` | :arrow_up: |
| [...hai\_cli/commands/execution/run/frontend\_command.py](https://codecov.io/gh/valohai/valohai-cli/pull/195/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai#diff-dmFsb2hhaV9jbGkvY29tbWFuZHMvZXhlY3V0aW9uL3J1bi9mcm9udGVuZF9jb21tYW5kLnB5) | `90.62% <100.00%> (-1.44%)` | :arrow_down: |
| [valohai\_cli/utils/commits.py](https://codecov.io/gh/valohai/valohai-cli/pull/195/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai#diff-dmFsb2hhaV9jbGkvdXRpbHMvY29tbWl0cy5weQ==) | `76.19% <100.00%> (+1.19%)` | :arrow_up: |
| [valohai\_cli/utils/file\_input.py](https://codecov.io/gh/valohai/valohai-cli/pull/195/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai#diff-dmFsb2hhaV9jbGkvdXRpbHMvZmlsZV9pbnB1dC5weQ==) | `69.23% <0.00%> (-7.70%)` | :arrow_down: |
| ... and [2 more](https://codecov.io/gh/valohai/valohai-cli/pull/195/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/valohai/valohai-cli/pull/195?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/valohai/valohai-cli/pull/195?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai). Last update [f24b166...e550acc](https://codecov.io/gh/valohai/valohai-cli/pull/195?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=valohai).


Patch:
diff --git a/valohai_cli/adhoc.py b/valohai_cli/adhoc.py
index 84a2d5c..79f74c7 100644
--- a/valohai_cli/adhoc.py
+++ b/valohai_cli/adhoc.py
@@ -14,7 +14,7 @@ from valohai_cli.utils.file_size_format import filesizeformat
 from valohai_cli.utils.hashing import get_fp_sha256
 
 
-def package_adhoc_commit(project: Project, validate: bool = True) -> Dict[str, Any]:
+def package_adhoc_commit(project: Project, validate: bool = True, yaml_path: Optional[str] = None) -> Dict[str, Any]:
     """
     Create an ad-hoc tarball and commit of the project directory.
 
@@ -37,7 +37,7 @@ def package_adhoc_commit(project: Project, validate: bool = True) -> Dict[str, A
         else:
             click.echo(f'Packaging {directory}...')
 
-        yaml_path = project.get_yaml_path()
+        yaml_path = yaml_path or project.get_yaml_path()
         tarball = package_directory(directory=directory, progress=True, validate=validate, yaml_path=yaml_path)
         return create_adhoc_commit_from_tarball(project=project, tarball=tarball, yaml_path=yaml_path, description=description)
     finally:
@@ -109,5 +109,6 @@ def _upload_commit_code(*, project: Project, tarball: str, yaml_path: str, descr
                 data=monitor,
                 headers={'Content-Type': monitor.content_type},
             ).json()
-    success(f"Uploaded ad-hoc code {commit_obj['identifier']}")
+    config_detail = f' from configuration YAML at {yaml_path}' if yaml_path else ''
+    success(f"Uploaded ad-hoc code {commit_obj['identifier']}{config_detail}")
     return commit_obj
diff --git a/valohai_cli/commands/execution/run/frontend_command.py b/valohai_cli/commands/execution/run/frontend_command.py
index fc8fa4d..883f74a 100644
--- a/valohai_cli/commands/execution/run/frontend_command.py
+++ b/valohai_cli/commands/execution/run/frontend_command.py
@@ -34,6 +34,7 @@ run_epilog = (
 @click.option('--sync', '-s', 'download_directory', type=click.Path(file_okay=False), help='Download execution outputs to DIRECTORY.', default=None)
 @click.option('--adhoc', '-a', is_flag=True, help='Upload the current state of the working directory, then run it as an ad-hoc execution.')
 @click.option('--validate-adhoc/--no-validate-adhoc', help='Enable or disable validation of adhoc packaged code, on by default', default=True)
+@click.option('--yaml', default=None, help='The path to the configuration YAML (valohai.yaml) file to use.')
 @click.option('--debug-port', type=int)
 @click.option('--debug-key-file', type=click.Path(file_okay=True, readable=True, writable=False))
 @click.argument('args', nargs=-1, type=click.UNPROCESSED, metavar='STEP-OPTIONS...')
@@ -44,6 +45,7 @@ def run(
     adhoc: bool,
     args: List[str],
     commit: Optional[str],
+    yaml: Optional[str],
     download_directory: Optional[str],
     environment: Optional[str],
     environment_variables: List[str],
@@ -82,7 +84,7 @@ def run(
     # We need to pass commit=None when adhoc=True to `get_config`, but
     # the further steps do need the real commit identifier from remote,
     # so this is done before `commit` is mangled by `create_adhoc_commit`.
-    config = project.get_config(commit_identifier=commit)
+    config = project.get_config(commit_identifier=commit, yaml_path=yaml)
     matched_step = match_step(config, step_name)
     step = config.steps[matched_step]
 
@@ -91,6 +93,7 @@ def run(
         commit=commit,
         adhoc=adhoc,
         validate_adhoc_commit=validate_adhoc,
+        yaml_path=yaml,
     )
 
     runtime_config = {}  # type: dict[str, Any]
diff --git a/valohai_cli/commands/pipeline/run/run.py b/valohai_cli/commands/pipeline/run/run.py
index 29799e0..d422449 100644
--- a/valohai_cli/commands/pipeline/run/run.py
+++ b/valohai_cli/commands/pipeline/run/run.py
@@ -21,8 +21,9 @@ from valohai_cli.utils.commits import create_or_resolve_commit
 @click.option('--commit', '-c', default=None, metavar='SHA', help='The commit to use. Defaults to the current HEAD.')
 @click.option('--title', '-c', default=None, help='The optional title of the pipeline run.')
 @click.option('--adhoc', '-a', is_flag=True, help='Upload the current state of the working directory, then run it as an ad-hoc execution.')
+@click.option('--yaml', default=None, help='The path to the configuration YAML file (valohai.yaml) file to use.')
 @click.pass_context
-def run(ctx: Context, name: Optional[str], commit: Optional[str], title: Optional[str], adhoc: bool) -> None:
+def run(ctx: Context, name: Optional[str], commit: Optional[str], title: Optional[str], adhoc: bool, yaml: Optional[str]) -> None:
     """
     Start a pipeline run.
     """
@@ -36,7 +37,10 @@ def run(ctx: Context, name: Optional[str], commit: Optional[str], title: Optiona
     project = get_project(require=True)
     assert project
 
-    commit = create_or_resolve_commit(project, commit=commit, adhoc=adhoc)
+    if yaml and not adhoc:
+        raise click.UsageError('--yaml is only valid with --adhoc')
+
+    commit = create_or_resolve_commit(project, commit=commit, adhoc=adhoc, yaml_path=yaml)
     config = project.get_config()
 
     matched_pipeline = match_pipeline(config, name)
diff --git a/valohai_cli/commands/yaml/pipeline.py b/valohai_cli/commands/yaml/pipeline.py
index 6b4f812..577424c 100644
--- a/valohai_cli/commands/yaml/pipeline.py
+++ b/valohai_cli/commands/yaml/pipeline.py
@@ -1,24 +1,23 @@
-from typing import List
+from typing import List, Optional
 
 import click
 from valohai.internals.pipeline import get_pipeline_from_source
 from valohai.yaml import config_to_yaml
-from valohai_yaml.objs.config import Config
 
 from valohai_cli.ctx import get_project
 from valohai_cli.exceptions import ConfigurationError
 from valohai_cli.messages import error, info
-from valohai_cli.models.project import Project
 
 
 @click.command()
+@click.option('--yaml', default=None, help='Path to the YAML file to update.')
 @click.argument(
     "filenames",
     nargs=-1,
     type=click.Path(file_okay=True, exists=True, dir_okay=False),
     required=True,
 )
-def pipeline(filenames: List[str]) -> None:
+def pipeline(filenames: List[str], yaml: Optional[str]) -> None:
     """
     Update a pipeline config(s) in valohai.yaml based on Python source file(s).
     Python source file is expected to have def main(config: Config) -> Pipeline
@@ -30,10 +29,16 @@ def pipeline(filenames: List[str]) -> None:
     :param filenames: Path(s) of the Python source code files.
     """
     project = get_project(require=True)
-    yaml_filename = project.get_config_filename()
+    yaml_filename = project.get_config_filename(yaml_path=yaml)
     did_update = False
     for source_path in filenames:
-        old_config = get_current_config(project)
+        try:
+            old_config = project.get_config(yaml_path=yaml)
+        except FileNotFoundError as fnfe:
+            raise ConfigurationError(
+                f"Did not find {yaml_filename}. "
+                f"Can't create a pipeline without preconfigured steps."
+            ) from fnfe
         try:
             new_config = get_pipeline_from_source(source_path, old_config)
         except Exception:
@@ -53,14 +58,3 @@ def pipeline(filenames: List[str]) -> None:
         info(f"{yaml_filename} updated.")
     else:
         info(f"{yaml_filename} already up-to-date.")
-
-
-def get_current_config(project: Project) -> Config:
-    try:
-        return project.get_config()
-    except FileNotFoundError as fnfe:
-        valohai_yaml_name = project.get_config_filename()
-        raise ConfigurationError(
-            f"Did not find {valohai_yaml_name}. "
-            f"Can't create a pipeline without preconfigured steps."
-        ) from fnfe
diff --git a/valohai_cli/commands/yaml/step.py b/valohai_cli/commands/yaml/step.py
index cfe7c92..6f87361 100644
--- a/valohai_cli/commands/yaml/step.py
+++ b/valohai_cli/commands/yaml/step.py
@@ -13,8 +13,9 @@ from valohai_cli.models.project import Project
 
 
 @click.command()
+@click.option('--yaml', default=None, help='Path to the YAML file to write to.')
 @click.argument('filenames', nargs=-1, type=click.Path(file_okay=True, exists=True, dir_okay=False), required=True)
-def step(filenames: List[str]) -> None:
+def step(filenames: List[str], yaml: Optional[str]) -> None:
     """
     Update a step config(s) in valohai.yaml based on Python source file(s).
 
@@ -30,29 +31,30 @@ def step(filenames: List[str]) -> None:
         project = Project(data={}, directory=os.getcwd())
         project.name = "YAML command simulated project"
 
-    config_path = project.get_config_filename()
+    config_path = project.get_config_filename(yaml_path=yaml)
+    yaml = yaml or project.get_yaml_path()
 
     for source_path in filenames:
         if not os.path.isfile(config_path):
-            update_yaml_from_source(source_path, project)
-            info("valohai.yaml generated.")
+            update_yaml_from_source(source_path, project, yaml)
+            info(f"{yaml} generated.")
             create_or_update_requirements(project)
-        elif yaml_needs_update(source_path, project):
-            update_yaml_from_source(source_path, project)
-            info("valohai.yaml updated.")
+        elif yaml_needs_update(source_path, project, yaml):
+            update_yaml_from_source(source_path, project, yaml)
+            info(f"{yaml} updated.")
             create_or_update_requirements(project)
         else:
-            info("valohai.yaml already up-to-date.")
+            info(f"{yaml} already up-to-date.")
 
 
-def get_current_config(project: Project) -> Optional[Config]:
+def get_current_config(project: Project, yaml_file: Optional[str] = None) -> Optional[Config]:
     try:
-        return project.get_config()
+        return project.get_config(yaml_path=yaml_file)
     except FileNotFoundError:
         return None
 
 
-def get_updated_config(source_path: str, project: Project) -> Config:
+def get_updated_config(source_path: str, project: Project, yaml_file: Optional[str] = None) -> Config:
     """Opens the old valohai.yaml, parses source Python file and merges the resulting config to the old
 
     Call to valohai.prepare() will contain step name, parameters and inputs.
@@ -63,15 +65,15 @@ def get_updated_config(source_path: str, project: Project) -> Config:
     :param project: Currently linked Valohai project
 
     """
-    old_config = get_current_config(project)
-    new_config = parse_config_from_source(source_path, project.get_config_filename())
+    old_config = get_current_config(project, yaml_file)
+    new_config = parse_config_from_source(source_path, project.get_config_filename(yaml_file))
     if old_config:
         return old_config.merge_with(new_config, python_to_yaml_merge_strategy)
     assert isinstance(new_config, Config)
     return new_config
 
 
-def update_yaml_from_source(source_path: str, project: Project) -> bool:
+def update_yaml_from_source(source_path: str, project: Project, yaml_file: Optional[str]) -> bool:
     """Updates valohai.yaml by parsing the source code file for a call to valohai.prepare()
 
     Call to valohai.prepare() will contain step name, parameters and inputs.
@@ -82,17 +84,17 @@ def update_yaml_from_source(source_path: str, project: Project) -> bool:
     :param project: Currently linked Valohai project
 
     """
-    old_config = get_current_config(project)
-    new_config = get_updated_config(source_path, project)
+    old_config = get_current_config(project, yaml_file)
+    new_config = get_updated_config(source_path, project, yaml_file)
     if old_config != new_config:
         project.refresh_details()
-        with open(project.get_config_filename(), 'w') as out_file:
+        with open(project.get_config_filename(yaml_path=yaml_file), 'w') as out_file:
             out_file.write(config_to_yaml(new_config))
         return True
     return False
 
 
-def yaml_needs_update(source_path: str, project: Project) -> bool:
+def yaml_needs_update(source_path: str, project: Project, yaml_file: Optional[str]) -> bool:
     """Checks if valohai.yaml needs updating based on source Python code.
 
     Call to valohai.prepare() will contain step name, parameters and inputs.
@@ -103,8 +105,8 @@ def yaml_needs_update(source_path: str, project: Project) -> bool:
     :param project: Currently linked Valohai project
 
     """
-    old_config = get_current_config(project)
-    new_config = get_updated_config(source_path, project)
+    old_config = get_current_config(project, yaml_file)
+    new_config = get_updated_config(source_path, project, yaml_file)
 
     if not old_config or not new_config:
         return True
diff --git a/valohai_cli/models/project.py b/valohai_cli/models/project.py
index 72e80bc..d3b0109 100644
--- a/valohai_cli/models/project.py
+++ b/valohai_cli/models/project.py
@@ -29,22 +29,24 @@ class Project:
     def name(self) -> str:
         return str(self.data['name'])
 
-    def get_config(self, commit_identifier: Optional[str] = None) -> Config:
+    def get_config(self, commit_identifier: Optional[str] = None, yaml_path: Optional[str] = None) -> Config:
         """
         Get the `valohai_yaml.Config` object from the current working directory,
         or a given commit.
 
         :param commit_identifier: Hexadecimal commit identifier; optional.
+        :param yaml_path: Override of the project's `yaml_path`; optional.
         :return: valohai_yaml.Config
         """
+        yaml_path = yaml_path or self.get_yaml_path()
         if not commit_identifier:  # Current working directory
-            filename = self.get_config_filename()
+            filename = self.get_config_filename(yaml_path=yaml_path)
             with open(filename) as infp:
                 return self._parse_config(infp, filename)
         else:  # Arbitrary commit
-            filename = f'{commit_identifier}:valohai.yaml'
+            filename = f'{commit_identifier}:{yaml_path}'
             directory = self.directory
-            config_bytes = get_file_at_commit(directory, commit_identifier, 'valohai.yaml')
+            config_bytes = get_file_at_commit(directory, commit_identifier, yaml_path)
             config_sio = io.StringIO(config_bytes.decode('utf-8'))
             return self._parse_config(config_sio, filename)
 
@@ -59,8 +61,9 @@ class Project:
                 n=len(ves.errors),
             ))
 
-    def get_config_filename(self) -> str:
-        return os.path.join(self.directory, self.get_yaml_path())
+    def get_config_filename(self, yaml_path: Optional[str] = None) -> str:
+        used_yaml_path = yaml_path or self.get_yaml_path()
+        return os.path.join(self.directory, used_yaml_path)
 
     def get_yaml_path(self) -> str:
         # Make sure the older API versions that don't expose yaml_path don't completely break
diff --git a/valohai_cli/models/remote_project.py b/valohai_cli/models/remote_project.py
index f30ae17..a5452f8 100644
--- a/valohai_cli/models/remote_project.py
+++ b/valohai_cli/models/remote_project.py
@@ -8,7 +8,7 @@ from valohai_cli.models.project import Project
 class RemoteProject(Project):
     is_remote = True
 
-    def get_config(self, commit_identifier: Optional[str] = None) -> Config:
+    def get_config(self, commit_identifier: Optional[str] = None, yaml_path: Optional[str] = None) -> Config:  # NOQA U100
         if not commit_identifier:
             raise ValueError('RemoteProjects require an explicit commit identifier')
         commit = self.load_full_commit(commit_identifier)
@@ -16,5 +16,5 @@ class RemoteProject(Project):
             raise ValueError(f'No configuration found for commit {commit_identifier}')
         return self._parse_config(commit['config'], filename='<remote config>')
 
-    def get_config_filename(self) -> str:  # pragma: no cover  # typing: ignore[override]
+    def get_config_filename(self, yaml_path: Optional[str] = None) -> str:  # pragma: no cover  # typing: ignore[override]
         raise NotImplementedError('RemoteProject.get_config_filename() should never get called')
diff --git a/valohai_cli/utils/commits.py b/valohai_cli/utils/commits.py
index be390b5..a0469e3 100644
--- a/valohai_cli/utils/commits.py
+++ b/valohai_cli/utils/commits.py
@@ -14,6 +14,7 @@ def create_or_resolve_commit(
     project: Union[RemoteProject, Project],
     *,
     commit: Optional[str],
+    yaml_path: Optional[str],
     adhoc: bool,
     validate_adhoc_commit: bool = True,
 ) -> str:
@@ -22,7 +23,9 @@ def create_or_resolve_commit(
             raise click.UsageError('--adhoc can not be used with remote projects.')
         if commit:
             raise click.UsageError('--commit and --adhoc are mutually exclusive.')
-        commit = str(package_adhoc_commit(project, validate=validate_adhoc_commit)['identifier'])
+        commit = str(package_adhoc_commit(project, validate=validate_adhoc_commit, yaml_path=yaml_path)['identifier'])
+    elif yaml_path:
+        raise click.UsageError('--yaml can only be used with --adhoc.')
 
     commit = commit or get_git_commit(project)
     return resolve_commit(commit, project)


Test patch:
diff --git a/tests/commands/execution/test_run.py b/tests/commands/execution/test_run.py
index d33e421..6352b86 100644
--- a/tests/commands/execution/test_run.py
+++ b/tests/commands/execution/test_run.py
@@ -1,7 +1,7 @@
 import pytest
 import yaml
 
-from tests.commands.run_test_utils import RunAPIMock, RunTestSetup
+from tests.commands.run_test_utils import ALTERNATIVE_YAML, RunAPIMock, RunTestSetup
 from tests.fixture_data import CONFIG_YAML, PROJECT_DATA
 from valohai_cli.commands.execution.run import run
 from valohai_cli.ctx import get_project
@@ -58,6 +58,19 @@ def test_run_tags(run_test_setup):
     run_test_setup.run()
 
 
+def test_run_with_yaml_path(run_test_setup):
+    run_test_setup.args.remove('train')
+    # Use a step which is only present in the evaluation YAML
+    run_test_setup.args.append('batch feature extraction')
+    run_test_setup.args.append(f'--yaml={ALTERNATIVE_YAML}')
+    output = run_test_setup.run(verify_adhoc=run_test_setup.adhoc)
+    # Adhoc success case already verified in `run()
+    if not run_test_setup.adhoc:
+        assert '--yaml can only be used with --adhoc' in output
+    else:
+        assert f'from configuration YAML at {ALTERNATIVE_YAML}' in output
+
+
 def test_run_input(run_test_setup):
     run_test_setup.args.append('--in1=http://url')
     run_test_setup.args.append('--in1=http://anotherurl')
diff --git a/tests/commands/pipeline/test_run.py b/tests/commands/pipeline/test_run.py
index 91874ec..2752698 100644
--- a/tests/commands/pipeline/test_run.py
+++ b/tests/commands/pipeline/test_run.py
@@ -11,7 +11,7 @@ from valohai_cli.ctx import get_project
 def test_pipeline_run_success(runner, logged_in_and_linked):
     add_valid_pipeline_yaml()
     args = ['training']
-    with RunAPIMock(PROJECT_DATA.get('id')):
+    with RunAPIMock(PROJECT_DATA['id']):
         output = runner.invoke(run, args).output
     assert 'Success' in output
 
@@ -19,17 +19,26 @@ def test_pipeline_run_success(runner, logged_in_and_linked):
 def test_pipeline_adhoc_run_success(runner, logged_in_and_linked):
     add_valid_pipeline_yaml()
     args = ['--adhoc', 'training']
-    with RunAPIMock(PROJECT_DATA.get('id')):
+    with RunAPIMock(PROJECT_DATA['id']):
         print(run, args)
         output = runner.invoke(run, args).output
     assert 'Success' in output
     assert 'Uploaded ad-hoc code' in output
 
 
+def test_pipeline_adhoc_with_yaml_path_run_success(runner, logged_in_and_linked):
+    add_valid_pipeline_yaml(yaml_path='bark.yaml')
+    args = ['--adhoc', '--yaml=bark.yaml', 'training']
+    with RunAPIMock(PROJECT_DATA['id']):
+        output = runner.invoke(run, args).output
+    assert 'Success' in output
+    assert 'Uploaded ad-hoc code' in output
+
+
 def test_pipeline_run_no_name(runner, logged_in_and_linked):
     add_valid_pipeline_yaml()
     args = ['']
-    with RunAPIMock(PROJECT_DATA.get('id')):
+    with RunAPIMock(PROJECT_DATA['id']):
         output = runner.invoke(run, args).output
     assert 'Usage: ' in output
 
@@ -48,6 +57,8 @@ def test_match_pipeline_ambiguous(runner, logged_in_and_linked):
         match_pipeline(config, 'Train')
 
 
-def add_valid_pipeline_yaml():
-    with open(get_project().get_config_filename(), 'w') as yaml_fp:
+def add_valid_pipeline_yaml(yaml_path=None):
+    project = get_project()
+    config_filename = project.get_config_filename(yaml_path=yaml_path)
+    with open(config_filename, 'w') as yaml_fp:
         yaml_fp.write(PIPELINE_YAML)
diff --git a/tests/commands/run_test_utils.py b/tests/commands/run_test_utils.py
index 016063a..64a9252 100644
--- a/tests/commands/run_test_utils.py
+++ b/tests/commands/run_test_utils.py
@@ -5,12 +5,21 @@ import re
 import requests_mock
 from click.testing import CliRunner
 
-from tests.fixture_data import CONFIG_DATA, CONFIG_YAML, EXECUTION_DATA, PIPELINE_DATA, PROJECT_DATA
+from tests.fixture_data import (
+    CONFIG_DATA,
+    CONFIG_YAML,
+    EXECUTION_DATA,
+    PIPELINE_DATA,
+    PROJECT_DATA,
+    YAML_WITH_EXTRACT_TRAIN_EVAL,
+)
 from valohai_cli import git
 from valohai_cli.commands.execution.run import run
 from valohai_cli.ctx import get_project
 from valohai_cli.utils import get_random_string
 
+ALTERNATIVE_YAML = 'batch.yaml'
+
 
 class RunAPIMock(requests_mock.Mocker):
 
@@ -77,7 +86,7 @@ class RunAPIMock(requests_mock.Mocker):
     def handle_create_execution(self, request, context):
         body_json = json.loads(request.body.decode('utf-8'))
         assert body_json['project'] == self.project_id
-        assert body_json['step'] == 'Train model'
+        assert body_json['step'] in ('Train model', 'Batch feature extraction')
         assert body_json['commit'] == self.commit_id
         for key, expected_value in self.additional_payload_values.items():
             body_value = body_json[key]
@@ -117,6 +126,10 @@ class RunTestSetup:
         with open(get_project().get_config_filename(), 'w') as yaml_fp:
             yaml_fp.write(CONFIG_YAML)
 
+        # Create an alternative yaml as well (monorepo style)
+        with open(get_project().get_config_filename(yaml_path=ALTERNATIVE_YAML), 'w') as yaml_fp:
+            yaml_fp.write(YAML_WITH_EXTRACT_TRAIN_EVAL)
+
         self.args = [step_name]
         if adhoc:
             self.args.insert(0, '--adhoc')
diff --git a/tests/commands/yaml/test_pipeline.py b/tests/commands/yaml/test_pipeline.py
index c28103c..c866b92 100644
--- a/tests/commands/yaml/test_pipeline.py
+++ b/tests/commands/yaml/test_pipeline.py
@@ -1,5 +1,8 @@
 import os
 
+import pytest
+
+from tests.commands.yaml.utils import build_args
 from tests.fixture_data import (
     PYTHON_SOURCE,
     PYTHON_SOURCE_DEFINING_PIPELINE,
@@ -10,40 +13,44 @@ from valohai_cli.commands.yaml.pipeline import pipeline
 from valohai_cli.ctx import get_project
 
 
-def test_pipeline(runner, logged_in_and_linked):
+@pytest.mark.parametrize('yaml_path', ['bark.yaml', None])
+def test_pipeline(runner, logged_in_and_linked, yaml_path):
     # Create config with only two steps
-    config_path = os.path.join(get_project().directory, 'valohai.yaml')
+    project = get_project()
+    config_path = project.get_config_filename(yaml_path)
+    yaml_path = yaml_path or project.get_yaml_path()
+
     with open(config_path, 'w') as yaml_fp:
         yaml_fp.write(YAML_WITH_TRAIN_EVAL)
 
     # Try to generate pipeline from random .py source code.
-    source_path = os.path.join(get_project().directory, 'random.py')
+    source_path = os.path.join(project.directory, 'random.py')
     with open(source_path, 'w') as python_fp:
         python_fp.write(PYTHON_SOURCE)
-    args = ([source_path])
+    args = build_args(source_path, yaml_path)
     rv = runner.invoke(pipeline, args, catch_exceptions=True)
     # The .py source was random so we should get an error
     assert isinstance(rv.exception, AttributeError)
 
     # Try generating pipeline from .py source code that is correctly using valohai-utils & papi
-    source_path = os.path.join(get_project().directory, 'mysnake.py')
+    source_path = os.path.join(project.directory, 'mysnake.py')
     with open(source_path, 'w') as python_fp:
         python_fp.write(PYTHON_SOURCE_DEFINING_PIPELINE)
-    args = ([source_path])
+    args = build_args(source_path, yaml_path)
     rv = runner.invoke(pipeline, args, catch_exceptions=True)
     # The current config is missing one of the three steps on purpose. This should raise an exception.
     assert isinstance(rv.exception, ValueError)
 
     # Now re-create config with all the three steps
-    config_path = os.path.join(get_project().directory, 'valohai.yaml')
+    config_path = os.path.join(project.directory, yaml_path)
     with open(config_path, 'w') as yaml_fp:
         yaml_fp.write(YAML_WITH_EXTRACT_TRAIN_EVAL)
 
     # With all the three steps, we are expecting great success
-    args = ([source_path])
+    args = build_args(source_path, yaml_path)
     rv = runner.invoke(pipeline, args, catch_exceptions=False)
-    assert "valohai.yaml updated." in rv.output
+    assert f"{yaml_path} updated." in rv.output
 
     # Try the same update again
     rv = runner.invoke(pipeline, args, catch_exceptions=False)
-    assert "valohai.yaml already up-to-date." in rv.output
+    assert f"{yaml_path} already up-to-date." in rv.output
diff --git a/tests/commands/yaml/test_step.py b/tests/commands/yaml/test_step.py
index 09523d6..b435c29 100644
--- a/tests/commands/yaml/test_step.py
+++ b/tests/commands/yaml/test_step.py
@@ -1,35 +1,40 @@
 import os
 
+import pytest
+
+from tests.commands.yaml.utils import build_args
 from tests.fixture_data import CONFIG_YAML, PYTHON_SOURCE, PYTHON_SOURCE_USING_VALOHAI_UTILS
 from valohai_cli.commands.yaml.step import step
 from valohai_cli.ctx import get_project
 
 
-def test_yaml(runner, logged_in_and_linked, default_run_api_mock):
+@pytest.mark.parametrize('yaml_path', ['bark.yaml', None])
+def test_yaml(runner, logged_in_and_linked, default_run_api_mock, yaml_path):
     # Try to generate YAML from random .py source code
-    source_path = os.path.join(get_project().directory, 'mysnake.py')
+    project = get_project()
+    source_path = os.path.join(project.directory, 'mysnake.py')
+    yaml_path = yaml_path or project.get_yaml_path()
     with open(source_path, 'w') as python_fp:
         python_fp.write(PYTHON_SOURCE)
-    args = ([source_path])
+    args = build_args(source_path, yaml_path)
     rv = runner.invoke(step, args, catch_exceptions=True)
     assert isinstance(rv.exception, ValueError)
 
     # Generate YAML from .py source code that is using valohai-utils
-    source_path = os.path.join(get_project().directory, 'mysnake.py')
     with open(source_path, 'w') as python_fp:
         python_fp.write(PYTHON_SOURCE_USING_VALOHAI_UTILS)
-    args = ([source_path])
+    args = build_args(source_path, yaml_path)
     rv = runner.invoke(step, args, catch_exceptions=False)
-    assert "valohai.yaml generated." in rv.output
+    assert f"{yaml_path} generated." in rv.output
 
     # Update existing YAML from source code
-    config_path = os.path.join(get_project().directory, 'valohai.yaml')
+    config_path = project.get_config_filename(yaml_path=yaml_path)
     with open(config_path, 'w') as yaml_fp:
         yaml_fp.write(CONFIG_YAML)
-    args = ([source_path])
+    args = build_args(source_path, yaml_path)
     rv = runner.invoke(step, args, catch_exceptions=False)
-    assert "valohai.yaml updated." in rv.output
+    assert f"{yaml_path} updated." in rv.output
 
     # Try the same update again
     rv = runner.invoke(step, args, catch_exceptions=False)
-    assert "valohai.yaml already up-to-date." in rv.output
+    assert f"{yaml_path} already up-to-date." in rv.output
diff --git a/tests/commands/yaml/utils.py b/tests/commands/yaml/utils.py
new file mode 100644
index 0000000..7fc58e7
--- /dev/null
+++ b/tests/commands/yaml/utils.py
@@ -0,0 +1,8 @@
+from typing import Optional
+
+
+def build_args(source_path: str, yaml_path: Optional[str] = None) -> list:
+    args = [source_path]
+    if yaml_path:
+        args.append(f'--yaml={yaml_path}')
+    return args