Repository: getindata/data-pipelines-cli (language: python)

Issue:
Add a command for saving `profiles.yml` in $HOME
`dp gen-home-profiles` generates and saves `profiles.yml` in `$HOME/.dbt`.

---
Keep in mind:
- [X] Documentation updates
- [X] [Changelog](CHANGELOG.md) updates
Release 0.7.0
Bump version and CHANGELOG for next release.


Discussion / hints:

Fix patch:
diff --git a/.codeclimate.yml b/.codeclimate.yml
index dbf9a37..b1b6288 100644
--- a/.codeclimate.yml
+++ b/.codeclimate.yml
@@ -1,3 +1,50 @@
+version: "2"         # required to adjust maintainability checks
+
+checks:
+  argument-count:
+    enabled: true
+    config:
+      threshold: 4
+  complex-logic:
+    enabled: true
+    config:
+      threshold: 4
+  file-lines:
+    enabled: true
+    config:
+      threshold: 250
+  method-complexity:
+    enabled: true
+    config:
+      threshold: 5
+  method-count:
+    enabled: true
+    config:
+      threshold: 20
+  method-lines:
+    enabled: true
+    config:
+      threshold: 25
+  nested-control-flow:
+    enabled: true
+    config:
+      threshold: 4
+  return-statements:
+    enabled: true
+    config:
+      threshold: 4
+  similar-code:
+    enabled: true
+    config:
+      threshold: #language-specific defaults. overrides affect all languages.
+  identical-code:
+    enabled: true
+    config:
+      threshold: #language-specific defaults. overrides affect all languages.
+
 plugins:
   pylint:
     enabled: true
+    checks:
+      import-error:
+        enabled: false
diff --git a/.github/workflows/python-package.yml b/.github/workflows/python-package.yml
index 864c956..e71ac10 100644
--- a/.github/workflows/python-package.yml
+++ b/.github/workflows/python-package.yml
@@ -25,12 +25,23 @@ jobs:
           virtualenv venv
           source venv/bin/activate
           pip install --upgrade pip
+
       - name: Check pre-commit status
         run: |
           pip install .[tests]
           pip freeze
           pipdeptree
           pre-commit run --all-files
+
       - name: Test with tox
         run: |
           tox -e py37
+
+      - name: Report coverage
+        uses: paambaati/codeclimate-action@v2.7.5
+        env:
+          CC_TEST_REPORTER_ID: ${{ secrets.CODE_CLIMATE }}
+        with:
+          coverageCommand: coverage xml
+          debug: true
+          coverageLocations: coverage.xml:coverage.py
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index f5ef28d..f453fc3 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -6,17 +6,25 @@ repos:
     - id: isort
 
 - repo: https://github.com/psf/black
-  rev: 21.11b1
+  rev: 21.12b0
   hooks:
     - id: black
 
 - repo: https://github.com/pre-commit/pre-commit-hooks
-  rev: v2.5.0
+  rev: v4.1.0
   hooks:
     - id: trailing-whitespace
       args: [--markdown-linebreak-ext=md]
     - id: check-merge-conflict
     - id: debug-statements
+    - id: name-tests-test
+      args: ['--django']  # PyCharm requires tests to have 'test' as prefix, not a suffix
+    - id: check-json
+    - id: check-yaml
+
+- repo: https://github.com/pycqa/flake8
+  rev: 4.0.1
+  hooks:
     - id: flake8
       additional_dependencies: [
         'flake8-blind-except',
@@ -25,7 +33,7 @@ repos:
       ]
 
 - repo: https://github.com/pre-commit/mirrors-mypy
-  rev: v0.910-1
+  rev: v0.930
   hooks:
     - id: mypy
       additional_dependencies:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8577615..1da0e67 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,8 +2,26 @@
 
 ## [Unreleased]
 
+## [0.7.0] - 2021-12-29
+
 ### Added
-- Add documentation in the style of [Read the Docs](https://readthedocs.org/).
+
+-   Add documentation in the style of [Read the Docs](https://readthedocs.org/).
+-   Exception classes in `errors.py`, deriving from `DataPipelinesError` base exception class.
+-   Unit tests to massively improve code coverage.
+-   `--version` flag to **dp** command.
+-   Add `dp prepare-env` command that prepares local environment for standalone **dbt** (right now, it only generates and saves `profiles.yml` in `$HOME/.dbt`).
+
+### Changed
+
+-   `dp compile`:
+    -   `--env` option has a default value: `base`,
+    -   `--datahub` is changed to `--datahub-gms-uri`, `--repository` is changed to `--docker-repository-uri`.
+-   `dp deploy`'s `--docker-push` is not a flag anymore and requires a Docker repository URI parameter; `--repository` got removed then.
+-   `dp run` and `dp test` run `dp compile` before actual **dbt** command.
+-   Functions raise exceptions instead of exiting using `sys.exit(1)`; `cli.cli()` entrypoint is expecting exception and exits only there.
+-   `dp deploy` raises an exception if there is no Docker image to push or `build/config/dag` directory does not exist.
+-   Rename `gcp` to `gcs` in requirements (now one should run `pip install data-pipelines-cli[gcs]`).
 
 ## [0.6.0] - 2021-12-16
 
@@ -80,7 +98,9 @@
 -   Draft of `dp init`, `dp create`, `dp template new`, `dp template list` and `dp dbt`
 -   Draft of `dp compile` and `dp deploy`
 
-[Unreleased]: https://github.com/getindata/data-pipelines-cli/compare/0.6.0...HEAD
+[Unreleased]: https://github.com/getindata/data-pipelines-cli/compare/0.7.0...HEAD
+
+[0.7.0]: https://github.com/getindata/data-pipelines-cli/compare/0.6.0...0.7.0
 
 [0.6.0]: https://github.com/getindata/data-pipelines-cli/compare/0.5.1...0.6.0
 
diff --git a/README.md b/README.md
index 04b949b..bacc6cd 100644
--- a/README.md
+++ b/README.md
@@ -1,12 +1,23 @@
 # data-pipelines-cli
 
+[![Python Version](https://img.shields.io/badge/python-3.7%20%7C%203.8%20%7C%203.9-blue.svg)](https://github.com/getindata/data-pipelines-cli)
+[![PyPI Version](https://badge.fury.io/py/data-pipelines-cli.svg)](https://pypi.org/project/data-pipelines-cli/)
+[![Downloads](https://pepy.tech/badge/data-pipelines-cli)](https://pepy.tech/project/data-pipelines-cli)
+[![Maintainability](https://api.codeclimate.com/v1/badges/e44ed9383a42b59984f6/maintainability)](https://codeclimate.com/github/getindata/data-pipelines-cli/maintainability)
+[![Test Coverage](https://api.codeclimate.com/v1/badges/e44ed9383a42b59984f6/test_coverage)](https://codeclimate.com/github/getindata/data-pipelines-cli/test_coverage)
+[![Documentation Status](https://readthedocs.org/projects/data-pipelines-cli/badge/?version=latest)](https://data-pipelines-cli.readthedocs.io/en/latest/?badge=latest)
+
 CLI for data platform
 
+## Documentation
+
+Read the full documentation at [https://data-pipelines-cli.readthedocs.io/](https://data-pipelines-cli.readthedocs.io/en/latest/index.html)
+
 ## Installation
 Use the package manager [pip](https://pip.pypa.io/en/stable/) to install [dp (data-pipelines-cli)](https://pypi.org/project/data-pipelines-cli/):
 
 ```bash
-pip install data-pipelines-cli[docker,datahub,gcp]
+pip install data-pipelines-cli[docker,datahub,gcs]
 ```
 
 ## Usage
@@ -39,12 +50,15 @@ directly to a template repository. If `<LINK_TO_TEMPLATE_REPOSITORY>` proves to
 ### Project deployment
 
 `dp deploy` will sync with your bucket provider. The provider will be chosen automatically based on the remote URL.
-`dp deploy` also requires the user to point to JSON or YAML file with provider-specific data like access tokens or project
+Usually, it is worth pointing `dp deploy` to JSON or YAML file with provider-specific data like access tokens or project
 names. E.g., to connect with Google Cloud Storage, one should run:
 ```bash
 echo '{"token": "<PATH_TO_YOUR_TOKEN>", "project_name": "<YOUR_PROJECT_NAME>"}' > gs_args.json
 dp deploy "gs://<YOUR_GS_PATH>" --blob-args gs_args.json
 ```
+However, in some cases you do not need to do so, e.g. when using `gcloud` with properly set local credentials. In such
+case, you can try to run just the `dp deploy "gs://<YOUR_GS_PATH>"` command. Please refer to
+[documentation](https://data-pipelines-cli.readthedocs.io/en/latest/usage.html#project-deployment) for more information.
 
 When finished, call `dp clean` to remove compilation related directories.
 
diff --git a/data_pipelines_cli/__init__.py b/data_pipelines_cli/__init__.py
index aaf05b3..7a0c9fe 100644
--- a/data_pipelines_cli/__init__.py
+++ b/data_pipelines_cli/__init__.py
@@ -1,1 +1,1 @@
-version = "0.6.0"
+version = "0.7.0"
diff --git a/data_pipelines_cli/cli.py b/data_pipelines_cli/cli.py
index 0470aac..8648024 100644
--- a/data_pipelines_cli/cli.py
+++ b/data_pipelines_cli/cli.py
@@ -1,3 +1,5 @@
+import sys
+
 import click
 
 from .cli_commands.clean import clean_command
@@ -5,21 +7,34 @@
 from .cli_commands.create import create_command
 from .cli_commands.deploy import deploy_command
 from .cli_commands.init import init_command
+from .cli_commands.prepare_env import prepare_env_command
 from .cli_commands.run import run_command
 from .cli_commands.template import list_templates_command
 from .cli_commands.test import test_command
+from .cli_utils import echo_error
+from .errors import DataPipelinesError
 
 
 @click.group()
-def cli() -> None:
+@click.version_option(prog_name="dp")
+def _cli() -> None:
     pass
 
 
-cli.add_command(clean_command)
-cli.add_command(compile_project_command)
-cli.add_command(create_command)
-cli.add_command(deploy_command)
-cli.add_command(init_command)
-cli.add_command(run_command)
-cli.add_command(list_templates_command)
-cli.add_command(test_command)
+def cli() -> None:
+    try:
+        _cli()
+    except DataPipelinesError as err:
+        echo_error(f"CLI Error: {err.message}")
+        sys.exit(1)
+
+
+_cli.add_command(clean_command)
+_cli.add_command(compile_project_command)
+_cli.add_command(create_command)
+_cli.add_command(deploy_command)
+_cli.add_command(init_command)
+_cli.add_command(prepare_env_command)
+_cli.add_command(run_command)
+_cli.add_command(list_templates_command)
+_cli.add_command(test_command)
diff --git a/data_pipelines_cli/cli_commands/compile.py b/data_pipelines_cli/cli_commands/compile.py
index 89871f7..e7de109 100644
--- a/data_pipelines_cli/cli_commands/compile.py
+++ b/data_pipelines_cli/cli_commands/compile.py
@@ -1,6 +1,5 @@
 import pathlib
 import shutil
-import sys
 from typing import Optional
 
 import click
@@ -12,7 +11,7 @@
     IMAGE_TAG_TO_REPLACE,
     INGEST_ENDPOINT_TO_REPLACE,
 )
-from ..cli_utils import echo_error, echo_info, get_argument_or_environment_variable
+from ..cli_utils import echo_info, echo_warning, get_argument_or_environment_variable
 from ..config_generation import (
     copy_config_dir_to_build_dir,
     copy_dag_dir_to_build_dir,
@@ -20,11 +19,14 @@
 )
 from ..data_structures import DockerArgs
 from ..dbt_utils import run_dbt_command
+from ..errors import DataPipelinesError, DockerNotInstalledError
 from ..io_utils import replace
 
 
 def _replace_image_tag(k8s_config: pathlib.Path, docker_args: DockerArgs) -> None:
-    echo_info(f"Replacing <IMAGE_TAG> with commit SHA = {docker_args.commit_sha}")
+    echo_info(
+        f"Replacing {IMAGE_TAG_TO_REPLACE} with commit SHA = {docker_args.commit_sha}"
+    )
     replace(k8s_config, IMAGE_TAG_TO_REPLACE, docker_args.commit_sha)
 
 
@@ -32,20 +34,22 @@ def _replace_docker_repository_url(
     k8s_config: pathlib.Path, docker_args: DockerArgs
 ) -> None:
     echo_info(
-        "Replacing <DOCKER_REPOSITORY_URL> with repository URL = "
+        f"Replacing {DOCKER_REPOSITORY_URL_TO_REPLACE} with repository URL = "
         f"{docker_args.repository}"
     )
     replace(k8s_config, DOCKER_REPOSITORY_URL_TO_REPLACE, docker_args.repository)
 
 
 def _docker_build(docker_args: DockerArgs) -> None:
+    """
+    :param docker_args: Arguments required by the Docker to make a push to \
+        the repository
+    :raises DataPipelinesError: Docker not installed
+    """
     try:
         import docker
     except ModuleNotFoundError:
-        echo_error(
-            "'docker' not installed. Run 'pip install data-pipelines-cli[docker]'"
-        )
-        sys.exit(1)
+        raise DockerNotInstalledError()
 
     echo_info("Building Docker image")
     docker_client = docker.from_env()
@@ -78,59 +82,86 @@ def _copy_dbt_manifest() -> None:
     )
 
 
-def _replace_datahub_address(datahub_address: str) -> None:
-    echo_info(f"Replacing INGEST_ENDPOINT with datahub address = {datahub_address}")
+def _try_replace_datahub_address(datahub_gms_uri: Optional[str]) -> None:
+    try:
+        datahub_gms_uri = get_argument_or_environment_variable(
+            datahub_gms_uri, "datahub-gms-uri", DATAHUB_URL_ENV
+        )
+    except DataPipelinesError as err:
+        echo_warning(
+            f"{err.message}\n{INGEST_ENDPOINT_TO_REPLACE} will not be replaced"
+        )
+        return
+
+    echo_info(
+        f"Replacing {INGEST_ENDPOINT_TO_REPLACE} with DataHub URI = {datahub_gms_uri}"
+    )
     replace(
         BUILD_DIR.joinpath("dag", "config", "base", "datahub.yml"),
         INGEST_ENDPOINT_TO_REPLACE,
-        datahub_address,
+        datahub_gms_uri,
     )
 
 
+def _replace_k8s_settings(docker_args: DockerArgs) -> None:
+    k8s_config: pathlib.Path = BUILD_DIR.joinpath("dag", "config", "base", "k8s.yml")
+    _replace_image_tag(k8s_config, docker_args)
+    _replace_docker_repository_url(k8s_config, docker_args)
+
+
 def compile_project(
-    repository: Optional[str],
-    datahub: Optional[str],
-    docker_build: bool,
     env: str,
+    docker_repository_uri: Optional[str] = None,
+    datahub_gms_uri: Optional[str] = None,
+    docker_build: bool = False,
 ) -> None:
     """
     Create local working directories and build artifacts
 
-    :param repository: URI of the Docker repository
-    :type repository: Optional[str]
-    :param datahub: URI of the DataHub ingestion endpoint
-    :type datahub: Optional[str]
-    :param docker_build: Whether to build a Docker image
-    :type docker_build: bool
     :param env: Name of the environment
     :type env: str
+    :param docker_repository_uri: URI of the Docker repository
+    :type docker_repository_uri: Optional[str]
+    :param datahub_gms_uri: URI of the DataHub ingestion endpoint
+    :type datahub_gms_uri: Optional[str]
+    :param docker_build: Whether to build a Docker image
+    :type docker_build: bool
+    :raises DataPipelinesError:
     """
-    datahub_address = get_argument_or_environment_variable(
-        datahub, "datahub", DATAHUB_URL_ENV
-    )
-    docker_args = DockerArgs(repository)
-
     copy_dag_dir_to_build_dir()
     copy_config_dir_to_build_dir()
 
-    k8s_config: pathlib.Path = BUILD_DIR.joinpath("dag", "config", "base", "k8s.yml")
-    _replace_image_tag(k8s_config, docker_args)
-    _replace_docker_repository_url(k8s_config, docker_args)
+    docker_args = None
+    if docker_repository_uri:
+        docker_args = DockerArgs(docker_repository_uri)
+        _replace_k8s_settings(docker_args)
 
     _dbt_compile(env)
     _copy_dbt_manifest()
-    _replace_datahub_address(datahub_address)
+    _try_replace_datahub_address(datahub_gms_uri)
 
-    if docker_build:
+    if docker_build and docker_args:
         _docker_build(docker_args)
 
 
 @click.command(
-    name="compile", help="Create local working directories and build artifacts"
+    name="compile",
+    help="Create local working directories and build artifacts",
+)
+@click.option(
+    "--env",
+    default="base",
+    type=str,
+    show_default=True,
+    required=True,
+    help="Name of the environment",
+)
+@click.option(
+    "--docker-repository-uri", default=None, help="URI of the Docker repository"
+)
+@click.option(
+    "--datahub-gms-uri", default=None, help="URI of the DataHub ingestion endpoint"
 )
-@click.option("--env", required=True, help="Name of the environment")
-@click.option("--repository", default=None, help="URI of the Docker repository")
-@click.option("--datahub", default=None, help="URI of the DataHub ingestion endpoint")
 @click.option(
     "--docker-build",
     is_flag=True,
@@ -139,8 +170,8 @@ def compile_project(
 )
 def compile_project_command(
     env: str,
-    repository: Optional[str],
-    datahub: Optional[str],
+    docker_repository_uri: Optional[str],
+    datahub_gms_uri: Optional[str],
     docker_build: bool,
 ) -> None:
-    compile_project(repository, datahub, docker_build, env)
+    compile_project(env, docker_repository_uri, datahub_gms_uri, docker_build)
diff --git a/data_pipelines_cli/cli_commands/create.py b/data_pipelines_cli/cli_commands/create.py
index 4569643..8874b15 100644
--- a/data_pipelines_cli/cli_commands/create.py
+++ b/data_pipelines_cli/cli_commands/create.py
@@ -1,22 +1,24 @@
-import sys
 from typing import Dict, Optional, Sequence
 
 import click
 import copier
 import questionary
 
-from data_pipelines_cli.cli_utils import echo_error, echo_warning
-from data_pipelines_cli.data_structures import TemplateConfig, read_config_or_exit
+from data_pipelines_cli.cli_utils import echo_warning
+from data_pipelines_cli.data_structures import TemplateConfig, read_config
+from data_pipelines_cli.errors import DataPipelinesError
 
 
 def _choose_template(config_templates: Dict[str, TemplateConfig]) -> TemplateConfig:
+    """
+    :raises DataPipelinesError: no template found in *config_templates*
+    """
     if len(config_templates) == 0:
-        echo_error(
+        raise DataPipelinesError(
             "No template provided. Either run 'dp create <project_path> "
             "<link_to_template>' to use template from the link, or add template "
             "to `~/.dp.yml` file",
         )
-        sys.exit(1)
 
     template_name = questionary.select("", choices=list(config_templates.keys())).ask()
     template_config = config_templates[template_name]
@@ -27,6 +29,7 @@ def _choose_template(config_templates: Dict[str, TemplateConfig]) -> TemplateCon
 def _get_template_path(
     config_templates: Dict[str, TemplateConfig], template_path: Optional[str]
 ) -> str:
+    """:raises DataPipelinesError: no template found in *config_templates*"""
     if template_path:
         if template_path in config_templates.keys():
             to_return = config_templates[template_path]["template_path"]
@@ -45,8 +48,9 @@ def create(project_path: str, template_path: Optional[str]) -> None:
     :type project_path: str
     :param template_path: Path or URI to the repository of the project template
     :type template_path: Optional[str]
+    :raises DataPipelinesError: no template found in `.dp.yml` config file
     """
-    config = read_config_or_exit()
+    config = read_config()
     config_templates = config["templates"]
     src_template_path = _get_template_path(config_templates, template_path)
     copier.copy(src_path=src_template_path, dst_path=project_path)
diff --git a/data_pipelines_cli/cli_commands/deploy.py b/data_pipelines_cli/cli_commands/deploy.py
index b8605dd..303e22f 100644
--- a/data_pipelines_cli/cli_commands/deploy.py
+++ b/data_pipelines_cli/cli_commands/deploy.py
@@ -2,15 +2,19 @@
 import json
 import os
 import pathlib
-import sys
-from typing import Dict, Optional, cast
+from typing import Any, Dict, Optional, cast
 
 import click
 import yaml
 
 from ..cli_constants import BUILD_DIR
-from ..cli_utils import echo_error, echo_info, subprocess_run
+from ..cli_utils import echo_info, subprocess_run
 from ..data_structures import DockerArgs
+from ..errors import (
+    DataPipelinesError,
+    DependencyNotInstalledError,
+    DockerNotInstalledError,
+)
 from ..filesystem_utils import LocalRemoteSync
 
 
@@ -24,27 +28,30 @@ class DeployCommand:
     """Whether to ingest DataHub metadata"""
     blob_address_path: str
     """URI of the cloud storage to send build artifacts to"""
-    provider_kwargs_dict: Dict[str, str]
+    provider_kwargs_dict: Dict[str, Any]
     """Dictionary of arguments required by a specific cloud storage provider,
     e.g. path to a token, username, password, etc."""
 
     def __init__(
         self,
-        repository: Optional[str],
+        docker_push: Optional[str],
         blob_address: str,
-        provider_kwargs_dict: Dict[str, str],
-        docker_push: bool,
+        provider_kwargs_dict: Optional[Dict[str, Any]],
         datahub_ingest: bool,
-    ):
-        self.docker_args = DockerArgs(repository) if docker_push else None
+    ) -> None:
+        self.docker_args = DockerArgs(docker_push) if docker_push else None
         self.datahub_ingest = datahub_ingest
         self.blob_address_path = os.path.join(
             blob_address, "dags", DeployCommand._get_project_name()
         )
-        self.provider_kwargs_dict = provider_kwargs_dict
+        self.provider_kwargs_dict = provider_kwargs_dict or {}
 
     def deploy(self) -> None:
-        """Push and deploy the project to the remote machine"""
+        """Push and deploy the project to the remote machine
+
+        :raises DependencyNotInstalledError: DataHub or Docker not installed
+        :raises DataPipelinesError: Error while pushing Docker image
+        """
         if self.docker_args:
             self._docker_push()
 
@@ -55,18 +62,19 @@ def deploy(self) -> None:
 
     @staticmethod
     def _get_project_name() -> str:
-        with open(pathlib.Path().joinpath("dbt_project.yml")) as f:
+        with open(pathlib.Path().cwd().joinpath("dbt_project.yml")) as f:
             dbt_project_config = yaml.safe_load(f)
             return dbt_project_config["name"]
 
     def _docker_push(self) -> None:
+        """
+        :raises DockerNotInstalledError: Docker not installed
+        :raises DataPipelinesError: Error while pushing Docker image
+        """
         try:
             import docker
         except ModuleNotFoundError:
-            echo_error(
-                "'docker' not installed. Run 'pip install data-pipelines-cli[docker]'"
-            )
-            sys.exit(1)
+            raise DockerNotInstalledError()
 
         echo_info("Pushing Docker image")
         docker_client = docker.from_env()
@@ -75,18 +83,24 @@ def _docker_push(self) -> None:
             repository=docker_args.repository,
             tag=docker_args.commit_sha,
             stream=True,
+            decode=True,
         ):
-            click.echo(line, nl=False)
+            click.echo(line)
+
+            if "error" in line:
+                raise DataPipelinesError(
+                    "Error raised when pushing Docker image. Ensure that "
+                    "Docker image you try to push exists. Maybe try running "
+                    "'dp compile' first?"
+                )
 
     @staticmethod
     def _datahub_ingest() -> None:
+        """:raises DependencyNotInstalledError: DataHub not installed"""
         try:
             import datahub  # noqa: F401
         except ModuleNotFoundError:
-            echo_error(
-                "'datahub' not installed. Run 'pip install data-pipelines-cli[datahub]'"
-            )
-            sys.exit(1)
+            raise DependencyNotInstalledError("datahub")
 
         echo_info("Ingesting datahub metadata")
         subprocess_run(
@@ -105,22 +119,20 @@ def _sync_bucket(self) -> None:
         ).sync(delete=True)
 
 
-@click.command(name="deploy", help="Push and deploy the project to the remote machine")
+@click.command(
+    name="deploy",
+    help="Push and deploy the project to the remote machine",
+    no_args_is_help=True,
+)
 @click.argument("address")
 @click.option(
     "--blob-args",
-    required=True,
+    required=False,
     type=click.File("r"),
     help="Path to JSON or YAML file with arguments that should be passed to "
     "your Bucket/blob provider",
 )
-@click.option("--repository", default=None, help="Path to the Docker repository")
-@click.option(
-    "--docker-push",
-    is_flag=True,
-    default=False,
-    help="Whether to push a Docker image to the repository",
-)
+@click.option("--docker-push", default=None, help="Path to the Docker repository")
 @click.option(
     "--datahub-ingest",
     is_flag=True,
@@ -129,20 +141,22 @@ def _sync_bucket(self) -> None:
 )
 def deploy_command(
     address: str,
-    blob_args: io.TextIOWrapper,
-    repository: Optional[str],
-    docker_push: bool,
+    blob_args: Optional[io.TextIOWrapper],
+    docker_push: Optional[str],
     datahub_ingest: bool,
 ) -> None:
-    try:
-        provider_kwargs_dict = json.load(blob_args)
-    except json.JSONDecodeError:
-        provider_kwargs_dict = yaml.safe_load(blob_args)
+    if blob_args:
+        try:
+            provider_kwargs_dict = json.load(blob_args)
+        except json.JSONDecodeError:
+            blob_args.seek(0)
+            provider_kwargs_dict = yaml.safe_load(blob_args)
+    else:
+        provider_kwargs_dict = None
 
     DeployCommand(
-        repository,
+        docker_push,
         address,
         provider_kwargs_dict,
-        docker_push,
         datahub_ingest,
     ).deploy()
diff --git a/data_pipelines_cli/cli_commands/init.py b/data_pipelines_cli/cli_commands/init.py
index 6df88f5..740a968 100644
--- a/data_pipelines_cli/cli_commands/init.py
+++ b/data_pipelines_cli/cli_commands/init.py
@@ -1,5 +1,4 @@
 import pathlib
-import sys
 import tempfile
 from typing import Optional, Sequence
 
@@ -10,6 +9,7 @@
 
 from ..cli_constants import CONFIGURATION_PATH, DEFAULT_GLOBAL_CONFIG
 from ..data_structures import DataPipelinesConfig
+from ..errors import DataPipelinesError
 
 
 def _download_global_config(config_path: str) -> DataPipelinesConfig:
@@ -26,6 +26,7 @@ def init(config_path: Optional[str]) -> None:
 
     :param config_path: URI of the repository with a template of the config file
     :type config_path: Optional[str]
+    :raises DataPipelinesError: user do not want to overwrite existing config file
     """
     if CONFIGURATION_PATH.is_file():
         overwrite_confirm = questionary.confirm(
@@ -33,7 +34,7 @@ def init(config_path: Optional[str]) -> None:
             default=False,
         ).ask()
         if not overwrite_confirm:
-            sys.exit(1)
+            raise DataPipelinesError("Could not overwrite existing config")
 
     if config_path:
         config = _download_global_config(config_path)
diff --git a/data_pipelines_cli/cli_commands/prepare_env.py b/data_pipelines_cli/cli_commands/prepare_env.py
new file mode 100644
index 0000000..3433e53
--- /dev/null
+++ b/data_pipelines_cli/cli_commands/prepare_env.py
@@ -0,0 +1,96 @@
+import os
+import pathlib
+from typing import Any, Dict
+
+import click
+import yaml
+from jinja2 import BaseLoader, DictLoader
+from jinja2.nativetypes import NativeEnvironment
+
+from data_pipelines_cli.cli_utils import echo_subinfo
+from data_pipelines_cli.config_generation import DbtProfile, generate_profiles_dict
+from data_pipelines_cli.dbt_utils import read_dbt_vars_from_configs
+from data_pipelines_cli.errors import JinjaVarKeyError
+
+
+def _prepare_jinja_replace_environment(
+    jinja_loader: BaseLoader, dbt_vars: Dict[str, Any]
+) -> NativeEnvironment:
+    def _jinja_vars(var_name: str) -> Any:
+        return dbt_vars[var_name]
+
+    def _jinja_env_vars(var_name: str) -> Any:
+        return os.environ[var_name]
+
+    jinja_env = NativeEnvironment(loader=jinja_loader)
+    # Hacking Jinja to use our functions, following:
+    # https://stackoverflow.com/a/6038550
+    jinja_env.globals["var"] = _jinja_vars
+    jinja_env.globals["env_var"] = _jinja_env_vars
+
+    return jinja_env
+
+
+def _replace_outputs_vars_with_values(
+    outputs_setting_dict: Dict[str, Any], dbt_vars: Dict[str, Any]
+) -> Dict[str, Any]:
+    jinja_loader = DictLoader(outputs_setting_dict)
+    jinja_env = _prepare_jinja_replace_environment(jinja_loader, dbt_vars)
+
+    rendered_settings = {}
+    for setting_key, setting_old_value in outputs_setting_dict.items():
+        try:
+            rendered_settings[setting_key] = jinja_env.get_template(
+                setting_key
+            ).render()
+        except TypeError:
+            # Jinja is accepting only str or Template and fails on int, etc.
+            rendered_settings[setting_key] = setting_old_value
+        except KeyError as key_error:
+            # Variable does not exist and _jinja_vars or _jinja_env_vars thrown
+            raise JinjaVarKeyError(key_error.args[0])
+    return rendered_settings
+
+
+def _replace_profiles_vars_with_values(
+    profile: Dict[str, DbtProfile], dbt_vars: Dict[str, Any]
+) -> Dict[str, DbtProfile]:
+    for targets_dict in profile.values():
+        new_outputs = {}
+        for target, settings_dict in targets_dict["outputs"].items():
+            new_outputs[target] = _replace_outputs_vars_with_values(
+                settings_dict, dbt_vars
+            )
+        targets_dict["outputs"] = new_outputs
+
+    return profile
+
+
+def prepare_env(env: str) -> None:
+    """
+    Prepares local environment for use with applications expecting a "traditional"
+    dbt structure, such as plugins to VS Code. If in doubt, use ``dp run`` and
+    ``dp test`` instead.
+
+    :param env: Name of the environment
+    :type env: str
+    """
+    profile = _replace_profiles_vars_with_values(
+        generate_profiles_dict(env, True), read_dbt_vars_from_configs(env)
+    )
+
+    home_profiles_path = pathlib.Path.home().joinpath(".dbt", "profiles.yml")
+    home_profiles_path.parent.mkdir(parents=True, exist_ok=True)
+    with open(home_profiles_path, "w") as profiles:
+        yaml.dump(profile, profiles, default_flow_style=False)
+
+    echo_subinfo(f"Saved profiles.yml in {home_profiles_path}")
+
+
+@click.command(
+    name="prepare-env",
+    help="Prepare local environment for apps interfacing with dbt",
+)
+@click.option("--env", default="local", type=str, help="Name of the environment")
+def prepare_env_command(env: str) -> None:
+    prepare_env(env)
diff --git a/data_pipelines_cli/cli_commands/run.py b/data_pipelines_cli/cli_commands/run.py
index c062107..9764c35 100644
--- a/data_pipelines_cli/cli_commands/run.py
+++ b/data_pipelines_cli/cli_commands/run.py
@@ -1,6 +1,8 @@
 import click
 
+from ..config_generation import get_profiles_yml_build_path
 from ..dbt_utils import run_dbt_command
+from .compile import compile_project
 
 
 def run(env: str) -> None:
@@ -10,10 +12,18 @@ def run(env: str) -> None:
     :param env: Name of the environment
     :type env: str
     """
-    run_dbt_command(("run",), env, None)
+    compile_project(env)
+    profiles_path = get_profiles_yml_build_path(env)
+    run_dbt_command(("run",), env, profiles_path)
 
 
 @click.command(name="run", help="Run the project on the local machine")
-@click.option("--env", default="local", type=str, help="Name of the environment")
+@click.option(
+    "--env",
+    default="local",
+    type=str,
+    show_default=True,
+    help="Name of the environment",
+)
 def run_command(env: str) -> None:
     run(env)
diff --git a/data_pipelines_cli/cli_commands/template.py b/data_pipelines_cli/cli_commands/template.py
index 7035f55..beafe09 100644
--- a/data_pipelines_cli/cli_commands/template.py
+++ b/data_pipelines_cli/cli_commands/template.py
@@ -1,12 +1,12 @@
 import click
 import yaml
 
-from data_pipelines_cli.data_structures import read_config_or_exit
+from data_pipelines_cli.data_structures import read_config
 
 
 def list_templates() -> None:
     """Print a list of all templates saved in the config file"""
-    config = read_config_or_exit()
+    config = read_config()
 
     click.echo("AVAILABLE TEMPLATES:\n")
     for tc in config["templates"].values():
diff --git a/data_pipelines_cli/cli_utils.py b/data_pipelines_cli/cli_utils.py
index 73568bc..5b79cb8 100644
--- a/data_pipelines_cli/cli_utils.py
+++ b/data_pipelines_cli/cli_utils.py
@@ -7,6 +7,12 @@
 
 import click
 
+from data_pipelines_cli.errors import (
+    DataPipelinesError,
+    SubprocessNonZeroExitError,
+    SubprocessNotFound,
+)
+
 
 def echo_error(text: str, **kwargs: Any) -> None:
     """
@@ -56,10 +62,9 @@ def get_argument_or_environment_variable(
     argument: Optional[str], argument_name: str, environment_variable_name: str
 ) -> str:
     """
-    Given *argument* is not `None`, returns its value. Otherwise, searches
+    Given *argument* is not ``None``, returns its value. Otherwise, searches
     for *environment_variable_name* amongst environment variables and returns
-    it. If such a variable is not set, exits program with system exit status
-    set to 1.
+    it. If such a variable is not set, raises :exc:`.DataPipelinesError`.
 
     :param argument: Optional value passed to the CLI as the *argument_name*
     :type argument: Optional[str]
@@ -68,30 +73,33 @@ def get_argument_or_environment_variable(
     :param environment_variable_name: Name of the environment variable to search for
     :type environment_variable_name: str
     :return: Value of the *argument* or specified environment variable
+    :raises DataPipelinesError: *argument* is ``None`` and \
+        *environment_variable_name* is not set
     """
     result = argument or os.environ.get(environment_variable_name)
     if not result:
-        echo_error(
+        raise DataPipelinesError(
             f"Could not get {environment_variable_name}. Either set it as an "
             f"environment variable {environment_variable_name} or pass as a "
-            f"`--{argument_name}` CLI argument"
+            f"`--{argument_name}` CLI argument."
         )
-        sys.exit(1)
     return result
 
 
 def subprocess_run(args: List[str]) -> subprocess.CompletedProcess[bytes]:
     """
     Runs subprocess and returns its state if completed with a success. If not,
-    exits the program with system exit status set to 1.
+    raises :exc:`.SubprocessNonZeroExitError`.
 
     :param args: List of strings representing subprocess and its arguments
     :type args: List[str]
     :return: State of the completed process
     :rtype: subprocess.CompletedProcess[bytes]
+    :raises SubprocessNonZeroExitError: subprocess exited with non-zero exit code
     """
     try:
         return subprocess.run(args, check=True)
+    except FileNotFoundError:
+        raise SubprocessNotFound(args[0])
     except subprocess.CalledProcessError as err:
-        echo_error(f"{args[0]} has exited with non-zero exit code: {err.returncode}")
-        sys.exit(1)
+        raise SubprocessNonZeroExitError(args[0], err.returncode)
diff --git a/data_pipelines_cli/config_generation.py b/data_pipelines_cli/config_generation.py
index eeec3ea..2372be8 100644
--- a/data_pipelines_cli/config_generation.py
+++ b/data_pipelines_cli/config_generation.py
@@ -94,7 +94,22 @@ class DbtProfile(TypedDict):
     """Dictionary of a warehouse data and credentials, referenced by `target` name"""
 
 
-def _generate_profile_dict(env: str) -> Dict[str, DbtProfile]:
+def generate_profiles_dict(env: str, copy_config_dir: bool) -> Dict[str, DbtProfile]:
+    """
+    Generates and saves ``profiles.yml`` file at ``build/profiles/local`` or
+    ``build/profiles/env_execution``, depending on `env` argument.
+
+    :param env: Name of the environment
+    :type env: str
+    :param copy_config_dir: Whether to copy ``config`` directory to ``build`` \
+        working directory
+    :type copy_config_dir: bool
+    :return: Dictionary representing data to be saved in ``profiles.yml``
+    :rtype: Dict[str, DbtProfile]
+    """
+    if copy_config_dir:
+        copy_config_dir_to_build_dir()
+
     dbt_env_config = read_dictionary_from_config_directory(
         BUILD_DIR.joinpath("dag"), env, "dbt.yml"
     )
@@ -121,26 +136,40 @@ def _generate_profile_dict(env: str) -> Dict[str, DbtProfile]:
     }
 
 
+def get_profiles_yml_build_path(env: str) -> pathlib.Path:
+    """
+    Returns path to ``build/profiles/<profile_name>/profiles.yml``,
+    depending on `env` argument.
+
+    :param env: Name of the environment
+    :type env: str
+    :return:
+    :rtype: pathlib.Path
+    """
+    profile_name = get_dbt_profiles_env_name(env)
+    return BUILD_DIR.joinpath("profiles", profile_name, "profiles.yml")
+
+
 def generate_profiles_yml(env: str, copy_config_dir: bool = True) -> pathlib.Path:
     """
     Generates and saves ``profiles.yml`` file at ``build/profiles/local`` or
     ``build/profiles/env_execution``, depending on `env` argument.
 
-    :param env: str
-    :param copy_config_dir: bool
+    :param env: Name of the environment
+    :type env: str
+    :param copy_config_dir: Whether to copy ``config`` directory to ``build`` \
+        working directory
+    :type copy_config_dir: bool
     :return: Path to ``build/profiles/{env}``
+    :rtype: pathlib.Path
     """
-    if copy_config_dir:
-        copy_config_dir_to_build_dir()
     echo_info("Generating profiles.yml")
-    profile = _generate_profile_dict(env)
+    profile = generate_profiles_dict(env, copy_config_dir)
 
-    profile_name = get_dbt_profiles_env_name(env)
-    profiles_path = BUILD_DIR.joinpath("profiles", profile_name, "profiles.yml")
+    profiles_path = get_profiles_yml_build_path(env)
     profiles_path.parent.mkdir(parents=True, exist_ok=True)
     with open(profiles_path, "w") as profiles:
         yaml.dump(profile, profiles, default_flow_style=False)
-
     echo_subinfo(f"Generated profiles.yml in {profiles_path}")
 
     return profiles_path.parent
diff --git a/data_pipelines_cli/data_structures.py b/data_pipelines_cli/data_structures.py
index 0bd3859..dd1a240 100644
--- a/data_pipelines_cli/data_structures.py
+++ b/data_pipelines_cli/data_structures.py
@@ -4,10 +4,10 @@
 import yaml
 
 from data_pipelines_cli.cli_utils import (
-    echo_error,
     echo_warning,
     get_argument_or_environment_variable,
 )
+from data_pipelines_cli.errors import DataPipelinesError, NoConfigFileError
 from data_pipelines_cli.io_utils import git_revision_hash
 
 if sys.version_info >= (3, 8):
@@ -35,12 +35,14 @@ class DataPipelinesConfig(TypedDict):
     """Variables to be passed to dbt as `--vars` argument"""
 
 
-def read_config() -> Optional[DataPipelinesConfig]:
+def read_config() -> DataPipelinesConfig:
     """
-    Parses `.dp.yml` config file, if it exists
+    Parses `.dp.yml` config file, if it exists. Otherwise, raises
+    :exc:`.NoConfigFileError`
 
     :return: POD representing `.dp.yml` config file, if it exists
-    :rtype: Optional[DataPipelinesConfig]
+    :rtype: DataPipelinesConfig
+    :raises NoConfigFileError: `.dp.yml` file not found
     """
     # Avoiding a dependency loop between `cli_constants` and `data_structures`
     from data_pipelines_cli.cli_constants import CONFIGURATION_PATH
@@ -49,43 +51,30 @@ def read_config() -> Optional[DataPipelinesConfig]:
         echo_warning(
             "No configuration file found. Run 'dp init' to create it.",
         )
-        return None
+        raise NoConfigFileError()
 
     with open(CONFIGURATION_PATH, "r") as f:
         return yaml.safe_load(f)
 
 
-def read_config_or_exit() -> DataPipelinesConfig:
-    """
-    Parses `.dp.yml` config file, if it exists. Otherwise, exits program with
-    a system exit status set to 1.
+class DockerArgs:
+    """Arguments required by the Docker to make a push to the repository
 
-    :return: POD representing `.dp.yml` config file
-    :rtype: DataPipelinesConfig
+    :raises DataPipelinesError: *repository* variable not set or git hash not found
     """
 
-    config = read_config()
-    if not config:
-        sys.exit(1)
-    return config
-
-
-class DockerArgs:
-    """Arguments required by the Docker to make a push to the repository"""
-
     repository: str
     """URI of the Docker images repository"""
     commit_sha: str
     """Long hash of the current Git revision. Used as an image tag"""
 
-    def __init__(self, repository: Optional[str]):
+    def __init__(self, docker_repository_uri: Optional[str]) -> None:
         self.repository = get_argument_or_environment_variable(
-            repository, "repository", "REPOSITORY_URL"
+            docker_repository_uri, "repository", "REPOSITORY_URL"
         )
         commit_sha = git_revision_hash()
         if not commit_sha:
-            echo_error("Could not get git revision hash.")
-            sys.exit(1)
+            raise DataPipelinesError("Could not get git revision hash.")
         self.commit_sha = commit_sha
 
     def docker_build_tag(self) -> str:
diff --git a/data_pipelines_cli/dbt_utils.py b/data_pipelines_cli/dbt_utils.py
index ab40c63..23132c4 100644
--- a/data_pipelines_cli/dbt_utils.py
+++ b/data_pipelines_cli/dbt_utils.py
@@ -1,29 +1,48 @@
 import pathlib
 import sys
-from typing import Any, Dict, Optional, Tuple
+from typing import Any, Dict, Tuple
 
 import yaml
 
 from .cli_constants import BUILD_DIR, get_dbt_profiles_env_name
 from .cli_utils import echo_subinfo, subprocess_run
-from .config_generation import (
-    generate_profiles_yml,
-    read_dictionary_from_config_directory,
-)
+from .config_generation import read_dictionary_from_config_directory
 from .data_structures import DataPipelinesConfig, read_config
+from .errors import NoConfigFileError
 
 
-def _read_dbt_vars_from_configs(dbt_env_config: Dict[str, Any]) -> str:
-    dp_config = read_config() or DataPipelinesConfig(templates={}, vars={})
+def read_dbt_vars_from_configs(env: str) -> Dict[str, Any]:
+    """
+    Reads `vars` field from dp configuration file (``$HOME/.dp.yml``), base
+    ``dbt.yml`` config (``config/base/dbt.yml``) and environment-specific config
+    (``config/{env}/dbt.yml``) and compiles into one dictionary.
+
+    :param env: Name of the environment
+    :type env: str
+    :return: Dictionary with `vars` and their keys
+    :rtype: Dict[str, Any]
+    """
+    dbt_env_config = read_dictionary_from_config_directory(
+        BUILD_DIR.joinpath("dag"), env, "dbt.yml"
+    )
+
+    try:
+        dp_config = read_config()
+    except NoConfigFileError:
+        dp_config = DataPipelinesConfig(templates={}, vars={})
     dp_vars = dp_config.get("vars", {})
     dbt_vars: Dict[str, str] = dbt_env_config.get("vars", {})
-    return yaml.dump(
-        dict(dp_vars, **dbt_vars), default_flow_style=True, width=sys.maxsize
-    )
+
+    return dict(dp_vars, **dbt_vars)
+
+
+def _dump_dbt_vars_from_configs_to_string(env: str) -> str:
+    dbt_vars = read_dbt_vars_from_configs(env)
+    return yaml.dump(dbt_vars, default_flow_style=True, width=sys.maxsize)
 
 
 def run_dbt_command(
-    command: Tuple[str, ...], env: str, profiles_path: Optional[pathlib.Path]
+    command: Tuple[str, ...], env: str, profiles_path: pathlib.Path
 ) -> None:
     """
     Runs dbt subprocess in a context of specified *env*
@@ -33,16 +52,17 @@ def run_dbt_command(
     :param env: Name of the environment
     :type env: str
     :param profiles_path: Path to the directory containing `profiles.yml` file
-    :type profiles_path: Optional[pathlib.Path]
+    :type profiles_path: pathlib.Path
+    :raises SubprocessNotFound: dbt not installed
+    :raises SubprocessNonZeroExitError: dbt exited with error
     """
     command_str = " ".join(list(command))
     echo_subinfo(f"dbt {command_str}")
 
-    profiles_path = profiles_path or generate_profiles_yml(env)
     dbt_env_config = read_dictionary_from_config_directory(
         BUILD_DIR.joinpath("dag"), env, "dbt.yml"
     )
-    dbt_vars = _read_dbt_vars_from_configs(dbt_env_config)
+    dbt_vars = _dump_dbt_vars_from_configs_to_string(env)
     subprocess_run(
         [
             "dbt",
diff --git a/data_pipelines_cli/errors.py b/data_pipelines_cli/errors.py
new file mode 100644
index 0000000..762dfd9
--- /dev/null
+++ b/data_pipelines_cli/errors.py
@@ -0,0 +1,62 @@
+class DataPipelinesError(Exception):
+    """Base class for all exceptions in data_pipelines_cli module"""
+
+    message: str
+    """explanation of the error"""
+
+    def __init__(self, message: str) -> None:
+        self.message = message
+
+
+class DependencyNotInstalledError(DataPipelinesError):
+    """Exception raised if certain dependency is not installed"""
+
+    def __init__(self, program_name: str) -> None:
+        self.message = (
+            f"'{program_name}' not installed. Run 'pip install "
+            f"data-pipelines-cli[{program_name}]'"
+        )
+
+
+class NoConfigFileError(DataPipelinesError):
+    """Exception raised if `.dp.yml` does not exist"""
+
+    def __init__(self) -> None:
+        self.message = (
+            "`.dp.yml` config file does not exists. Run 'dp init' to create it."
+        )
+
+
+class SubprocessNonZeroExitError(DataPipelinesError):
+    """Exception raised if subprocess exits with non-zero exit code"""
+
+    def __init__(self, subprocess_name: str, exit_code: int) -> None:
+        self.message = (
+            f"{subprocess_name} has exited with non-zero exit code: {exit_code}"
+        )
+
+
+class SubprocessNotFound(DataPipelinesError):
+    """Exception raised if subprocess cannot be found"""
+
+    def __init__(self, subprocess_name: str) -> None:
+        self.message = (
+            f"{subprocess_name} cannot be found. "
+            "Ensure it is installed and listed in your $PATH."
+        )
+
+
+class DockerNotInstalledError(DependencyNotInstalledError):
+    """Exception raised if 'docker' is not installed"""
+
+    def __init__(self) -> None:
+        super().__init__("docker")
+
+
+class JinjaVarKeyError(DataPipelinesError):
+    def __init__(self, key: str) -> None:
+        self.message = (
+            f"Variable {key} cannot be found neither in 'dbt.yml' and "
+            "'$HOME/.dp.yml' vars nor in environment variables, causing Jinja "
+            "template rendering to fail."
+        )
diff --git a/data_pipelines_cli/filesystem_utils.py b/data_pipelines_cli/filesystem_utils.py
index df646e5..3176ba2 100644
--- a/data_pipelines_cli/filesystem_utils.py
+++ b/data_pipelines_cli/filesystem_utils.py
@@ -1,12 +1,14 @@
 from __future__ import annotations
 
 import os
+import pathlib
 from typing import Dict, Set, Union
 
 import fsspec
 from fsspec import AbstractFileSystem
 
 from .cli_utils import echo_subinfo
+from .errors import DataPipelinesError
 
 
 class LocalRemoteSync:
@@ -25,7 +27,12 @@ def __init__(
         local_path: Union[str, os.PathLike[str]],
         remote_path: str,
         remote_kwargs: Dict[str, str],
-    ):
+    ) -> None:
+        if not pathlib.Path(local_path).exists():
+            raise DataPipelinesError(
+                f"{local_path} does not exists. Run 'dp compile' before."
+            )
+
         self.local_path_str = str(local_path).rstrip("/")
         self.local_fs = fsspec.filesystem("file")
         self.remote_fs, self.remote_path_str = fsspec.core.url_to_fs(
diff --git a/docs/cli.rst b/docs/cli.rst
index 57a5731..61561d7 100644
--- a/docs/cli.rst
+++ b/docs/cli.rst
@@ -4,6 +4,6 @@ CLI Commands Reference
 If you are looking for extensive information on a specific CLI command,
 this part of the documentation is for you.
 
-.. click:: data_pipelines_cli.cli:cli
+.. click:: data_pipelines_cli.cli:_cli
    :prog: dp
    :nested: full
diff --git a/docs/index.rst b/docs/index.rst
index 0376e2d..93447c9 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -1,6 +1,27 @@
 ``data-pipelines-cli``: CLI for data platform
 ==============================================
 
+.. image:: https://img.shields.io/badge/python-3.7%20%7C%203.8%20%7C%203.9-blue.svg
+   :target: https://github.com/getindata/data-pipelines-cli
+   :alt: Python Version
+
+.. image:: https://badge.fury.io/py/data-pipelines-cli.svg
+   :target: https://pypi.org/project/data-pipelines-cli/
+   :alt: PyPI Version
+
+.. image:: https://pepy.tech/badge/data-pipelines-cli
+   :target: https://pepy.tech/project/data-pipelines-cli
+   :alt: Downloads
+
+.. image:: https://api.codeclimate.com/v1/badges/e44ed9383a42b59984f6/maintainability
+   :target: https://codeclimate.com/github/getindata/data-pipelines-cli/maintainability
+   :alt: Maintainability
+
+.. image:: https://api.codeclimate.com/v1/badges/e44ed9383a42b59984f6/test_coverage
+   :target: https://codeclimate.com/github/getindata/data-pipelines-cli/test_coverage
+   :alt: Test Coverage
+
+
 Installation
 ------------
 Use the package manager `pip <https://pip.pypa.io/en/stable/>`_ to
diff --git a/docs/source/data_pipelines_cli.cli_commands.rst b/docs/source/data_pipelines_cli.cli_commands.rst
index 9381ece..bf27f0a 100644
--- a/docs/source/data_pipelines_cli.cli_commands.rst
+++ b/docs/source/data_pipelines_cli.cli_commands.rst
@@ -49,6 +49,14 @@ data\_pipelines\_cli.cli\_commands.init module
    :undoc-members:
    :show-inheritance:
 
+data\_pipelines\_cli.cli\_commands.prepare\_env module
+------------------------------------------------------
+
+.. automodule:: data_pipelines_cli.cli_commands.prepare_env
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 data\_pipelines\_cli.cli\_commands.run module
 ---------------------------------------------
 
@@ -72,3 +80,4 @@ data\_pipelines\_cli.cli\_commands.test module
    :members:
    :undoc-members:
    :show-inheritance:
+
diff --git a/docs/source/data_pipelines_cli.rst b/docs/source/data_pipelines_cli.rst
index 1d637e9..d2a4fdc 100644
--- a/docs/source/data_pipelines_cli.rst
+++ b/docs/source/data_pipelines_cli.rst
@@ -65,6 +65,14 @@ data\_pipelines\_cli.dbt\_utils module
    :undoc-members:
    :show-inheritance:
 
+data\_pipelines\_cli.errors module
+----------------------------------
+
+.. automodule:: data_pipelines_cli.errors
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 data\_pipelines\_cli.filesystem\_utils module
 ---------------------------------------------
 
@@ -80,3 +88,4 @@ data\_pipelines\_cli.io\_utils module
    :members:
    :undoc-members:
    :show-inheritance:
+
diff --git a/docs/usage.rst b/docs/usage.rst
index 61a510e..31bd94c 100644
--- a/docs/usage.rst
+++ b/docs/usage.rst
@@ -147,14 +147,34 @@ Project deployment
 ------------------
 
 ``dp deploy`` will sync with your bucket provider. The provider will be chosen automatically based on the remote URL.
-``dp deploy`` also requires the user to point to JSON or YAML file with provider-specific data like access tokens or project
-names. E.g., to connect with Google Cloud Storage, one should run:
+Usually, it is worth pointing ``dp deploy`` to a JSON or YAML file with provider-specific data like access tokens or project
+names. The *provider-specific data* should be interpreted as the ``**kwargs`` (keyword arguments) expected by a specific
+`fsspec <https://filesystem-spec.readthedocs.io/en/latest/>`_'s FileSystem implementation. One would most likely want to
+look at the `S3FileSystem <https://s3fs.readthedocs.io/en/latest/api.html#s3fs.core.S3FileSystem>`_ or
+`GCSFileSystem <https://gcsfs.readthedocs.io/en/latest/api.html#gcsfs.core.GCSFileSystem>`_ documentation.
+
+E.g., to connect with Google Cloud Storage, one should run:
 
 .. code-block:: bash
 
  echo '{"token": "<PATH_TO_YOUR_TOKEN>", "project_name": "<YOUR_PROJECT_NAME>"}' > gs_args.json
  dp deploy "gs://<YOUR_GS_PATH>" --blob-args gs_args.json
 
+However, in some cases you do not need to do so, e.g. when using **gcloud** with properly set local credentials. In such
+case, you can try to run just the ``dp deploy "gs://<YOUR_GS_PATH>"`` command and let ``gcsfs`` search for the credentials.
+Please refer to the documentation of the specific ``fsspec``'s implementation for more information about the required
+keyword arguments.
+
+Preparing dbt environment
+-------------------------
+
+Sometimes you would like to use standalone **dbt** or an application that interfaces with it (like VS Code plugin).
+``dp prepare-env`` prepares your local environment to be more conformant with a standalone **dbt** requirements, e.g.
+by saving ``profiles.yml`` in the home directory.
+
+However, be aware that most of the time you do not need to do so, and you can comfortably use ``dp run`` and ``dp test``
+commands to interface with the **dbt** instead.
+
 Clean project
 -------------
 
diff --git a/setup.cfg b/setup.cfg
index 8ee625b..f4400f1 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.6.0
+current_version = 0.7.0
 
 [bumpversion:file:setup.py]
 
diff --git a/setup.py b/setup.py
index 2289fa2..6a83ad1 100644
--- a/setup.py
+++ b/setup.py
@@ -12,11 +12,12 @@
     "types-PyYAML>=6.0",
     "copier==5.1.0",
     "dbt>=0.21, <0.22",
+    "Jinja2>=2.11,<2.12",
     "fsspec",
 ]
 
 EXTRA_FILESYSTEMS_REQUIRE = {
-    "gcp": ["gcsfs"],
+    "gcs": ["gcsfs"],
     "s3": ["s3fs"],
 }
 
@@ -49,7 +50,7 @@
 
 setup(
     name="data_pipelines_cli",
-    version="0.6.0",
+    version="0.7.0",
     description="CLI for data platform",
     long_description=README,
     long_description_content_type="text/markdown",
diff --git a/tox.ini b/tox.ini
index c41178c..8c2f186 100644
--- a/tox.ini
+++ b/tox.ini
@@ -6,7 +6,7 @@ envlist = py37,py39
 extras =
     tests
 commands=
-    python -m pytest -s --cov data_pipelines_cli --cov-report xml --cov-report term-missing --ignore=venv
+    python -m pytest --cov data_pipelines_cli --cov-report xml --cov-report term-missing --ignore=venv
 
 # Lint
 [flake8]


Test patch:
diff --git a/data_pipelines_cli/cli_commands/test.py b/data_pipelines_cli/cli_commands/test.py
index cdd6260..780ec34 100644
--- a/data_pipelines_cli/cli_commands/test.py
+++ b/data_pipelines_cli/cli_commands/test.py
@@ -1,6 +1,8 @@
 import click
 
+from ..config_generation import get_profiles_yml_build_path
 from ..dbt_utils import run_dbt_command
+from .compile import compile_project
 
 
 def test(env: str) -> None:
@@ -10,10 +12,18 @@ def test(env: str) -> None:
     :param env: Name of the environment
     :type env: str
     """
-    run_dbt_command(("test",), env, None)
+    compile_project(env)
+    profiles_path = get_profiles_yml_build_path(env)
+    run_dbt_command(("test",), env, profiles_path)
 
 
 @click.command(name="test", help="Run tests of the project on the local machine")
-@click.option("--env", default="local", type=str, help="Name of the environment")
+@click.option(
+    "--env",
+    default="local",
+    type=str,
+    show_default=True,
+    help="Name of the environment",
+)
 def test_command(env: str) -> None:
     test(env)
diff --git a/tests/cli_commands/__init__.py b/tests/cli_commands/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/tests/cli_commands/test_clean.py b/tests/cli_commands/test_clean.py
new file mode 100644
index 0000000..c53174e
--- /dev/null
+++ b/tests/cli_commands/test_clean.py
@@ -0,0 +1,43 @@
+import pathlib
+import tempfile
+import unittest
+from typing import List
+from unittest.mock import patch
+
+from click.testing import CliRunner
+
+from data_pipelines_cli.cli import _cli
+
+
+class CleanCommandTestCase(unittest.TestCase):
+    def setUp(self) -> None:
+        self.subprocess_run_args = []
+
+    def _mock_run(self, args: List[str]):
+        self.subprocess_run_args = args
+
+    def test_clean(self):
+        with patch(
+            "data_pipelines_cli.cli_commands.clean.subprocess_run", self._mock_run
+        ):
+            runner = CliRunner()
+            result = runner.invoke(_cli, ["clean"])
+            self.assertEqual(0, result.exit_code, msg=result.exception)
+            self.assertListEqual(["dbt", "clean"], self.subprocess_run_args)
+
+    def test_clean_remove_dir(self):
+        with patch(
+            "data_pipelines_cli.cli_commands.clean.subprocess_run", self._mock_run
+        ), tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_commands.clean.BUILD_DIR", pathlib.Path(tmp_dir)
+        ):
+            assert pathlib.Path(tmp_dir).exists()
+            runner = CliRunner()
+            result = runner.invoke(_cli, ["clean"])
+            self.assertEqual(0, result.exit_code, msg=result.exception)
+            self.assertListEqual(["dbt", "clean"], self.subprocess_run_args)
+            assert not pathlib.Path(tmp_dir).exists()
+
+            # Fix for Python 3.7.
+            # Otherwise it throws, as there is no directory to remove.
+            pathlib.Path(tmp_dir).mkdir(parents=True, exist_ok=True)
diff --git a/tests/cli_commands/test_compile.py b/tests/cli_commands/test_compile.py
new file mode 100644
index 0000000..6a1efe9
--- /dev/null
+++ b/tests/cli_commands/test_compile.py
@@ -0,0 +1,207 @@
+import json
+import os
+import pathlib
+import tempfile
+import unittest
+from typing import List
+from unittest.mock import MagicMock, patch
+
+import yaml
+from click.testing import CliRunner
+
+from data_pipelines_cli.cli import _cli
+from data_pipelines_cli.errors import DockerNotInstalledError
+
+goldens_dir_path = pathlib.Path(__file__).parent.parent.joinpath("goldens")
+
+
+class CompileCommandTestCase(unittest.TestCase):
+    @staticmethod
+    def _datahub_content(ingest_endpoint):
+        return {"sink": {"config": {"server": f"https://{ingest_endpoint}:8080"}}}
+
+    @staticmethod
+    def _k8s_content(repository_url: str, tag: str):
+        return {
+            "image": {
+                "repository": repository_url,
+                "tag": tag,
+            },
+            "variable1": 1337,
+            "var2": "Hello, world!",
+            "envs": {
+                "SOME_BOOLEAN": True,
+            },
+        }
+
+    def setUp(self) -> None:
+        self.all_subprocess_run_args = []
+
+    def _mock_run(self, args: List[str]):
+        self.all_subprocess_run_args += args
+
+    @patch("pathlib.Path.cwd", lambda: goldens_dir_path)
+    def test_no_args(self):
+        runner = CliRunner()
+        with tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_commands.compile.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.config_generation.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.dbt_utils.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.dbt_utils.subprocess_run", self._mock_run
+        ):
+            result = runner.invoke(_cli, ["compile"])
+            self.assertEqual(0, result.exit_code, msg=result.exception)
+
+            # 4 = 2 from goldens/dag, manifest.json, 'config' directory
+            self.assertEqual(4, len(os.listdir(pathlib.Path(tmp_dir).joinpath("dag"))))
+
+            args_str = " ".join(self.all_subprocess_run_args)
+            self.assertIn("dbt deps", args_str)
+            self.assertIn("dbt compile", args_str)
+            self.assertIn("dbt docs generate", args_str)
+            self.assertIn("dbt source freshness", args_str)
+
+            tmp_dir_path = pathlib.Path(tmp_dir)
+            with open(
+                tmp_dir_path.joinpath("dag", "manifest.json"), "r"
+            ) as tmp_manifest, open(
+                goldens_dir_path.joinpath("target", "manifest.json"), "r"
+            ) as golden_manifest:
+                self.assertDictEqual(
+                    json.load(golden_manifest), json.load(tmp_manifest)
+                )
+            with open(
+                tmp_dir_path.joinpath("dag", "config", "base", "datahub.yml"), "r"
+            ) as tmp_datahub, open(
+                goldens_dir_path.joinpath("config", "base", "datahub.yml")
+            ) as golden_datahub:
+                self.assertDictEqual(
+                    yaml.safe_load(golden_datahub), yaml.safe_load(tmp_datahub)
+                )
+            with open(
+                tmp_dir_path.joinpath("dag", "config", "base", "k8s.yml"), "r"
+            ) as tmp_k8s, open(
+                goldens_dir_path.joinpath("config", "base", "k8s.yml")
+            ) as golden_k8s:
+                self.assertDictEqual(
+                    yaml.safe_load(golden_k8s), yaml.safe_load(tmp_k8s)
+                )
+
+    @patch("pathlib.Path.cwd", lambda: goldens_dir_path)
+    @patch("data_pipelines_cli.data_structures.git_revision_hash")
+    def test_docker_uri_no_build(self, mock_git_revision_hash):
+        commit_sha = "aaa9876aaa"
+        mock_git_revision_hash.return_value = commit_sha
+
+        runner = CliRunner()
+        with tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_commands.compile.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.config_generation.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.dbt_utils.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.dbt_utils.subprocess_run", self._mock_run
+        ):
+            result = runner.invoke(_cli, ["compile", "--docker-repository-uri", "rep"])
+            self.assertEqual(0, result.exit_code, msg=result.exception)
+
+            tmp_dir_path = pathlib.Path(tmp_dir)
+            with open(
+                tmp_dir_path.joinpath("dag", "config", "base", "k8s.yml"), "r"
+            ) as tmp_k8s:
+                self.assertDictEqual(
+                    self._k8s_content("rep", "aaa9876aaa"), yaml.safe_load(tmp_k8s)
+                )
+
+    @patch("pathlib.Path.cwd", lambda: goldens_dir_path)
+    @patch("data_pipelines_cli.data_structures.git_revision_hash")
+    def test_docker_not_installed(self, mock_git_revision_hash):
+        commit_sha = "aaa9876aaa"
+        mock_git_revision_hash.return_value = commit_sha
+
+        runner = CliRunner()
+        with patch.dict(
+            "sys.modules", docker=None
+        ), tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_commands.compile.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.config_generation.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.dbt_utils.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.dbt_utils.subprocess_run", self._mock_run
+        ):
+            result = runner.invoke(
+                _cli,
+                ["compile", "--docker-repository-uri", "rep", "--docker-build"],
+            )
+        self.assertEqual(1, result.exit_code)
+        self.assertIsInstance(result.exception, DockerNotInstalledError)
+
+    @patch("pathlib.Path.cwd", lambda: goldens_dir_path)
+    @patch("data_pipelines_cli.data_structures.git_revision_hash")
+    def test_docker_uri_build(self, mock_git_revision_hash):
+        commit_sha = "aaa9876aaa"
+        mock_git_revision_hash.return_value = commit_sha
+
+        docker_tag = None
+
+        def _mock_docker(**kwargs):
+            nonlocal docker_tag
+            docker_tag = kwargs["tag"]
+            return None, []
+
+        docker_images_mock = MagicMock()
+        docker_images_mock.configure_mock(**{"build": _mock_docker})
+        docker_client_mock = MagicMock()
+        docker_client_mock.configure_mock(**{"images": docker_images_mock})
+        docker_mock = MagicMock()
+        docker_mock.configure_mock(**{"from_env": lambda: docker_client_mock})
+
+        runner = CliRunner()
+        with patch.dict(
+            "sys.modules", docker=docker_mock
+        ), tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_commands.compile.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.config_generation.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.dbt_utils.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.dbt_utils.subprocess_run", self._mock_run
+        ):
+            result = runner.invoke(
+                _cli, ["compile", "--docker-repository-uri", "rep", "--docker-build"]
+            )
+            self.assertEqual(0, result.exit_code, msg=result.exception)
+            self.assertEqual("rep:aaa9876aaa", docker_tag)
+
+    @patch("pathlib.Path.cwd", lambda: goldens_dir_path)
+    def test_datahub_uri(self):
+        runner = CliRunner()
+        with tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_commands.compile.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.config_generation.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.dbt_utils.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.dbt_utils.subprocess_run", self._mock_run
+        ):
+            result = runner.invoke(
+                _cli, ["compile", "--datahub-gms-uri", "DaTaHuBuRi.example.io"]
+            )
+            self.assertEqual(0, result.exit_code, msg=result.exception)
+
+            tmp_dir_path = pathlib.Path(tmp_dir)
+            with open(
+                tmp_dir_path.joinpath("dag", "config", "base", "datahub.yml"), "r"
+            ) as tmp_datahub:
+                self.assertDictEqual(
+                    self._datahub_content("DaTaHuBuRi.example.io"),
+                    yaml.safe_load(tmp_datahub),
+                )
diff --git a/tests/cli_commands/test_create.py b/tests/cli_commands/test_create.py
new file mode 100644
index 0000000..7aa7ebf
--- /dev/null
+++ b/tests/cli_commands/test_create.py
@@ -0,0 +1,82 @@
+import pathlib
+import tempfile
+import unittest
+from unittest.mock import MagicMock, patch
+
+import yaml
+from click.testing import CliRunner
+
+from data_pipelines_cli.cli import _cli
+from data_pipelines_cli.errors import DataPipelinesError, NoConfigFileError
+
+
+class CreateCommandTestCase(unittest.TestCase):
+    copier_src_path = "source_path"
+    copier_dst_path = "destination_path"
+    goldens_dir_path = pathlib.Path(__file__).parent.parent.joinpath("goldens")
+
+    def _mock_copier(self, src_path: str, dst_path: str):
+        self.assertEqual(self.copier_src_path, src_path)
+        self.assertEqual(self.copier_dst_path, dst_path)
+
+    def test_create_no_config(self):
+        with tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+            pathlib.Path(tmp_dir).joinpath("non_existing_file"),
+        ):
+            runner = CliRunner(mix_stderr=False)
+            result = runner.invoke(_cli, ["create", "some_path"])
+            self.assertEqual(1, result.exit_code)
+            self.assertIsInstance(result.exception, NoConfigFileError)
+
+    @patch(
+        "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+        goldens_dir_path.joinpath("example_config.yml"),
+    )
+    def test_create_with_template_path(self):
+        with patch("copier.copy", self._mock_copier):
+            runner = CliRunner(mix_stderr=False)
+            result = runner.invoke(
+                _cli, ["create", self.copier_dst_path, self.copier_src_path]
+            )
+            self.assertEqual(0, result.exit_code, msg=result.exception)
+
+    @patch(
+        "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+        goldens_dir_path.joinpath("example_config.yml"),
+    )
+    def test_create_with_template_name(self):
+        with patch("copier.copy", self._mock_copier):
+            runner = CliRunner(mix_stderr=False)
+            result = runner.invoke(
+                _cli, ["create", self.copier_dst_path, "create_test"]
+            )
+            self.assertEqual(0, result.exit_code, msg=result.exception)
+
+    @patch(
+        "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+        goldens_dir_path.joinpath("example_config.yml"),
+    )
+    @patch("questionary.select")
+    def test_create_without_template_path(self, mock_select):
+        magic_mock = MagicMock()
+        magic_mock.configure_mock(**{"ask": lambda: "create_test"})
+        mock_select.return_value = magic_mock
+
+        with patch("copier.copy", self._mock_copier):
+            runner = CliRunner(mix_stderr=False)
+            result = runner.invoke(_cli, ["create", self.copier_dst_path])
+            self.assertEqual(0, result.exit_code, msg=result.exception)
+
+    def test_no_templates(self):
+        with tempfile.NamedTemporaryFile() as tmp_file, patch(
+            "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+            pathlib.Path(tmp_file.name),
+        ):
+            with open(tmp_file.name, "w") as f:
+                yaml.dump({"vars": {}, "templates": {}}, f)
+            runner = CliRunner(mix_stderr=False)
+            result = runner.invoke(_cli, ["create", "some_path"])
+            self.assertEqual(1, result.exit_code)
+            self.assertIsInstance(result.exception, DataPipelinesError)
+            self.assertRegex(result.exception.message, r"^No template provided\..*")
diff --git a/tests/cli_commands/test_deploy.py b/tests/cli_commands/test_deploy.py
new file mode 100644
index 0000000..21337db
--- /dev/null
+++ b/tests/cli_commands/test_deploy.py
@@ -0,0 +1,234 @@
+import json
+import os
+import pathlib
+import shutil
+import tempfile
+import unittest
+from typing import List
+from unittest.mock import MagicMock, patch
+
+import yaml
+from click.testing import CliRunner
+
+from data_pipelines_cli.cli import _cli
+from data_pipelines_cli.cli_commands.deploy import DeployCommand
+from data_pipelines_cli.errors import DataPipelinesError, DependencyNotInstalledError
+
+
+def _noop():
+    pass
+
+
+class DeployCommandTestCase(unittest.TestCase):
+    provider_args = {
+        "schmoken": "/user/schmoken/path",
+        "fooser": "user",
+        "barsword": "password",
+        "answer_to_universe_question": 42,
+        "admin_favourite_colour": {
+            "r": 255,
+            "g": 0,
+            "b": 255,
+        },
+        # `auto_mkdir` is not made up. When passed to LocalFileSystem, it
+        # creates directories containing the file if necessary
+        "auto_mkdir": True,
+    }
+    goldens_dir_path = pathlib.Path(__file__).parent.parent.joinpath("goldens")
+    dbt_project_config = {"name": "test_project"}
+
+    def setUp(self) -> None:
+        blob_json_fd, self.blob_json_filename = tempfile.mkstemp(text=True)
+        with open(blob_json_fd, "w") as f:
+            yaml.dump(self.provider_args, f)
+
+        self.dbt_project_config_dir = pathlib.Path(tempfile.mkdtemp())
+        with open(self.dbt_project_config_dir.joinpath("dbt_project.yml"), "w") as f:
+            yaml.dump(self.dbt_project_config, f)
+
+        self.storage_uri = tempfile.mkdtemp()  # this way fsspec uses LocalFS
+
+        self.subprocess_run_args = []
+
+    def _mock_run(self, args: List[str]):
+        self.subprocess_run_args = args
+
+    def tearDown(self) -> None:
+        shutil.rmtree(self.storage_uri)
+        shutil.rmtree(self.dbt_project_config_dir)
+        os.remove(self.blob_json_filename)
+
+    def test_deploy_no_args(self):
+        runner = CliRunner()
+        result = runner.invoke(_cli, ["deploy"])
+        self.assertEqual(0, result.exit_code, msg=result.exception)
+        self.assertRegex(result.output, r"^Usage: .* deploy .*")
+
+    def test_blob_args_types(self):
+        for dump, format_name in [(json.dump, "json"), (yaml.dump, "yaml")]:
+            with self.subTest(format=format_name):
+                runner = CliRunner()
+
+                with tempfile.NamedTemporaryFile(mode="w", delete=False) as tmp_file:
+                    dump(self.provider_args, tmp_file)
+
+                result_provider_kwargs = {}
+                result_blob_address = ""
+
+                def mock_init(
+                    _self,
+                    _docker_push,
+                    blob_address,
+                    provider_kwargs_dict,
+                    _datahub_ingest,
+                ):
+                    nonlocal result_provider_kwargs
+                    nonlocal result_blob_address
+                    result_provider_kwargs = provider_kwargs_dict
+                    result_blob_address = blob_address
+
+                with patch(
+                    "data_pipelines_cli.cli_commands.deploy.DeployCommand.__init__",
+                    mock_init,
+                ), patch(
+                    "data_pipelines_cli.cli_commands.deploy.DeployCommand.deploy",
+                    lambda _self: _noop,
+                ):
+                    result = runner.invoke(
+                        _cli, ["deploy", self.storage_uri, "--blob-args", tmp_file.name]
+                    )
+                self.assertEqual(0, result.exit_code, msg=result.exception)
+                self.assertDictEqual(self.provider_args, result_provider_kwargs)
+                self.assertEqual(self.storage_uri, result_blob_address)
+
+    @patch("data_pipelines_cli.cli_commands.deploy.BUILD_DIR", goldens_dir_path)
+    def test_sync_bucket(self):
+        # Worth noting: we are not extensively testing
+        # 'filesystem_utils.LocalRemoteSync' here. It gets tested in
+        # a dedicated 'test_filesystem_utils' file.
+        runner = CliRunner()
+        with patch("pathlib.Path.cwd", lambda _: self.dbt_project_config_dir):
+            result = runner.invoke(
+                _cli,
+                ["deploy", self.storage_uri, "--blob-args", self.blob_json_filename],
+            )
+        self.assertEqual(0, result.exit_code, msg=result.exception)
+        self.assertEqual(
+            2,
+            len(
+                os.listdir(
+                    pathlib.Path(self.storage_uri).joinpath(
+                        "dags", self.dbt_project_config["name"]
+                    )
+                )
+            ),
+        )
+
+    def test_no_module_cli(self):
+        for module_name, cli_args in [
+            ("datahub", ["--datahub-ingest"]),
+            ("docker", ["--docker-push", "rep"]),
+        ]:
+            with self.subTest(dep=module_name):
+                runner = CliRunner()
+                with patch.dict("sys.modules", **{module_name: None}), patch(
+                    "pathlib.Path.cwd", lambda _: self.dbt_project_config_dir
+                ):
+                    result = runner.invoke(
+                        _cli,
+                        [
+                            "deploy",
+                            self.storage_uri,
+                            "--blob-args",
+                            self.blob_json_filename,
+                            *cli_args,
+                        ],
+                    )
+                self.assertEqual(1, result.exit_code)
+                self.assertIsInstance(result.exception, DependencyNotInstalledError)
+
+    def test_no_datahub_method(self):
+        with patch.dict("sys.modules", datahub=None), patch(
+            "pathlib.Path.cwd", lambda _: self.dbt_project_config_dir
+        ):
+            with self.assertRaises(DependencyNotInstalledError):
+                DeployCommand(None, self.storage_uri, self.provider_args, True).deploy()
+
+    @patch("data_pipelines_cli.cli_commands.deploy.BUILD_DIR", goldens_dir_path)
+    def test_datahub_run(self):
+        with patch("pathlib.Path.cwd", lambda _: self.dbt_project_config_dir), patch(
+            "data_pipelines_cli.cli_commands.deploy.subprocess_run", self._mock_run
+        ), patch.dict("sys.modules", datahub=MagicMock()):
+            DeployCommand(None, self.storage_uri, self.provider_args, True).deploy()
+            self.assertListEqual(
+                [
+                    "datahub",
+                    "ingest",
+                    "-c",
+                    f"{self.goldens_dir_path}/dag/config/base/datahub.yml",
+                ],
+                self.subprocess_run_args,
+            )
+
+    def test_no_docker_method(self):
+        with patch.dict("sys.modules", docker=None), patch(
+            "pathlib.Path.cwd", lambda _: self.dbt_project_config_dir
+        ):
+            with self.assertRaises(DependencyNotInstalledError):
+                DeployCommand(
+                    "rep", self.storage_uri, self.provider_args, False
+                ).deploy()
+
+    @patch("data_pipelines_cli.cli_commands.deploy.BUILD_DIR", goldens_dir_path)
+    def test_docker_run(self):
+        docker_kwargs = {}
+
+        def _mock_docker(**kwargs):
+            nonlocal docker_kwargs
+            docker_kwargs = kwargs
+            return []
+
+        docker_images_mock = MagicMock()
+        docker_images_mock.configure_mock(**{"push": _mock_docker})
+        docker_client_mock = MagicMock()
+        docker_client_mock.configure_mock(**{"images": docker_images_mock})
+        docker_mock = MagicMock()
+        docker_mock.configure_mock(**{"from_env": lambda: docker_client_mock})
+
+        with patch.dict("sys.modules", docker=docker_mock), patch(
+            "pathlib.Path.cwd", lambda _: self.dbt_project_config_dir
+        ), patch("docker.from_env", lambda: docker_client_mock), patch(
+            "data_pipelines_cli.data_structures.git_revision_hash", lambda: "sha1234"
+        ):
+            DeployCommand("rep", self.storage_uri, self.provider_args, False).deploy()
+
+        self.assertEqual("rep", docker_kwargs.get("repository"))
+        self.assertEqual("sha1234", docker_kwargs.get("tag"))
+
+    @patch("data_pipelines_cli.cli_commands.deploy.BUILD_DIR", goldens_dir_path)
+    def test_docker_error(self):
+        def _mock_docker(**_kwargs):
+            return [
+                '{"status":"The push refers to repository [docker.io/library/rep]"}',
+                '{"errorDetail":{"message":"An image does not exist locally with '
+                'the tag: rep"},"error":"An image does not exist locally with '
+                'the tag: rep"}',
+            ]
+
+        docker_images_mock = MagicMock()
+        docker_images_mock.configure_mock(**{"push": _mock_docker})
+        docker_client_mock = MagicMock()
+        docker_client_mock.configure_mock(**{"images": docker_images_mock})
+        docker_mock = MagicMock()
+        docker_mock.configure_mock(**{"from_env": lambda: docker_client_mock})
+
+        with patch(
+            "pathlib.Path.cwd", lambda _: self.dbt_project_config_dir
+        ), patch.dict("sys.modules", docker=docker_mock), patch(
+            "data_pipelines_cli.data_structures.git_revision_hash", lambda: "sha1234"
+        ):
+
+            with self.assertRaises(DataPipelinesError):
+                DeployCommand(
+                    "rep", self.storage_uri, self.provider_args, False
+                ).deploy()
diff --git a/tests/cli_commands/test_init.py b/tests/cli_commands/test_init.py
new file mode 100644
index 0000000..e054c16
--- /dev/null
+++ b/tests/cli_commands/test_init.py
@@ -0,0 +1,112 @@
+import pathlib
+import tempfile
+import unittest
+from unittest.mock import MagicMock, patch
+
+import yaml
+from click.testing import CliRunner
+
+from data_pipelines_cli.cli import _cli
+from data_pipelines_cli.cli_commands.init import init
+from data_pipelines_cli.cli_constants import DEFAULT_GLOBAL_CONFIG
+from data_pipelines_cli.data_structures import (
+    DataPipelinesConfig,
+    TemplateConfig,
+    read_config,
+)
+from data_pipelines_cli.errors import DataPipelinesError
+
+
+class InitCommandTestCase(unittest.TestCase):
+    test_config_template_path = pathlib.Path(__file__).parent.parent.joinpath(
+        "goldens", "config_template"
+    )
+    example_config_dict = DataPipelinesConfig(
+        templates={
+            "my-template": TemplateConfig(
+                template_name="my-template",
+                template_path="https://example.com/git/example.git",
+            ),
+            "local-template": TemplateConfig(
+                template_name="local-template",
+                template_path="/var/tmp/Documents/project-template",
+            ),
+        },
+        vars={
+            "username": "test_user",
+        },
+    )
+
+    def test_init(self):
+        runner = CliRunner()
+        with tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_commands.init.CONFIGURATION_PATH",
+            pathlib.Path(tmp_dir).joinpath(".dp.yml"),
+        ), patch(
+            "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+            pathlib.Path(tmp_dir).joinpath(".dp.yml"),
+        ):
+            result = runner.invoke(
+                _cli,
+                ["init", str(self.test_config_template_path)],
+                input="test_user\n/var/tmp",
+            )
+            self.assertEqual(0, result.exit_code, msg=result.exception)
+            self.assertEqual(self.example_config_dict, read_config())
+
+    def test_global_config(self):
+        runner = CliRunner()
+        with tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_commands.init.CONFIGURATION_PATH",
+            pathlib.Path(tmp_dir).joinpath(".dp.yml"),
+        ), patch(
+            "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+            pathlib.Path(tmp_dir).joinpath(".dp.yml"),
+        ):
+            result = runner.invoke(_cli, ["init"])
+            self.assertEqual(0, result.exit_code, msg=result.exception)
+            self.assertEqual(DEFAULT_GLOBAL_CONFIG, read_config())
+
+    @patch("questionary.confirm")
+    def test_overwrite_yes(self, mock_questionary):
+        magic_mock = MagicMock()
+        magic_mock.configure_mock(**{"ask": lambda: True})
+        mock_questionary.return_value = magic_mock
+
+        with tempfile.TemporaryDirectory() as tmp_dir:
+            tmp_dp_path = pathlib.Path(tmp_dir).joinpath(".dp.yml")
+            with patch(
+                "data_pipelines_cli.cli_commands.init.CONFIGURATION_PATH",
+                tmp_dp_path,
+            ), patch(
+                "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+                tmp_dp_path,
+            ):
+                with open(tmp_dp_path, "w") as tmp_file:
+                    yaml.dump(self.example_config_dict, tmp_file)
+                    self.assertEqual(self.example_config_dict, read_config())
+                init(None)
+                self.assertEqual(DEFAULT_GLOBAL_CONFIG, read_config())
+
+    @patch("questionary.confirm")
+    def test_overwrite_no(self, mock_questionary):
+        magic_mock = MagicMock()
+        magic_mock.configure_mock(**{"ask": lambda: False})
+        mock_questionary.return_value = magic_mock
+
+        with tempfile.TemporaryDirectory() as tmp_dir:
+            tmp_dp_path = pathlib.Path(tmp_dir).joinpath(".dp.yml")
+            with patch(
+                "data_pipelines_cli.cli_commands.init.CONFIGURATION_PATH",
+                tmp_dp_path,
+            ), patch(
+                "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+                tmp_dp_path,
+            ):
+                with open(tmp_dp_path, "w") as tmp_file:
+                    yaml.dump(self.example_config_dict, tmp_file)
+                    self.assertEqual(self.example_config_dict, read_config())
+
+                with self.assertRaises(DataPipelinesError):
+                    init(None)
+                self.assertEqual(self.example_config_dict, read_config())
diff --git a/tests/cli_commands/test_prepare_env.py b/tests/cli_commands/test_prepare_env.py
new file mode 100644
index 0000000..699b204
--- /dev/null
+++ b/tests/cli_commands/test_prepare_env.py
@@ -0,0 +1,129 @@
+import pathlib
+import tempfile
+import unittest
+from unittest.mock import patch
+
+import yaml
+from click.testing import CliRunner
+
+from data_pipelines_cli.cli import _cli
+from data_pipelines_cli.cli_commands.prepare_env import prepare_env
+from data_pipelines_cli.errors import JinjaVarKeyError
+
+
+class GenHomeProfilesCommandTestCase(unittest.TestCase):
+    goldens_dir_path = pathlib.Path(__file__).parent.parent.joinpath("goldens")
+    rendered_from_vars_profile = {
+        "bigquery": {
+            "target": "env_execution",
+            "outputs": {
+                "env_execution": {
+                    "method": "service-account",
+                    "project": "example-project",
+                    "dataset": "var21-dataset",
+                    "keyfile": "/tmp/a/b/c/d.json",
+                    "timeout_seconds": 150,
+                    "priority": "interactive",
+                    "location": "us-west1",
+                    "threads": 1337,
+                    "retries": 1,
+                    "type": "bigquery",
+                }
+            },
+        }
+    }
+
+    def setUp(self) -> None:
+        self.maxDiff = None
+
+    def test_no_var_profiles_generation(self):
+        runner = CliRunner()
+
+        with tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_constants.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.config_generation.BUILD_DIR",
+            pathlib.Path(tmp_dir),
+        ), patch(
+            "data_pipelines_cli.dbt_utils.BUILD_DIR",
+            pathlib.Path(tmp_dir),
+        ), patch(
+            "pathlib.Path.cwd", lambda: self.goldens_dir_path
+        ), tempfile.TemporaryDirectory() as tmp_dir2, patch(
+            "pathlib.Path.home", lambda: pathlib.Path(tmp_dir2)
+        ):
+            runner.invoke(_cli, ["prepare-env"])
+            with open(
+                pathlib.Path(tmp_dir2).joinpath(".dbt", "profiles.yml"), "r"
+            ) as generated, open(
+                self.goldens_dir_path.joinpath(
+                    "example_profiles", "local_snowflake.yml"
+                ),
+                "r",
+            ) as prepared:
+                self.assertDictEqual(
+                    yaml.safe_load(prepared), yaml.safe_load(generated)
+                )
+
+    def test_vars_profiles_generation(self):
+        with tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_constants.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.config_generation.BUILD_DIR",
+            pathlib.Path(tmp_dir),
+        ), patch(
+            "data_pipelines_cli.dbt_utils.BUILD_DIR",
+            pathlib.Path(tmp_dir),
+        ), patch.dict(
+            "os.environ", BIGQUERY_KEYFILE="/tmp/a/b/c/d.json"
+        ), patch(
+            "pathlib.Path.cwd", lambda: self.goldens_dir_path
+        ), tempfile.TemporaryDirectory() as tmp_dir2, patch(
+            "pathlib.Path.home", lambda: pathlib.Path(tmp_dir2)
+        ):
+            prepare_env("staging")
+
+            with open(
+                pathlib.Path(tmp_dir2).joinpath(".dbt", "profiles.yml"), "r"
+            ) as generated:
+                self.assertDictEqual(
+                    self.rendered_from_vars_profile, yaml.safe_load(generated)
+                )
+
+    def test_raise_missing_variable(self):
+        with tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_constants.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.config_generation.BUILD_DIR",
+            pathlib.Path(tmp_dir),
+        ), patch(
+            "data_pipelines_cli.cli_commands.prepare_env.read_dbt_vars_from_configs",
+            lambda _env: {},
+        ), patch.dict(
+            "os.environ", BIGQUERY_KEYFILE="/tmp/a/b/c/d.json"
+        ), patch(
+            "pathlib.Path.cwd", lambda: self.goldens_dir_path
+        ), tempfile.TemporaryDirectory() as tmp_dir2, patch(
+            "pathlib.Path.home", lambda: pathlib.Path(tmp_dir2)
+        ):
+            with self.assertRaises(JinjaVarKeyError):
+                prepare_env("staging")
+
+    def test_raise_missing_environment_variable(self):
+        with tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_constants.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch(
+            "data_pipelines_cli.config_generation.BUILD_DIR",
+            pathlib.Path(tmp_dir),
+        ), patch(
+            "data_pipelines_cli.dbt_utils.BUILD_DIR",
+            pathlib.Path(tmp_dir),
+        ), patch.dict(
+            "os.environ", {}
+        ), patch(
+            "pathlib.Path.cwd", lambda: self.goldens_dir_path
+        ), tempfile.TemporaryDirectory() as tmp_dir2, patch(
+            "pathlib.Path.home", lambda: pathlib.Path(tmp_dir2)
+        ):
+            with self.assertRaises(JinjaVarKeyError):
+                prepare_env("staging")
diff --git a/tests/cli_commands/test_run_test.py b/tests/cli_commands/test_run_test.py
new file mode 100644
index 0000000..efd4aaa
--- /dev/null
+++ b/tests/cli_commands/test_run_test.py
@@ -0,0 +1,78 @@
+import pathlib
+import tempfile
+import unittest
+from typing import List
+from unittest.mock import patch
+
+from click.testing import CliRunner
+
+from data_pipelines_cli.cli import _cli
+
+
+class RunTestCommandTestCase(unittest.TestCase):
+    commands_to_test = ["run", "test"]
+    goldens_dir_path = pathlib.Path(__file__).parent.parent.joinpath("goldens")
+
+    def setUp(self) -> None:
+        self.subprocess_run_args = []
+
+    def _mock_run(self, args: List[str]):
+        self.subprocess_run_args = args
+
+    @patch(
+        "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+        goldens_dir_path.joinpath("example_config.yml"),
+    )
+    def test_no_arg(self):
+        for cmd in self.commands_to_test:
+            with self.subTest(command=cmd), patch(
+                "pathlib.Path.cwd", lambda: self.goldens_dir_path
+            ), tempfile.TemporaryDirectory() as tmp_dir, patch(
+                "data_pipelines_cli.config_generation.BUILD_DIR", pathlib.Path(tmp_dir)
+            ), patch(
+                "data_pipelines_cli.cli_commands.compile.BUILD_DIR",
+                pathlib.Path(tmp_dir),
+            ), patch(
+                "data_pipelines_cli.dbt_utils.BUILD_DIR", pathlib.Path(tmp_dir)
+            ), patch(
+                "data_pipelines_cli.dbt_utils.subprocess_run", self._mock_run
+            ):
+                runner = CliRunner()
+                result = runner.invoke(_cli, [cmd])
+                self.assertEqual(0, result.exit_code, msg=result.exception)
+
+                self.assertEqual("dbt", self.subprocess_run_args[0])
+                self.assertEqual(cmd, self.subprocess_run_args[1])
+                args_str = " ".join(self.subprocess_run_args)
+                self.assertIn("--profile snowflake", args_str)
+                self.assertIn("--target local", args_str)
+
+    @patch(
+        "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+        goldens_dir_path.joinpath("example_config.yml"),
+    )
+    def test_dev_arg(self):
+        for cmd in self.commands_to_test:
+            for env in ["dev", "staging"]:
+                with self.subTest(command=cmd, environment=env), patch(
+                    "pathlib.Path.cwd", lambda: self.goldens_dir_path
+                ), tempfile.TemporaryDirectory() as tmp_dir, patch(
+                    "data_pipelines_cli.config_generation.BUILD_DIR",
+                    pathlib.Path(tmp_dir),
+                ), patch(
+                    "data_pipelines_cli.cli_commands.compile.BUILD_DIR",
+                    pathlib.Path(tmp_dir),
+                ), patch(
+                    "data_pipelines_cli.dbt_utils.BUILD_DIR", pathlib.Path(tmp_dir)
+                ), patch(
+                    "data_pipelines_cli.dbt_utils.subprocess_run", self._mock_run
+                ):
+                    runner = CliRunner()
+                    result = runner.invoke(_cli, [cmd, "--env", env])
+                    self.assertEqual(0, result.exit_code, msg=result.exception)
+
+                    self.assertEqual("dbt", self.subprocess_run_args[0])
+                    self.assertEqual(cmd, self.subprocess_run_args[1])
+                    args_str = " ".join(self.subprocess_run_args)
+                    self.assertIn("--profile bigquery", args_str)
+                    self.assertIn("--target env_execution", args_str)
diff --git a/tests/cli_commands/test_template.py b/tests/cli_commands/test_template.py
new file mode 100644
index 0000000..e4571b6
--- /dev/null
+++ b/tests/cli_commands/test_template.py
@@ -0,0 +1,32 @@
+import pathlib
+import unittest
+from unittest.mock import patch
+
+from click.testing import CliRunner
+
+from data_pipelines_cli.cli import _cli
+
+
+class TemplateCommandTestCase(unittest.TestCase):
+    example_config_path = pathlib.Path(__file__).parent.parent.joinpath(
+        "goldens", "example_config.yml"
+    )
+
+    def test_list_templates(self):
+        runner = CliRunner()
+        with patch(
+            "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+            self.example_config_path,
+        ):
+            result = runner.invoke(_cli, ["template-list"])
+            self.assertEqual(0, result.exit_code, msg=result.exception)
+            self.assertEqual(
+                "AVAILABLE TEMPLATES:\n\n"
+                "template_name: template1\n"
+                "template_path: https://example.com/xyz/abcd.git\n\n"
+                "template_name: template2\n"
+                "template_path: https://example.com/git/example.git\n\n"
+                "template_name: create_test\n"
+                "template_path: source_path\n\n",
+                result.output,
+            )
diff --git a/tests/dag/config/base/bigquery.yml b/tests/goldens/config/base/bigquery.yml
similarity index 100%
rename from tests/dag/config/base/bigquery.yml
rename to tests/goldens/config/base/bigquery.yml
diff --git a/tests/goldens/config/base/datahub.yml b/tests/goldens/config/base/datahub.yml
new file mode 100644
index 0000000..7cd2fa2
--- /dev/null
+++ b/tests/goldens/config/base/datahub.yml
@@ -0,0 +1,3 @@
+sink:
+  config:
+    server: "https://<INGEST_ENDPOINT>:8080"
diff --git a/tests/dag/config/base/dbt.yml b/tests/goldens/config/base/dbt.yml
similarity index 100%
rename from tests/dag/config/base/dbt.yml
rename to tests/goldens/config/base/dbt.yml
diff --git a/tests/goldens/config/base/k8s.yml b/tests/goldens/config/base/k8s.yml
new file mode 100644
index 0000000..29d8ee0
--- /dev/null
+++ b/tests/goldens/config/base/k8s.yml
@@ -0,0 +1,7 @@
+image:
+  repository: <DOCKER_REPOSITORY_URL>
+  tag: <IMAGE_TAG>
+variable1: 1337
+var2: "Hello, world!"
+envs:
+  SOME_BOOLEAN: true
diff --git a/tests/dag/config/local/dbt.yml b/tests/goldens/config/local/dbt.yml
similarity index 100%
rename from tests/dag/config/local/dbt.yml
rename to tests/goldens/config/local/dbt.yml
diff --git a/tests/dag/config/local/snowflake.yml b/tests/goldens/config/local/snowflake.yml
similarity index 100%
rename from tests/dag/config/local/snowflake.yml
rename to tests/goldens/config/local/snowflake.yml
diff --git a/tests/goldens/config/staging/bigquery.yml b/tests/goldens/config/staging/bigquery.yml
new file mode 100644
index 0000000..3ebb757
--- /dev/null
+++ b/tests/goldens/config/staging/bigquery.yml
@@ -0,0 +1,9 @@
+method: service-account
+project: example-project
+dataset: "{{ var('variable_2') }}-dataset"
+threads: "{{ var('variable_1') }}"
+keyfile: "{{ env_var('BIGQUERY_KEYFILE') }}"
+timeout_seconds: 150
+priority: interactive
+location: us-west1
+retries: 1
diff --git a/tests/dag/config/staging/dbt.yml b/tests/goldens/config/staging/dbt.yml
similarity index 100%
rename from tests/dag/config/staging/dbt.yml
rename to tests/goldens/config/staging/dbt.yml
diff --git a/tests/config_template/copier.yml b/tests/goldens/config_template/copier.yml
similarity index 100%
rename from tests/config_template/copier.yml
rename to tests/goldens/config_template/copier.yml
diff --git a/tests/config_template/dp.yml.tmpl b/tests/goldens/config_template/dp.yml.tmpl
similarity index 100%
rename from tests/config_template/dp.yml.tmpl
rename to tests/goldens/config_template/dp.yml.tmpl
diff --git a/tests/test_sync_directory/test1.txt b/tests/goldens/dag/a.txt
similarity index 100%
rename from tests/test_sync_directory/test1.txt
rename to tests/goldens/dag/a.txt
diff --git a/tests/goldens/dag/b.txt b/tests/goldens/dag/b.txt
new file mode 100644
index 0000000..4632e06
--- /dev/null
+++ b/tests/goldens/dag/b.txt
@@ -0,0 +1,1 @@
+123456
\ No newline at end of file
diff --git a/tests/example_config.yml b/tests/goldens/example_config.yml
similarity index 74%
rename from tests/example_config.yml
rename to tests/goldens/example_config.yml
index 55ce7b2..980a5e0 100644
--- a/tests/example_config.yml
+++ b/tests/goldens/example_config.yml
@@ -7,3 +7,6 @@ templates:
   template2:
     template_name: template2
     template_path: https://example.com/git/example.git
+  create_test:
+    template_name: create_test
+    template_path: source_path
diff --git a/tests/example_profiles/dev_bigquery.yml b/tests/goldens/example_profiles/dev_bigquery.yml
similarity index 100%
rename from tests/example_profiles/dev_bigquery.yml
rename to tests/goldens/example_profiles/dev_bigquery.yml
diff --git a/tests/example_profiles/local_snowflake.yml b/tests/goldens/example_profiles/local_snowflake.yml
similarity index 100%
rename from tests/example_profiles/local_snowflake.yml
rename to tests/goldens/example_profiles/local_snowflake.yml
diff --git a/tests/example_profiles/staging_bigquery.yml b/tests/goldens/example_profiles/staging_bigquery.yml
similarity index 63%
rename from tests/example_profiles/staging_bigquery.yml
rename to tests/goldens/example_profiles/staging_bigquery.yml
index f2f0f2f..f094541 100644
--- a/tests/example_profiles/staging_bigquery.yml
+++ b/tests/goldens/example_profiles/staging_bigquery.yml
@@ -3,12 +3,12 @@ bigquery:
     env_execution:
       method: service-account
       project: example-project
-      dataset: example-dataset
-      keyfile: /var/keyfile.json
+      dataset: "{{ var('variable_2') }}-dataset"
+      keyfile: "{{ env_var('BIGQUERY_KEYFILE') }}"
       timeout_seconds: 150
       priority: interactive
       location: us-west1
-      threads: 1
+      threads: "{{ var('variable_1') }}"
       retries: 1
       type: bigquery
   target: env_execution
diff --git a/tests/goldens/target/manifest.json b/tests/goldens/target/manifest.json
new file mode 100644
index 0000000..3cc89a3
--- /dev/null
+++ b/tests/goldens/target/manifest.json
@@ -0,0 +1,9 @@
+{
+    "name": "manifest",
+    "description": "manifest for testing",
+    "keywords": ["key", "word"],
+    "author": "Lorem Ipsum",
+    "version": "1.2.3.4",
+    "true": false,
+    "number": 7312
+}
diff --git a/tests/test_sync_2nd_directory/a/b/c/xyz b/tests/goldens/test_sync_2nd_directory/a/b/c/xyz
similarity index 100%
rename from tests/test_sync_2nd_directory/a/b/c/xyz
rename to tests/goldens/test_sync_2nd_directory/a/b/c/xyz
diff --git a/tests/test_sync_2nd_directory/test2.txt b/tests/goldens/test_sync_2nd_directory/test2.txt
similarity index 100%
rename from tests/test_sync_2nd_directory/test2.txt
rename to tests/goldens/test_sync_2nd_directory/test2.txt
diff --git a/tests/test_sync_directory/a/b/c/xyz b/tests/goldens/test_sync_directory/a/b/c/xyz
similarity index 100%
rename from tests/test_sync_directory/a/b/c/xyz
rename to tests/goldens/test_sync_directory/a/b/c/xyz
diff --git a/tests/goldens/test_sync_directory/test1.txt b/tests/goldens/test_sync_directory/test1.txt
new file mode 100644
index 0000000..d96dc95
--- /dev/null
+++ b/tests/goldens/test_sync_directory/test1.txt
@@ -0,0 +1,1 @@
+abcdef
\ No newline at end of file
diff --git a/tests/test_sync_directory/test2.txt b/tests/goldens/test_sync_directory/test2.txt
similarity index 100%
rename from tests/test_sync_directory/test2.txt
rename to tests/goldens/test_sync_directory/test2.txt
diff --git a/tests/test_cli.py b/tests/test_cli.py
deleted file mode 100644
index 12a8e85..0000000
--- a/tests/test_cli.py
+++ /dev/null
@@ -1,107 +0,0 @@
-import pathlib
-import tempfile
-import unittest
-from unittest.mock import patch
-
-from click.testing import CliRunner
-
-from data_pipelines_cli.cli_commands.init import init_command
-from data_pipelines_cli.cli_commands.template import list_templates_command
-from data_pipelines_cli.cli_constants import DEFAULT_GLOBAL_CONFIG
-from data_pipelines_cli.data_structures import (
-    DataPipelinesConfig,
-    TemplateConfig,
-    read_config_or_exit,
-)
-
-
-class ConfigTestCase(unittest.TestCase):
-    example_config_dict = DataPipelinesConfig(
-        templates={
-            "template1": TemplateConfig(
-                template_name="template1",
-                template_path="https://example.com/xyz/abcd.git",
-            ),
-            "template2": TemplateConfig(
-                template_name="template2",
-                template_path="https://example.com/git/example.git",
-            ),
-        },
-        vars={"username": "testuser"},
-    )
-    example_config_path = pathlib.Path(__file__).parent.joinpath("example_config.yml")
-
-    def test_read_config(self):
-        with patch(
-            "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
-            self.example_config_path,
-        ):
-            self.assertEqual(self.example_config_dict, read_config_or_exit())
-
-    def test_list_templates(self):
-        runner = CliRunner()
-        with patch(
-            "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
-            self.example_config_path,
-        ):
-            result = runner.invoke(list_templates_command)
-            self.assertEqual(0, result.exit_code)
-            self.assertEqual(
-                "AVAILABLE TEMPLATES:\n\n"
-                "template_name: template1\n"
-                "template_path: https://example.com/xyz/abcd.git\n\n"
-                "template_name: template2\n"
-                "template_path: https://example.com/git/example.git\n\n",
-                result.output,
-            )
-
-
-class InitTestCase(unittest.TestCase):
-    test_config_template_path = pathlib.Path(__file__).parent.joinpath(
-        "config_template"
-    )
-    example_config_dict = DataPipelinesConfig(
-        templates={
-            "my-template": TemplateConfig(
-                template_name="my-template",
-                template_path="https://example.com/git/example.git",
-            ),
-            "local-template": TemplateConfig(
-                template_name="local-template",
-                template_path="/var/tmp/Documents/project-template",
-            ),
-        },
-        vars={
-            "username": "test_user",
-        },
-    )
-
-    def test_init(self):
-        runner = CliRunner()
-        with tempfile.TemporaryDirectory() as tmp_dir, patch(
-            "data_pipelines_cli.cli_commands.init.CONFIGURATION_PATH",
-            pathlib.Path(tmp_dir).joinpath(".dp.yml"),
-        ), patch(
-            "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
-            pathlib.Path(tmp_dir).joinpath(".dp.yml"),
-        ):
-            result = runner.invoke(
-                init_command,
-                [str(self.test_config_template_path)],
-                input="test_user\n/var/tmp",
-            )
-            self.assertEqual(0, result.exit_code)
-            self.assertEqual(self.example_config_dict, read_config_or_exit())
-
-    def test_global_config(self):
-        runner = CliRunner()
-        with tempfile.TemporaryDirectory() as tmp_dir, patch(
-            "data_pipelines_cli.cli_commands.init.CONFIGURATION_PATH",
-            pathlib.Path(tmp_dir).joinpath(".dp.yml"),
-        ), patch(
-            "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
-            pathlib.Path(tmp_dir).joinpath(".dp.yml"),
-        ):
-            result = runner.invoke(init_command)
-            self.assertEqual(0, result.exit_code)
-            self.assertEqual(DEFAULT_GLOBAL_CONFIG, read_config_or_exit())
diff --git a/tests/test_cli_utils.py b/tests/test_cli_utils.py
new file mode 100644
index 0000000..16227f0
--- /dev/null
+++ b/tests/test_cli_utils.py
@@ -0,0 +1,93 @@
+import os
+import subprocess
+import unittest
+from io import StringIO
+from unittest.mock import patch
+
+from data_pipelines_cli.cli_utils import (
+    echo_error,
+    echo_info,
+    echo_subinfo,
+    echo_warning,
+    get_argument_or_environment_variable,
+    subprocess_run,
+)
+from data_pipelines_cli.errors import (
+    DataPipelinesError,
+    SubprocessNonZeroExitError,
+    SubprocessNotFound,
+)
+
+
+class CliUtilsTest(unittest.TestCase):
+    echo_is_printing_to_out = [
+        (echo_error, False),
+        (echo_warning, False),
+        (echo_info, True),
+        (echo_subinfo, True),
+    ]
+
+    def test_echoes_to_proper_streams(self):
+        test_string = "Hello world!"
+        endlined_test_string = test_string + "\n"
+
+        for echo_fun, is_stdout in self.echo_is_printing_to_out:
+            with self.subTest(function=echo_fun.__name__), patch(
+                "sys.stdout", new=StringIO()
+            ) as fake_out, patch("sys.stderr", new=StringIO()) as fake_err:
+                echo_fun(test_string)
+                self.assertEqual(
+                    endlined_test_string if is_stdout else "", fake_out.getvalue()
+                )
+                self.assertEqual(
+                    endlined_test_string if not is_stdout else "", fake_err.getvalue()
+                )
+
+    some_env_variable_key = "SOME_VARIABLE"
+    some_env_variable_value = "some_value"
+
+    @patch.dict(os.environ, {some_env_variable_key: some_env_variable_value})
+    def test_get_argument_from_argument(self):
+        argument = "argument"
+        self.assertEqual(
+            argument,
+            get_argument_or_environment_variable(
+                argument, "arg", self.some_env_variable_key
+            ),
+        )
+
+    @patch.dict(os.environ, {some_env_variable_key: some_env_variable_value})
+    def test_get_argument_from_env_var(self):
+        self.assertEqual(
+            self.some_env_variable_value,
+            get_argument_or_environment_variable(
+                None, "arg", self.some_env_variable_key
+            ),
+        )
+
+    @patch.dict(os.environ, {})
+    def test_get_argument_throw(self):
+        with self.assertRaises(DataPipelinesError):
+            get_argument_or_environment_variable(
+                None, "arg", self.some_env_variable_key
+            )
+
+    @patch("data_pipelines_cli.cli_utils.subprocess.run")
+    def test_subprocess_run_return_code(self, mock_run):
+        mock_run.return_value.returncode = 0
+        result = subprocess_run(["testproc", "--arg"])
+        self.assertEqual(0, result.returncode)
+
+    @patch("data_pipelines_cli.cli_utils.subprocess.run")
+    def test_subprocess_run_not_exist(self, mock_run):
+        mock_run.side_effect = FileNotFoundError
+        with self.assertRaises(SubprocessNotFound) as exc:
+            _ = subprocess_run(["testproc", "--arg"])
+        self.assertRegex(exc.exception.message, r"^testproc.*$")
+
+    @patch("data_pipelines_cli.cli_utils.subprocess.run")
+    def test_subprocess_run_nonzero_throws(self, mock_run):
+        mock_run.side_effect = subprocess.CalledProcessError(21, cmd="")
+        with self.assertRaises(SubprocessNonZeroExitError) as exc:
+            _ = subprocess_run(["testproc", "--arg"])
+        self.assertRegex(exc.exception.message, r"^testproc.*21$")
diff --git a/tests/test_config_generation.py b/tests/test_config_generation.py
index c3d7600..ed9364e 100644
--- a/tests/test_config_generation.py
+++ b/tests/test_config_generation.py
@@ -1,5 +1,6 @@
 import os
 import pathlib
+import tempfile
 import unittest
 from unittest.mock import patch
 
@@ -18,34 +19,105 @@ class TestConfigGeneration(unittest.TestCase):
         ("staging", "bigquery"),
         ("local", "snowflake"),
     ]
+    configs_to_test = {
+        "dev": {
+            "target": "env_execution",
+            "target_type": "bigquery",
+            "vars": {
+                "variable_1": 123,
+                "variable_2": "var2",
+            },
+        },
+        "local": {
+            "target": "local",
+            "target_type": "snowflake",
+            "vars": {
+                "variable_1": 123,
+                "variable_2": "var2",
+            },
+        },
+        "staging": {
+            "target": "env_execution",
+            "target_type": "bigquery",
+            "vars": {
+                "variable_1": 1337,
+                "variable_2": "var21",
+            },
+        },
+    }
 
     def setUp(self) -> None:
         self.maxDiff = None
-        self.current_dir_path = pathlib.Path(__file__).parent
+        self.goldens_dir_path = pathlib.Path(__file__).parent.joinpath("goldens")
+
+    def test_copy_dag_dir(self):
+        with tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.config_generation.BUILD_DIR", pathlib.Path(tmp_dir)
+        ), patch("pathlib.Path.cwd", lambda: self.goldens_dir_path):
+            cgen.copy_dag_dir_to_build_dir()
+            tmp_dag_dir = pathlib.Path(tmp_dir).joinpath("dag")
+            self.assertEqual(2, len(os.listdir(tmp_dag_dir)))
+            with open(tmp_dag_dir.joinpath("a.txt"), "r") as f:
+                self.assertEqual("abcdef", f.read())
+            with open(tmp_dag_dir.joinpath("b.txt"), "r") as f:
+                self.assertEqual("123456", f.read())
+
+    def test_copy_dir_rmdir_if_exists(self):
+        with tempfile.TemporaryDirectory() as tmp_dir1, tempfile.TemporaryDirectory() as tmp_dir2:  # noqa: E501
+            path1 = pathlib.Path(tmp_dir2)
+            with open(path1.joinpath("a.txt"), "w") as f:
+                f.write("qwerty987")
+            path1.joinpath("b.txt").touch()
+            self.assertEqual(2, len(os.listdir(path1)))
+
+            path2 = pathlib.Path(tmp_dir1)
+            with open(path2.joinpath("a.txt"), "w") as f:
+                f.write("abc1234")
+            path2.joinpath("b.txt").touch()
+            path2.joinpath("c.txt").touch()
+            self.assertEqual(3, len(os.listdir(path2)))
+            with open(path2.joinpath("a.txt"), "r") as f:
+                self.assertEqual("abc1234", f.read())
+
+            cgen._copy_src_dir_to_dst_dir(path1, path2)
+
+            self.assertEqual(2, len(os.listdir(path2)))
+            with open(path2.joinpath("a.txt"), "r") as f:
+                self.assertEqual("qwerty987", f.read())
+
+    def test_read_from_config_dir(self):
+        for env, _ in self.envs_to_test:
+            with self.subTest(env=env):
+                self.assertDictEqual(
+                    self.configs_to_test[env],
+                    cgen.read_dictionary_from_config_directory(
+                        self.goldens_dir_path, env, "dbt.yml"
+                    ),
+                )
 
     def test_generation(self):
         for env, profile_type in self.envs_to_test:
-            with self.subTest(env=env, profile_type=profile_type), patch(
-                "data_pipelines_cli.cli_constants.BUILD_DIR", self.current_dir_path
+            with self.subTest(
+                env=env, profile_type=profile_type
+            ), tempfile.TemporaryDirectory() as tmp_dir, patch(
+                "data_pipelines_cli.cli_constants.BUILD_DIR", pathlib.Path(tmp_dir)
             ), patch(
                 "data_pipelines_cli.config_generation.BUILD_DIR",
-                self.current_dir_path,
+                pathlib.Path(tmp_dir),
             ), patch(
-                "data_pipelines_cli.config_generation.copy_config_dir_to_build_dir",
-                _noop,
+                "pathlib.Path.cwd", lambda: self.goldens_dir_path
             ):
-
                 self.profiles_path = cgen.generate_profiles_yml(env).joinpath(
                     "profiles.yml"
                 )
                 with open(self.profiles_path, "r") as generated, open(
-                    self.current_dir_path.joinpath(
+                    self.goldens_dir_path.joinpath(
                         "example_profiles", f"{env}_{profile_type}.yml"
                     ),
                     "r",
                 ) as prepared:
                     self.assertDictEqual(
-                        yaml.safe_load(generated), yaml.safe_load(prepared)
+                        yaml.safe_load(prepared), yaml.safe_load(generated)
                     )
 
                 os.remove(self.profiles_path)
diff --git a/tests/test_data_structures.py b/tests/test_data_structures.py
new file mode 100644
index 0000000..f77a966
--- /dev/null
+++ b/tests/test_data_structures.py
@@ -0,0 +1,81 @@
+import os
+import pathlib
+import tempfile
+import unittest
+from unittest.mock import patch
+
+from data_pipelines_cli.data_structures import (
+    DataPipelinesConfig,
+    DockerArgs,
+    TemplateConfig,
+    read_config,
+)
+from data_pipelines_cli.errors import DataPipelinesError, NoConfigFileError
+
+
+class DataStructuresTestCase(unittest.TestCase):
+    example_config_dict = DataPipelinesConfig(
+        templates={
+            "template1": TemplateConfig(
+                template_name="template1",
+                template_path="https://example.com/xyz/abcd.git",
+            ),
+            "template2": TemplateConfig(
+                template_name="template2",
+                template_path="https://example.com/git/example.git",
+            ),
+            "create_test": TemplateConfig(
+                template_name="create_test",
+                template_path="source_path",
+            ),
+        },
+        vars={"username": "testuser"},
+    )
+    example_config_path = pathlib.Path(__file__).parent.joinpath(
+        "goldens", "example_config.yml"
+    )
+
+    def test_read_config(self):
+        with patch(
+            "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+            self.example_config_path,
+        ):
+            self.assertEqual(self.example_config_dict, read_config())
+
+    def test_read_config_no_file(self):
+        with tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+            pathlib.Path(tmp_dir).joinpath("non_existing_file.yml"),
+        ):
+            with self.assertRaises(NoConfigFileError):
+                read_config()
+
+
+class DockerArgsTest(unittest.TestCase):
+    @patch("data_pipelines_cli.data_structures.git_revision_hash")
+    def test_build_tag(self, mock_git_revision_hash):
+        repository = "rep"
+        commit_sha = "eee440bfbe0801ec3f533f897c1d55e6a5afd5cd"
+        mock_git_revision_hash.return_value = commit_sha
+
+        docker_args = DockerArgs(repository)
+        self.assertEqual(f"{repository}:{commit_sha}", docker_args.docker_build_tag())
+        self.assertEqual(repository, docker_args.repository)
+        self.assertEqual(commit_sha, docker_args.commit_sha)
+
+    @patch("data_pipelines_cli.data_structures.git_revision_hash")
+    def test_no_git_hash(self, mock_git_revision_hash):
+        mock_git_revision_hash.return_value = None
+
+        with self.assertRaises(DataPipelinesError):
+            _ = DockerArgs("rep")
+
+    @patch("data_pipelines_cli.data_structures.git_revision_hash")
+    @patch.dict(os.environ, {"REPOSITORY_URL": "repo"})
+    def test_no_repository_argument(self, mock_git_revision_hash):
+        commit_sha = "eee440bfbe0801ec3f533f897c1d55e6a5afd5cd"
+        mock_git_revision_hash.return_value = commit_sha
+
+        docker_args = DockerArgs(None)
+        self.assertEqual("repo", docker_args.repository)
+        self.assertEqual(commit_sha, docker_args.commit_sha)
diff --git a/tests/test_dbt_utils.py b/tests/test_dbt_utils.py
new file mode 100644
index 0000000..a465c9a
--- /dev/null
+++ b/tests/test_dbt_utils.py
@@ -0,0 +1,75 @@
+import pathlib
+import tempfile
+import unittest
+from typing import List
+from unittest.mock import patch
+
+import yaml
+
+from data_pipelines_cli.dbt_utils import read_dbt_vars_from_configs, run_dbt_command
+from data_pipelines_cli.errors import NoConfigFileError
+
+
+class DbtUtilsTest(unittest.TestCase):
+    dp_config = {
+        "vars": {
+            "var1": 1,
+            "var2": "var2_value",
+        },
+        "templates": {},
+    }
+    dbt_config = {
+        "target": "env_execution",
+        "target_type": "bigquery",
+    }
+    goldens_dir_path = pathlib.Path(__file__).parent.joinpath("goldens")
+
+    def setUp(self) -> None:
+        self.subprocess_run_args = []
+
+    def _mock_run(self, args: List[str]):
+        self.subprocess_run_args = args
+
+    def test_dbt_run(self):
+        with tempfile.NamedTemporaryFile() as tmp_file, patch(
+            "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+            pathlib.Path(tmp_file.name),
+        ), patch(
+            "data_pipelines_cli.dbt_utils.read_dictionary_from_config_directory",
+            lambda _a, _b, _c: self.dbt_config,
+        ), patch(
+            "data_pipelines_cli.dbt_utils.subprocess_run", self._mock_run
+        ):
+            with open(tmp_file.name, "w") as f:
+                yaml.dump(self.dp_config, f)
+            run_dbt_command(
+                ("really", "long", "command"), "test_env", pathlib.Path("profiles_path")
+            )
+            self.assertListEqual(
+                [
+                    "dbt",
+                    "really",
+                    "long",
+                    "command",
+                    "--profile",
+                    "bigquery",
+                    "--profiles-dir",
+                    "profiles_path",
+                    "--target",
+                    "env_execution",
+                    "--vars",
+                    "{var1: 1, var2: var2_value}\n",
+                ],
+                self.subprocess_run_args,
+            )
+
+    def test_read_vars_no_throw(self):
+        with tempfile.TemporaryDirectory() as tmp_dir, patch(
+            "data_pipelines_cli.cli_constants.CONFIGURATION_PATH",
+            pathlib.Path(tmp_dir).joinpath("non_existing_config_file"),
+        ):
+            try:
+                result = read_dbt_vars_from_configs("env")
+                self.assertDictEqual({}, result)
+            except NoConfigFileError:
+                self.fail("_read_dbt_vars_from_configs() raised NoConfigFileError!")
diff --git a/tests/test_filesystems.py b/tests/test_filesystem_utils.py
similarity index 82%
rename from tests/test_filesystems.py
rename to tests/test_filesystem_utils.py
index 6b585d9..e756f24 100644
--- a/tests/test_filesystems.py
+++ b/tests/test_filesystem_utils.py
@@ -1,4 +1,6 @@
 import pathlib
+import random
+import string
 import unittest
 
 import boto3
@@ -7,6 +9,8 @@
 import moto
 from moto import mock_s3
 
+from data_pipelines_cli.errors import DataPipelinesError
+
 MY_BUCKET = "my_bucket"
 
 
@@ -21,6 +25,20 @@ async def read(self):  # type: ignore
         return self.text
 
 
+class TestError(unittest.TestCase):
+    def test_wrong_local_path(self):
+        from data_pipelines_cli.filesystem_utils import LocalRemoteSync
+
+        wrong_local_path = pathlib.Path(__file__).parent.joinpath(
+            "".join(random.choices(string.ascii_letters + string.digits, k=25))
+        )
+        remote_path = "".join(
+            random.choices(string.ascii_letters + string.digits + "/", k=25)
+        )
+        with self.assertRaises(DataPipelinesError):
+            LocalRemoteSync(wrong_local_path, remote_path, {}).sync(delete=False)
+
+
 class TestSynchronize(unittest.TestCase):
     test_sync_2nd_directory_layout = [
         "test2.txt",
@@ -31,7 +49,9 @@ class TestSynchronize(unittest.TestCase):
     def _test_synchronize(self, protocol: str, **remote_kwargs):
         from data_pipelines_cli.filesystem_utils import LocalRemoteSync
 
-        local_path = pathlib.Path(__file__).parent.joinpath("test_sync_directory")
+        local_path = pathlib.Path(__file__).parent.joinpath(
+            "goldens", "test_sync_directory"
+        )
         remote_path = f"{protocol}://{MY_BUCKET}/"
         LocalRemoteSync(local_path, remote_path, remote_kwargs).sync(delete=False)
 
@@ -45,7 +65,9 @@ def _test_synchronize(self, protocol: str, **remote_kwargs):
     def _test_synchronize_with_delete(self, protocol: str, **remote_kwargs):
         from data_pipelines_cli.filesystem_utils import LocalRemoteSync
 
-        local_path = pathlib.Path(__file__).parent.joinpath("test_sync_directory")
+        local_path = pathlib.Path(__file__).parent.joinpath(
+            "goldens", "test_sync_directory"
+        )
         remote_path = f"{protocol}://{MY_BUCKET}/"
         LocalRemoteSync(local_path, remote_path, remote_kwargs).sync(delete=True)
 
@@ -56,7 +78,9 @@ def _test_synchronize_with_delete(self, protocol: str, **remote_kwargs):
                 remote_fs.find(MY_BUCKET),
             )
 
-        local_path_2 = pathlib.Path(__file__).parent.joinpath("test_sync_2nd_directory")
+        local_path_2 = pathlib.Path(__file__).parent.joinpath(
+            "goldens", "test_sync_2nd_directory"
+        )
         LocalRemoteSync(
             local_path_2,
             remote_path,
diff --git a/tests/test_io_utils.py b/tests/test_io_utils.py
new file mode 100644
index 0000000..d182f48
--- /dev/null
+++ b/tests/test_io_utils.py
@@ -0,0 +1,59 @@
+import pathlib
+import subprocess
+import tempfile
+import unittest
+from io import StringIO
+from unittest.mock import MagicMock, patch
+
+from data_pipelines_cli.io_utils import git_revision_hash, replace
+
+
+class TestReplace(unittest.TestCase):
+    pattern_to_replace = "<(TEST-PaTtErN___!@#$%^abcdef__"
+    regex_to_replace = r"<\(TEST-PaTtErN___!@#\$%\^[a-z]+__"
+    replacement = "0x13370x42  ||  http://example,com<>"
+    text_to_replace = (
+        f'<(TEST-Pattern,1234{pattern_to_replace}>>>>=><><"\n\n{pattern_to_replace}xxx'
+    )
+    expected_result = f'<(TEST-Pattern,1234{replacement}>>>>=><><"\n\n{replacement}xxx'
+
+    def test_replace(self):
+        with tempfile.TemporaryDirectory() as tmp_dir:
+            filename = pathlib.Path(tmp_dir).joinpath("test")
+            with open(filename, "w") as tmp_file:
+                tmp_file.write(self.text_to_replace)
+            replace(tmp_file.name, self.regex_to_replace, self.replacement)
+            with open(filename, "r") as tmp_file:
+                output = tmp_file.readlines()
+                self.assertEqual(self.expected_result, "".join(output))
+
+
+class TestGitRevisionHash(unittest.TestCase):
+    @patch("data_pipelines_cli.io_utils.subprocess.run")
+    def test_git_revision_hash(self, mock_run):
+        git_sha = "abcdef1337"
+
+        mock_stdout = MagicMock()
+        mock_stdout.configure_mock(**{"stdout.decode.return_value": git_sha})
+        mock_run.return_value = mock_stdout
+
+        result = git_revision_hash()
+        self.assertEqual(git_sha, result)
+
+    @patch("data_pipelines_cli.io_utils.subprocess.run")
+    def test_git_does_not_exist(self, mock_run):
+        mock_run.side_effect = FileNotFoundError
+        result = git_revision_hash()
+        self.assertEqual(None, result)
+
+    @patch("data_pipelines_cli.io_utils.subprocess.run")
+    def test_git_error(self, mock_run):
+        test_error = "Some test error"
+        mock_run.side_effect = subprocess.CalledProcessError(
+            128, cmd="", stderr=test_error
+        )
+
+        with patch("sys.stderr", new=StringIO()) as fake_out:
+            result = git_revision_hash()
+            self.assertEqual(None, result)
+            self.assertIn(test_error, fake_out.getvalue())