Repository: iterative/dvc

Issue:
hooks: dvc appends hooks and breaks the existing hook
If git hook is a python file users end up with a broken hook - a mix of python and bash code. We need to find a way to mitigate this.




Discussion:
We originally didn't account for a hook being anything than a shell script and so we were just appending to it. I guess we could check the shebang to verify that it is `#!/bin/sh`(or a variation of it) and if it is - append, if not - throw an error or a warning(or a prompt?). That would be some pretty nasty heuristics, but I see no other way of solving this. Also, we could do the next step with the heuristics and append to pre-commit config file :slightly_smiling_face: But that also sounds as a pretty awful magic :sparkles: :sparkles: :sparkles:  
it's really annoying that Git hooks are implemented this way. I think we should stop appending if there is the is a file already - it's not nice that the top level command `dvc install` breaks the environment for users. Or at least ask a confirmation with some explanation of what's going on.

Are there some general frameworks that can be used to install hooks? Something like Husky, but not JS-specific?
@shcheklein 

> it's really annoying that Git hooks are implemented this way. I think we should stop appending if there is the is a file already - it's not nice that the top level command dvc install breaks the environment for users. Or at least ask a confirmation with some explanation of what's going on.

We've actually added that `append` functionality not that long ago in https://github.com/iterative/dvc/issues/1964 , before that we just errored-out if the hook existed already.

> Are there some general frameworks that can be used to install hooks? Something like Husky, but not JS-specific?

Looks like it is `pre-commit` after all. But in any case, different projects might have different git hooks frameworks and it doesn't look like we would be able to suit them all. We could start detecting and supporting those one-by-one, but that would be complicated, and it feels like we could simply write the instructions for now, that would tell people to use our commands in their hooks.


@efiop looks like the solution was wrong even for that ticket. Original author of that ticket has the problem with dvc breaking the existing hook, which makes experience even worse.
@shcheklein , what about changing the `dvc install` command (which edits the hook with some heuristics as @efiop noted above) to `dvc hook` that will output the content to be executed in a script.

For example, [`hub`](https://github.com/github/hub) offers a similar interface: `hub alias -s` that outputs `alias git=hub`, and the intended use is to append `eval "$(hub alias -s)"` to a `.*rc` file.

With this interface, users can use `dvc hook post-checkout > .git/hooks/checkout` or `dvc hook post-checkout >> .git/hooks/checkout` or let the users inspect the `dvc hook` itself and adapt it to their existent script.

What I mean, is to delegate the `install`ation to the users and provide the script itself
@mroutis I like the idea of delegating to the user and for the command to output the content. May be we can add an option to write directly to the file. We can also add some options to output it in a few languages - `--python`, `--shell`, etc

I'm not sure about the naming - `dvc hook` can conflict in the future with DVC hooks. Even though I understand that `dvc install` looks a bit weird - a global command just to manipulate hooks feels too much.

I just ran into this as well. I was looking to use the `black` autoformatter pre-commit hook along with dvc. Is it possible to support `.pre-commit-config.yaml` directly? Seems like it can support multiple hook types.
@AlJohri It does support multiple hooks, but not all :( Currently it supports `pre-commit,pre-push,prepare-commit-msg,commit-msg`, and we need `post-checkout` in addition to that. We could ask pre-commit guys to implement `post-checkout` so we are covered and/or temporarily use a hybrid solution.
First step is to detect if git hook already exists and then refuse to install our hook into it, to avoid nasty situations.
Let's first check that no hooks that we want to install exist. If they do - refuse to install anything (and refer to a documentation). If they don't exist - install.

Resolving patch:
diff --git a/dvc/exceptions.py b/dvc/exceptions.py
--- a/dvc/exceptions.py
+++ b/dvc/exceptions.py
@@ -294,3 +294,12 @@ def __init__(self):
             "the given path is a DVC-file, you must specify a data file "
             "or a directory"
         )
+
+
+class GitHookAlreadyExistsError(DvcException):
+    def __init__(self, hook_name):
+        super(GitHookAlreadyExistsError, self).__init__(
+            "Hook '{}' already exists. Please refer to "
+            "https://man.dvc.org/install "
+            "for more info.".format(hook_name)
+        )
diff --git a/dvc/scm/git/__init__.py b/dvc/scm/git/__init__.py
--- a/dvc/scm/git/__init__.py
+++ b/dvc/scm/git/__init__.py
@@ -5,6 +5,9 @@
 import os
 import logging
 
+from funcy import cached_property
+
+from dvc.exceptions import GitHookAlreadyExistsError
 from dvc.utils.compat import str, open
 from dvc.utils import fix_env, relpath
 from dvc.scm.base import (
@@ -234,7 +237,7 @@ def _install_hook(self, name, cmd):
             " || exec dvc {}".format(cmd)
         )
 
-        hook = os.path.join(self.root_dir, self.GIT_DIR, "hooks", name)
+        hook = self._hook_path(name)
 
         if os.path.isfile(hook):
             with open(hook, "r+") as fobj:
@@ -247,6 +250,8 @@ def _install_hook(self, name, cmd):
         os.chmod(hook, 0o777)
 
     def install(self):
+        self._verify_dvc_hooks()
+
         self._install_hook("post-checkout", "checkout")
         self._install_hook("pre-commit", "status")
         self._install_hook("pre-push", "push")
@@ -342,3 +347,19 @@ def get_rev(self):
 
     def close(self):
         self.repo.close()
+
+    @cached_property
+    def _hooks_home(self):
+        return os.path.join(self.root_dir, self.GIT_DIR, "hooks")
+
+    def _hook_path(self, name):
+        return os.path.join(self._hooks_home, name)
+
+    def _verify_hook(self, name):
+        if os.path.exists(self._hook_path(name)):
+            raise GitHookAlreadyExistsError(name)
+
+    def _verify_dvc_hooks(self):
+        self._verify_hook("post-checkout")
+        self._verify_hook("pre-commit")
+        self._verify_hook("pre-push")


Test patch:
diff --git a/tests/func/test_install.py b/tests/func/test_install.py
--- a/tests/func/test_install.py
+++ b/tests/func/test_install.py
@@ -2,6 +2,8 @@
 import sys
 
 import pytest
+
+from dvc.exceptions import GitHookAlreadyExistsError
 from dvc.utils import file_md5
 
 from dvc.main import main
@@ -30,36 +32,12 @@ def test_should_create_hooks(self, git, dvc_repo):
             with open(self._hook(fname), "r") as fobj:
                 assert command in fobj.read()
 
-    def test_should_append_hooks_if_file_already_exists(self, git, dvc_repo):
+    def test_should_fail_if_file_already_exists(self, git, dvc_repo):
         with open(self._hook("post-checkout"), "w") as fobj:
-            fobj.write("#!/bin/sh\n" "echo hello\n")
-
-        assert main(["install"]) == 0
-
-        expected_script = (
-            "#!/bin/sh\n"
-            "echo hello\n"
-            '[ "$3" = "0" ]'
-            ' || [ -z "$(git ls-files .dvc)" ]'
-            " || exec dvc checkout\n"
-        )
-
-        with open(self._hook("post-checkout"), "r") as fobj:
-            assert fobj.read() == expected_script
-
-    def test_should_be_idempotent(self, git, dvc_repo):
-        assert main(["install"]) == 0
-        assert main(["install"]) == 0
-
-        expected_script = (
-            "#!/bin/sh\n"
-            '[ "$3" = "0" ]'
-            ' || [ -z "$(git ls-files .dvc)" ]'
-            " || exec dvc checkout\n"
-        )
+            fobj.write("hook content")
 
-        with open(self._hook("post-checkout"), "r") as fobj:
-            assert fobj.read() == expected_script
+        with pytest.raises(GitHookAlreadyExistsError):
+            dvc_repo.scm.install()
 
     def test_should_post_checkout_hook_checkout(self, repo_dir, git, dvc_repo):
         assert main(["install"]) == 0