Repository: pre-commit/pre-commit

Issue:
documention regarding adding new python based hooks needs improvement
Apparently we need some kind of how-to or mini tutorial on how to add a new hook to pre-commit as the basic documentation does not help someone without previous pre-commit knowledge.

I wanted to add support for `bashate`, a shell script linter written in python, available on pypi and that installs a shell script with the same name that can be used just like other linters.

Initially I went to https://pre-commit.com/#new-hooks which didn't give me enough info. So, i looked for other linters based on python and I found yamllint, which pointed me to https://github.com/adrienverge/yamllint/blob/master/.pre-commit-hooks.yaml

So the idea was to add the hook definition directly to the linter package. In this case I had to fork bashate in order to test the new, hook. So I ended up creating https://github.com/ssbarnea/bashate/blob/master/.pre-commit-hooks.yaml -- folowing the same model used in yamllint.

Now, I wanted to add and test the hook on one of the repos I maintain so I did create https://github.com/pycontribs/jira/blob/feature/pre-commit/.pre-commit-config.yaml#L25

When I tried to run it using `pre-commit run bashate --all`, it failed with this error:
```
Bashate..................................................................An unexpected error has occurred: OSError: [Errno 2] No such file or directory
Check the log at /Users/ssbarnea/.cache/pre-commit/pre-commit.log

An unexpected error has occurred: OSError: [Errno 2] No such file or directory
Traceback (most recent call last):
  File "/Users/ssbarnea/.pyenv/versions/2.7.14/lib/python2.7/site-packages/pre_commit/error_handler.py", line 47, in error_handler
    yield
  File "/Users/ssbarnea/.pyenv/versions/2.7.14/lib/python2.7/site-packages/pre_commit/main.py", line 258, in main
    return run(runner, args)
  File "/Users/ssbarnea/.pyenv/versions/2.7.14/lib/python2.7/site-packages/pre_commit/commands/run.py", line 270, in run
    return _run_hooks(runner.config, repo_hooks, args, environ)
  File "/Users/ssbarnea/.pyenv/versions/2.7.14/lib/python2.7/site-packages/pre_commit/commands/run.py", line 199, in _run_hooks
    retval |= _run_single_hook(filenames, hook, repo, args, skips, cols)
  File "/Users/ssbarnea/.pyenv/versions/2.7.14/lib/python2.7/site-packages/pre_commit/commands/run.py", line 110, in _run_single_hook
    hook, tuple(filenames) if hook['pass_filenames'] else (),
  File "/Users/ssbarnea/.pyenv/versions/2.7.14/lib/python2.7/site-packages/pre_commit/repository.py", line 207, in run_hook
    return languages[language_name].run_hook(prefix, hook, file_args)
  File "/Users/ssbarnea/.pyenv/versions/2.7.14/lib/python2.7/site-packages/pre_commit/languages/script.py", line 16, in run_hook
    return xargs(cmd, file_args)
  File "/Users/ssbarnea/.pyenv/versions/2.7.14/lib/python2.7/site-packages/pre_commit/xargs.py", line 63, in xargs
    *run_cmd, encoding=None, retcode=None
  File "/Users/ssbarnea/.pyenv/versions/2.7.14/lib/python2.7/site-packages/pre_commit/util.py", line 167, in cmd_output
    proc = subprocess.Popen(cmd, **popen_kwargs)
  File "/Users/ssbarnea/.pyenv/versions/2.7.14/lib/python2.7/subprocess.py", line 390, in __init__
    errread, errwrite)
  File "/Users/ssbarnea/.pyenv/versions/2.7.14/lib/python2.7/subprocess.py", line 1025, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
```

At the moment I have no idea what I did wrong, probably something simple. Still, we need to make it easier to integrate new linters into pre-commit.


Discussion:
that configuration looks completely correct to me, I suspect there's some sort of regression as well as `OSError` isn't supposed to escape.

If you could provide more platform / environment information that would be extremely helpful :)

(an aside: `types: [file, shell]` is redundant as `shell` will only be applied to file object -- not broken but just extra)
The other bit being, your configuration works for me:

```
$ pre-commit try-repo https://github.com/ssbarnea/bashate --all-files
[INFO] Initializing environment for https://github.com/ssbarnea/bashate.
===============================================================================
Using config:
===============================================================================
repos:
-   repo: https://github.com/ssbarnea/bashate
    rev: 3b9aa602d8bab522e803d1f05f223b908f610731
    hooks:
    -   id: bashate
===============================================================================
[INFO] Installing environment for https://github.com/ssbarnea/bashate.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
bashate..................................................................Passed
```
I think I was able to find some extra info, apparently pre-commit does some kind of weird caching which is not updated when you update master. Shortly it was downloading a previous commit which didn't had the .pre-commit-config.yaml file. 

Even manually removing the cached repo from ~/.cache/pre-commit/repoqiSIRh/ did not made pre-commit check-out the last master. Instead it did checkout the same older commit. 

After removing the entire `~/.cache/pre-commit` folder it finally started to checkout the latest commit.
ah, I suspect you're hitting https://pre-commit.com/#using-the-latest-version-for-a-repository ?

either way, this shouldn't stacktrace and should produce a readable error message!
I also suspect this sub-heading should be promoted more in the "creating new hooks" section: https://pre-commit.com/#developing-hooks-interactively
For instance, this is what I expect it to produce:

```
root@3b2c4f333b6a:/t# pre-commit try-repo /bashate
[INFO] Initializing environment for /bashate.
===============================================================================
Using config:
===============================================================================
repos:
-   repo: /bashate
    rev: 0f94116426fe545ea7168ed6696e6e2d59605972
    hooks:
    -   id: bashate
===============================================================================
[INFO] Installing environment for /bashate.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
bashate..................................................................Failed
hookid: bashate

Executable `broken` not found
```

Resolving patch:
diff --git a/pre_commit/parse_shebang.py b/pre_commit/parse_shebang.py
--- a/pre_commit/parse_shebang.py
+++ b/pre_commit/parse_shebang.py
@@ -42,16 +42,21 @@ def find_executable(exe, _environ=None):
         return None
 
 
-def normexe(orig_exe):
-    if os.sep not in orig_exe:
-        exe = find_executable(orig_exe)
+def normexe(orig):
+    def _error(msg):
+        raise ExecutableNotFoundError('Executable `{}` {}'.format(orig, msg))
+
+    if os.sep not in orig and (not os.altsep or os.altsep not in orig):
+        exe = find_executable(orig)
         if exe is None:
-            raise ExecutableNotFoundError(
-                'Executable `{}` not found'.format(orig_exe),
-            )
+            _error('not found')
         return exe
+    elif not os.access(orig, os.X_OK):
+        _error('not found')
+    elif os.path.isdir(orig):
+        _error('is a directory')
     else:
-        return orig_exe
+        return orig
 
 
 def normalize_cmd(cmd):


Test patch:
diff --git a/tests/parse_shebang_test.py b/tests/parse_shebang_test.py
--- a/tests/parse_shebang_test.py
+++ b/tests/parse_shebang_test.py
@@ -85,6 +85,22 @@ def test_normexe_does_not_exist():
     assert excinfo.value.args == ('Executable `i-dont-exist-lol` not found',)
 
 
+def test_normexe_does_not_exist_sep():
+    with pytest.raises(OSError) as excinfo:
+        parse_shebang.normexe('./i-dont-exist-lol')
+    assert excinfo.value.args == ('Executable `./i-dont-exist-lol` not found',)
+
+
+def test_normexe_is_a_directory(tmpdir):
+    with tmpdir.as_cwd():
+        tmpdir.join('exe').ensure_dir()
+        exe = os.path.join('.', 'exe')
+        with pytest.raises(OSError) as excinfo:
+            parse_shebang.normexe(exe)
+        msg, = excinfo.value.args
+        assert msg == 'Executable `{}` is a directory'.format(exe)
+
+
 def test_normexe_already_full_path():
     assert parse_shebang.normexe(sys.executable) == sys.executable
 
@@ -107,14 +123,14 @@ def test_normalize_cmd_PATH():
 
 
 def test_normalize_cmd_shebang(in_tmpdir):
-    python = distutils.spawn.find_executable('python')
-    path = write_executable(python.replace(os.sep, '/'))
+    python = distutils.spawn.find_executable('python').replace(os.sep, '/')
+    path = write_executable(python)
     assert parse_shebang.normalize_cmd((path,)) == (python, path)
 
 
 def test_normalize_cmd_PATH_shebang_full_path(in_tmpdir):
-    python = distutils.spawn.find_executable('python')
-    path = write_executable(python.replace(os.sep, '/'))
+    python = distutils.spawn.find_executable('python').replace(os.sep, '/')
+    path = write_executable(python)
     with bin_on_path():
         ret = parse_shebang.normalize_cmd(('run',))
         assert ret == (python, os.path.abspath(path))