Repository: pre-commit/pre-commit

Issue:
Add statistics information for individual checks (especially time of execution)
I would love to have an option to add some statistics to be displayed for each check. This could be very useful for running pre-commit in CI. I had a case recently of a check that run accidentally a bit longer than expected (it was building unnecessary docker image) and I have not realised that it is taking longer - because we had other checks after and I could only see the total execution time.

Also some other useful information that we might see is how many parallel processes were run and how many files were passed as parameters. This might be really useful especially for people who do not understand that pre-commit runs in parallel by default  - this can have some undesireable side effects if you forget to set "require_serial" to True when you need. And seeing that in the output of CI might immediately show that something is wrong. For now the output is a bit of "black-box".

An option to see some additional information (`--add-statistics`) might be super-useful.

Example output I imagine (maybe a bit better aligned):

```
Check if image build is needed...........................................Passed [23.5s, 1 process, no-files]  
Add licence for all JINJA template files.................................Passed [2.25s, 8 processes, 48 files]
Check Shell scripts syntax correctness...................................Passed [6.00s, 8 processes, 10 files]
Lint dockerfile..........................................................Passed [2.20s, 3 processes, 3 files]
Run mypy.................................................................Passed [12.00s, 8 processes, 1456 files]
Run pylint for main sources.............................................Skipped 
Run pylint for tests....................................................Skipped
Run flake8...............................................................Passed [24.05s, 8 processes, 1456 files]
```


Discussion:
Happy to contribute it if there is an agreement to add it :)
the output is intentionally quiet and I don't want to change that, your proposal also makes things no longer line up

maybe just add this to verbose if at all ?
verbose is .... far too verbose. It's great to see only "passed" but then having some stats, would make it also actionable for cases that are "valid properties" of the check rather than just passed/failed status. Most of the UnitTest frameworks have such feature to display for example the timing. 

And it is perfectly ok to have a separate switch for that and do not have it in the output by default. I think it might be really useful for the CI case I described.
I don't want a combinatoric explosion of options, especially ones I'm not going to use
if it's only for CI who cares how verbose it is?
I think it might be useful for local pre-commits as well, but the main usage scenario for this flag I foresee for CI case. We are currently using it with `--all-files --show-diff-on-failure` and it works great (minimum output + we see exact diffs when diff-failure happens). 

In this case I only found out that we have a problem because the build steps were running for a really long time and failed (but I would immediately see the problem if I had the stats).
In my particular case we have the "build needed check" (see the #1143 ) and when it does not need to rebuild the image it will be "passed" in ms. But if (happened in my case) due to bug in build need  detection it took maybe 2-3 minutes. - I would immediately see this with statistics but in case of our build the only thing i could see is "passed" without knowing how much time this particular check took.

BTW. I have found that having CI build for pre-commit is great promotor of pre-commit in general - because when something fails, installing pre-commit and running that failed check is simplest way to do (and pre-commit nicely suggests that action). But CI is also good to observe if other build properties are met.

I don't want another option, let's add it to verbose output
I think just having "Passed in x.xs" by default (instead of just "Passed") would be useful to have by default, which could easily be aligned (assuming that nothing takes 100 seconds).
(Not sure about the extra stats (processes / files), likely they should go into "verbose" mode - never used that, it is too verbose by default already for me (77aab5e))
(came here via https://github.com/pre-commit/pre-commit/issues/1227)
please read the discussion -- I've already said no to that
@asottile 
As far as I can see you've said "no" to "Passed [23.5s, 1 process, no-files]" only.
Anyway, just my 2c - you've duped it after all.
please stay on topic, thanks
> please stay on topic, thanks

Please don't delete my comments, thanks.  This really makes me stay away from you / this repo now.

FWIW: You've a) duped a more specific issue here, and then b) deleted a comment from me speciying how this (durations) interacts with display (verbose or not) in general.  :shrug:

You've also not answered my correction about me telling you that your "please read the discussion -- I've already said no to that" comment was not fitting.
That would have been helpful with keeping this on-topic (since you've duped it here in the first place).
:shrug:


Resolving patch:
diff --git a/pre_commit/commands/run.py b/pre_commit/commands/run.py
--- a/pre_commit/commands/run.py
+++ b/pre_commit/commands/run.py
@@ -4,6 +4,7 @@
 import os
 import re
 import subprocess
+import time
 
 from identify.identify import tags_from_path
 
@@ -99,6 +100,7 @@ def _run_single_hook(classifier, hook, skips, cols, verbose, use_color):
                 cols=cols,
             ),
         )
+        duration = None
         retcode = 0
         files_modified = False
         out = b''
@@ -113,6 +115,7 @@ def _run_single_hook(classifier, hook, skips, cols, verbose, use_color):
                 cols=cols,
             ),
         )
+        duration = None
         retcode = 0
         files_modified = False
         out = b''
@@ -123,7 +126,9 @@ def _run_single_hook(classifier, hook, skips, cols, verbose, use_color):
         diff_cmd = ('git', 'diff', '--no-ext-diff')
         diff_before = cmd_output_b(*diff_cmd, retcode=None)
         filenames = tuple(filenames) if hook.pass_filenames else ()
+        time_before = time.time()
         retcode, out = hook.run(filenames, use_color)
+        duration = round(time.time() - time_before, 2) or 0
         diff_after = cmd_output_b(*diff_cmd, retcode=None)
 
         # if the hook makes changes, fail the commit
@@ -141,6 +146,9 @@ def _run_single_hook(classifier, hook, skips, cols, verbose, use_color):
     if verbose or hook.verbose or retcode or files_modified:
         _subtle_line('- hook id: {}'.format(hook.id), use_color)
 
+        if (verbose or hook.verbose) and duration is not None:
+            _subtle_line('- duration: {}s'.format(duration), use_color)
+
         if retcode:
             _subtle_line('- exit code: {}'.format(retcode), use_color)
 


Test patch:
diff --git a/testing/resources/arbitrary_bytes_repo/.pre-commit-hooks.yaml b/testing/resources/arbitrary_bytes_repo/.pre-commit-hooks.yaml
--- a/testing/resources/arbitrary_bytes_repo/.pre-commit-hooks.yaml
+++ b/testing/resources/arbitrary_bytes_repo/.pre-commit-hooks.yaml
@@ -1,6 +1,5 @@
--   id: python3-hook
-    name: Python 3 Hook
-    entry: python3-hook
-    language: python
-    language_version: python3
+-   id: hook
+    name: hook
+    entry: ./hook.sh
+    language: script
     files: \.py$
diff --git a/testing/resources/arbitrary_bytes_repo/hook.sh b/testing/resources/arbitrary_bytes_repo/hook.sh
new file mode 100755
--- /dev/null
+++ b/testing/resources/arbitrary_bytes_repo/hook.sh
@@ -0,0 +1,7 @@
+#!/usr/bin/env bash
+# Intentionally write mixed encoding to the output.  This should not crash
+# pre-commit and should write bytes to the output.
+# '☃'.encode('UTF-8') + '²'.encode('latin1')
+echo -e '\xe2\x98\x83\xb2'
+# exit 1 to trigger printing
+exit 1
diff --git a/testing/resources/arbitrary_bytes_repo/python3_hook.py b/testing/resources/arbitrary_bytes_repo/python3_hook.py
deleted file mode 100644
--- a/testing/resources/arbitrary_bytes_repo/python3_hook.py
+++ /dev/null
@@ -1,13 +0,0 @@
-# -*- coding: utf-8 -*-
-from __future__ import print_function
-from __future__ import unicode_literals
-
-import sys
-
-
-def main():
-    # Intentionally write mixed encoding to the output.  This should not crash
-    # pre-commit and should write bytes to the output.
-    sys.stdout.buffer.write('☃'.encode('UTF-8') + '²'.encode('latin1') + b'\n')
-    # Return 1 to trigger printing
-    return 1
diff --git a/testing/resources/arbitrary_bytes_repo/setup.py b/testing/resources/arbitrary_bytes_repo/setup.py
deleted file mode 100644
--- a/testing/resources/arbitrary_bytes_repo/setup.py
+++ /dev/null
@@ -1,8 +0,0 @@
-from setuptools import setup
-
-setup(
-    name='python3_hook',
-    version='0.0.0',
-    py_modules=['python3_hook'],
-    entry_points={'console_scripts': ['python3-hook=python3_hook:main']},
-)
diff --git a/tests/commands/run_test.py b/tests/commands/run_test.py
--- a/tests/commands/run_test.py
+++ b/tests/commands/run_test.py
@@ -5,6 +5,7 @@
 import os.path
 import pipes
 import sys
+import time
 
 import mock
 import pytest
@@ -25,6 +26,7 @@
 from testing.fixtures import modify_config
 from testing.fixtures import read_config
 from testing.fixtures import sample_meta_config
+from testing.fixtures import write_config
 from testing.util import cmd_output_mocked_pre_commit_home
 from testing.util import cwd
 from testing.util import git_commit
@@ -163,36 +165,55 @@ def test_exclude_types_hook_repository(cap_out, store, tempdir_factory):
         assert b'exe' not in printed
 
 
-def test_global_exclude(cap_out, store, tempdir_factory):
-    git_path = make_consuming_repo(tempdir_factory, 'script_hooks_repo')
-    with cwd(git_path):
-        with modify_config() as config:
-            config['exclude'] = '^foo.py$'
-        open('foo.py', 'a').close()
-        open('bar.py', 'a').close()
-        cmd_output('git', 'add', '.')
-        opts = run_opts(verbose=True)
-        ret, printed = _do_run(cap_out, store, git_path, opts)
-        assert ret == 0
-        # Does not contain foo.py since it was excluded
-        expected = b'- hook id: bash_hook\n\nbar.py\nHello World\n\n'
-        assert printed.endswith(expected)
+def test_global_exclude(cap_out, store, in_git_dir):
+    config = {
+        'exclude': r'^foo\.py$',
+        'repos': [{'repo': 'meta', 'hooks': [{'id': 'identity'}]}],
+    }
+    write_config('.', config)
+    open('foo.py', 'a').close()
+    open('bar.py', 'a').close()
+    cmd_output('git', 'add', '.')
+    opts = run_opts(verbose=True)
+    ret, printed = _do_run(cap_out, store, str(in_git_dir), opts)
+    assert ret == 0
+    # Does not contain foo.py since it was excluded
+    assert printed.startswith(b'identity' + b'.' * 65 + b'Passed\n')
+    assert printed.endswith(b'\n\n.pre-commit-config.yaml\nbar.py\n\n')
 
 
-def test_global_files(cap_out, store, tempdir_factory):
-    git_path = make_consuming_repo(tempdir_factory, 'script_hooks_repo')
-    with cwd(git_path):
-        with modify_config() as config:
-            config['files'] = '^bar.py$'
-        open('foo.py', 'a').close()
-        open('bar.py', 'a').close()
-        cmd_output('git', 'add', '.')
-        opts = run_opts(verbose=True)
-        ret, printed = _do_run(cap_out, store, git_path, opts)
-        assert ret == 0
-        # Does not contain foo.py since it was not included
-        expected = b'- hook id: bash_hook\n\nbar.py\nHello World\n\n'
-        assert printed.endswith(expected)
+def test_global_files(cap_out, store, in_git_dir):
+    config = {
+        'files': r'^bar\.py$',
+        'repos': [{'repo': 'meta', 'hooks': [{'id': 'identity'}]}],
+    }
+    write_config('.', config)
+    open('foo.py', 'a').close()
+    open('bar.py', 'a').close()
+    cmd_output('git', 'add', '.')
+    opts = run_opts(verbose=True)
+    ret, printed = _do_run(cap_out, store, str(in_git_dir), opts)
+    assert ret == 0
+    # Does not contain foo.py since it was excluded
+    assert printed.startswith(b'identity' + b'.' * 65 + b'Passed\n')
+    assert printed.endswith(b'\n\nbar.py\n\n')
+
+
+@pytest.mark.parametrize(
+    ('t1', 't2', 'expected'),
+    (
+        (1.234, 2., b'\n- duration: 0.77s\n'),
+        (1., 1., b'\n- duration: 0s\n'),
+    ),
+)
+def test_verbose_duration(cap_out, store, in_git_dir, t1, t2, expected):
+    write_config('.', {'repo': 'meta', 'hooks': [{'id': 'identity'}]})
+    cmd_output('git', 'add', '.')
+    opts = run_opts(verbose=True)
+    with mock.patch.object(time, 'time', side_effect=(t1, t2)):
+        ret, printed = _do_run(cap_out, store, str(in_git_dir), opts)
+    assert ret == 0
+    assert expected in printed
 
 
 @pytest.mark.parametrize(
diff --git a/tests/commands/try_repo_test.py b/tests/commands/try_repo_test.py
--- a/tests/commands/try_repo_test.py
+++ b/tests/commands/try_repo_test.py
@@ -3,6 +3,9 @@
 
 import os.path
 import re
+import time
+
+import mock
 
 from pre_commit import git
 from pre_commit.commands.try_repo import try_repo
@@ -40,7 +43,8 @@ def _run_try_repo(tempdir_factory, **kwargs):
 
 
 def test_try_repo_repo_only(cap_out, tempdir_factory):
-    _run_try_repo(tempdir_factory, verbose=True)
+    with mock.patch.object(time, 'time', return_value=0.0):
+        _run_try_repo(tempdir_factory, verbose=True)
     start, config, rest = _get_out(cap_out)
     assert start == ''
     assert re.match(
@@ -58,6 +62,7 @@ def test_try_repo_repo_only(cap_out, tempdir_factory):
 - hook id: bash_hook
 Bash hook................................................................Passed
 - hook id: bash_hook2
+- duration: 0s
 
 test-file