Repository: pre-commit/pre-commit

Issue:
Stashed changes lost if hook fails
I've run into this particular (in my eyes, critical) bug.
If I want to do a partial commit, e.g. I have 2 files but I only add 1 file to the staging area and the staged file will cause a hook to fail, I loose the changes in the 2nd file because pre-commit fails to reroll the patch it stashed before running.

Here's my terminal log and the steps to reproduce:
## Version

$ pre-commit -V
pre-commit 0.3.0
## Commands to reproduce

```
$ cat unstaged.py
"""I am unstaged"""
$ echo "'''I am unstaged, but I have changes'''" > unstaged.py
$ echo "x = 'This is the loooooooooooooooooooooooooooooooooooongest liiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiine eveeeeeeeeer'" > foo.py 
$ git status
On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   unstaged.py
    modified:   foo.py

no changes added to commit (use "git add" and/or "git commit -a")
$ git add foo.py 
$ git commit -m "Adding a long line"
[WARNING] Unstaged files detected.
[INFO] Stashing unstaged files to /home/k/.pre-commit/patch1412683352.
Flake8...............................................Failed
hookid: flake8

foo.py:1:80: E501 line too long (112 > 79 characters)

[WARNING] Stashed changes conflicted with hook auto-fixes... Rolling back fixes...
An unexpected error has occurred: CalledProcessError: Command: [u'git', u'apply', u'/home/k/.pre-commit/patch1412683352']
Return code: 128
Expected return code: 0
Output: (u'', u'fatal: unrecognized input\n')

Check the log at ~/.pre-commit/pre-commit.log
$ git status
On branch master
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    modified:   foo.py

$ echo "x = 'This is a shorter line, its better'" > foo.py
$ git status
On branch master
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    modified:   foo.py

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   foo.py

$ git add foo.py 
$ git commit -m "Fixed the error I got from the flake8 hook"
Flake8...............................................Passed
[master 78568e8] Fixed the error I got from the flake8 hook
 1 file changed, 1 insertion(+), 1 deletion(-)
$ git status
On branch master
nothing to commit, working directory clean
$ cat unstaged.py
"""I am unstaged"""
```
## Log

```
$ cat ~/.pre-commit/pre-commit.log
Traceback (most recent call last):
  File "/home/k/.virtualenvs/pre-commit-test/local/lib/python2.7/site-packages/pre_commit/error_handler.py", line 34, in error_handler
    yield
  File "/home/k/.virtualenvs/pre-commit-test/local/lib/python2.7/site-packages/pre_commit/main.py", line 108, in main
    return run(runner, args)
  File "/home/k/.virtualenvs/pre-commit-test/local/lib/python2.7/site-packages/pre_commit/commands/run.py", line 151, in run
    return _run_hooks(runner, args, write=write, environ=environ)
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/home/k/.virtualenvs/pre-commit-test/local/lib/python2.7/site-packages/pre_commit/staged_files_only.py", line 55, in staged_files_only
    cmd_runner.run(['git', 'apply', patch_filename])
  File "/home/k/.virtualenvs/pre-commit-test/local/lib/python2.7/site-packages/pre_commit/prefixed_command_runner.py", line 82, in run
    returncode, replaced_cmd, retcode, output=(stdout, stderr),
CalledProcessError: Command: [u'git', u'apply', u'/home/k/.pre-commit/patch1412683352']
Return code: 128
Expected return code: 0
Output: (u'', u'fatal: unrecognized input\n')
```
## .pre-commit-config.yaml

```
$ cat .pre-commit-config.yaml 
- repo: git@github.com:pre-commit/pre-commit-hooks
  sha: 6343700aa063fe30acc319d2dc84353a35a3d6d0
  hooks:
  - id: flake8
    args: ['--ignore=E712,F821']
```



Discussion:
Could you additionally provide the version of `git` you are using?  I believe it may be contributing to this problem.

I wonder if `git-apply` works significantly differently in a different version of git.  I may need to work around that to fix this bug.

For what it's worth, it seems to work as intended with git 1.7.9.5:

```
$ git status
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#   modified:   unstaged.py
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#   venv/
no changes added to commit (use "git add" and/or "git commit -a")
$ echo "x = 'This is the loooooooooooooooooooooooooooooooooooongest liiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiine eveeeeeeeeer'" > foo.py
$ git status
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#   modified:   foo.py
#   modified:   unstaged.py
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#   venv/
no changes added to commit (use "git add" and/or "git commit -a")
$ git add foo.py
$ git commit -m "Add a long line"
[WARNING] Unstaged files detected.
[INFO] Stashing unstaged files to /home/anthony/.pre-commit/patch1412701193.
Flake8...................................................................Failed
hookid: flake8

foo.py:1:80: E501 line too long (112 > 79 characters)

[INFO] Restored changes from /home/anthony/.pre-commit/patch1412701193.
```

Additionally, the contents of `/home/k/.pre-commit/patch1412683352` will be useful to diagnose

I'm using git version 1.9.1
Here are the contents of the patch file

```
$ cat /home/k/.pre-commit/patch1412683352
diff --git a/file.py b/file.py
index f99a4d5..a8c4aad 100644
--- a/file.py
+++ b/file.py
@@ -1 +1 @@
-"""I am unstaged"""
+'''I am unstaged, but I have changes'''
```

This is a set of commands similar to what pre-commit runs.  Could you run them and show me what the output is?

`test.sh`

```
#!/usr/bin/env bash
function printcmd() { echo '$$ '"$@"; "$@"; echo "## retcode $?"; }
printcmd git init test
printcmd cd test
printcmd sh -c 'echo '"'"'"""Docstring"""'"'"' > unstaged.py'
printcmd touch foo.py
printcmd git add unstaged.py foo.py
printcmd git commit -m "initial commit"
printcmd sh -c "echo "'"'"'''I am unstaged but I have changes'''"'"'" > unstaged.py"
printcmd sh -c "echo 'x = "'"'"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"'"'"' > foo.py"
printcmd git status
printcmd git add foo.py
# This mimics what pre-commit does in pre_commit/staged_files_only.py
printcmd sh -c "git diff --ignore-submodules --binary --exit-code > patch"
printcmd cat patch
printcmd git checkout -- .
printcmd git status
printcmd git apply patch
```

Output on my machine:

```
$ ./test.sh
$$ git init test
Initialized empty Git repository in /tmp/test/.git/
## retcode 0
$$ cd test
## retcode 0
$$ sh -c echo '"""Docstring"""' > unstaged.py
## retcode 0
$$ touch foo.py
## retcode 0
$$ git add unstaged.py foo.py
## retcode 0
$$ git commit -m initial commit
[master (root-commit) 446eafd] initial commit
 1 file changed, 1 insertion(+)
 create mode 100644 foo.py
 create mode 100644 unstaged.py
## retcode 0
$$ sh -c echo "'''I am unstaged but I have changes'''" > unstaged.py
## retcode 0
$$ sh -c echo 'x = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"' > foo.py
## retcode 0
$$ git status
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#   modified:   foo.py
#   modified:   unstaged.py
#
no changes added to commit (use "git add" and/or "git commit -a")
## retcode 0
$$ git add foo.py
## retcode 0
$$ sh -c git diff --ignore-submodules --binary --exit-code > patch
## retcode 1
$$ cat patch
diff --git a/unstaged.py b/unstaged.py
index c83d2e0..cbb1889 100644
--- a/unstaged.py
+++ b/unstaged.py
@@ -1 +1 @@
-"""Docstring"""
+'''I am unstaged but I have changes'''
## retcode 0
$$ git checkout -- .
## retcode 0
$$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#   modified:   foo.py
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#   patch
## retcode 0
$$ git apply patch
## retcode 0
```

Actually, I think it might be being affected by `.gitconfig`, could you paste the contents of `~/.gitconfig`?

Here's the output of the command:

```
$ . test.sh
$$ git init test
Initialized empty Git repository in /home/k/testtest/test/.git/
## retcode 0
$$ cd test
## retcode 0
$$ sh -c echo '"""Docstring"""' > unstaged.py
## retcode 0
$$ touch foo.py
## retcode 0
$$ git add unstaged.py foo.py
## retcode 0
$$ git commit -m initial commit
[master (root-commit) aa743a4] initial commit
 2 files changed, 1 insertion(+)
 create mode 100644 foo.py
 create mode 100644 unstaged.py
## retcode 0
$$ sh -c echo "'''I am unstaged but I have changes'''" > unstaged.py
## retcode 0
$$ sh -c echo 'x = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"' > foo.py
## retcode 0
$$ git status
On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   foo.py
    modified:   unstaged.py

no changes added to commit (use "git add" and/or "git commit -a")
## retcode 0
$$ git add foo.py
## retcode 0
$$ sh -c git diff --ignore-submodules --binary --exit-code > patch
## retcode 1
$$ cat patch
diff --git a/unstaged.py b/unstaged.py
index c83d2e0..cbb1889 100644
--- a/unstaged.py
+++ b/unstaged.py
@@ -1 +1 @@
-"""Docstring"""
+'''I am unstaged but I have changes'''
## retcode 0
$$ git checkout -- .
## retcode 0
$$ git status
On branch master
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    modified:   foo.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    patch

## retcode 0
$$ git apply patch
fatal: unrecognized input
## retcode 128
```

and here's `~/.gitconfig`

```
[user]
    name = Kasper Jacobsen
    email = k@mackwerk.dk

[color]
    branch = always
    diff = always
    grep = always
    interactive = always
    pager = true
    showbranch = always
    status = always
    ui = always

[apply]
    whitespace = fix

[branch]
    autosetupmerge =  true
    #autosetuprebase = always

[merge]
    # Include summaries of merged commits in newly created merge commit messages
    log = true
    tool = sublime

[mergetool "sublime"]
    cmd = subl -w $MERGED
    trustExitCode = false

# Use `origin` as the default remote on the `master` branch in all cases
[branch "master"]
    remote = origin
    merge = refs/heads/master

[push]
    default = tracking

[fetch]
    recurseSubmodules = true

[log]
    decorate = short

[pretty]
    custom = %C(yellow)%h%Creset -%C(green)%d%Creset %s %Cgreen(%cr) %C(bold blue)%an%Creset

[alias]
    dc = diff --cached
    df = diff --word-diff
    co = checkout
    cp = cherry-pick
    st = status
    br = branch -a
    lg = log --graph --pretty=medium --abbrev-commit --date=local
    lgb = log --graph --pretty=medium --abbrev-commit --date=local --branches --all
    lol = log --graph --oneline
    lola = log --graph --oneline --all
    l = log --graph --pretty=custom
    la = log --graph --pretty=custom --all

    # Show files ignored by git:
    ign = ls-files -o -i --exclude-standard

    # Show verbose output about tags, branches or remotes
    tags = tag -l
    branches = branch -a
    remotes = remote -v

    onto = !"git co -b __ONTOTMP; git co $1; git merge __ONTOTMP; git branch -d __ONTOTMP"
    today = log --since=midnight --author='Kasper Jacobsen' --oneline

    # exclude files locally
    exclude = !sh -c 'echo "$1" >> .git/info/exclude' -


# URL shorthands
[url "git@github.com:"]
    insteadOf = "gh:"
    pushInsteadOf = "github:"
    pushInsteadOf = "git://github.com/"
[url "git://github.com/"]
    insteadOf = "github:"
[url "git@gist.github.com:"]
    insteadOf = "gst:"
    pushInsteadOf = "gist:"
    pushInsteadOf = "git://gist.github.com/"
[url "git://gist.github.com/"]
    insteadOf = "gist:"
[credential]
    helper = cache
```


Resolving patch:
diff --git a/pre_commit/staged_files_only.py b/pre_commit/staged_files_only.py
--- a/pre_commit/staged_files_only.py
+++ b/pre_commit/staged_files_only.py
@@ -21,7 +21,10 @@ def staged_files_only(cmd_runner):
     """
     # Determine if there are unstaged files
     retcode, diff_stdout_binary, _ = cmd_runner.run(
-        ['git', 'diff', '--ignore-submodules', '--binary', '--exit-code'],
+        [
+            'git', 'diff', '--ignore-submodules', '--binary', '--exit-code',
+            '--no-color',
+        ],
         retcode=None,
         encoding=None,
     )


Test patch:
diff --git a/tests/staged_files_only_test.py b/tests/staged_files_only_test.py
--- a/tests/staged_files_only_test.py
+++ b/tests/staged_files_only_test.py
@@ -70,6 +70,11 @@ def test_foo_something_unstaged(foo_staged, cmd_runner):
     _test_foo_state(foo_staged, 'herp\nderp\n', 'AM')
 
 
+def test_foo_something_unstaged_diff_color_always(foo_staged, cmd_runner):
+    cmd_output('git', 'config', '--local', 'color.diff', 'always')
+    test_foo_something_unstaged(foo_staged, cmd_runner)
+
+
 def test_foo_both_modify_non_conflicting(foo_staged, cmd_runner):
     with io.open(foo_staged.foo_filename, 'w') as foo_file:
         foo_file.write(FOO_CONTENTS + '9\n')