Repository: pypa/pip

Issue:
--force-reinstall assumes default git branch is 'master'
* Pip version: 9.0.1 (reproduced on 8.0.2)
* Python version: 3.5 (reproduced on 2.7.11)
* Operating system: OS X (reproduced on Ubuntu)

* Git version: 2.11.0 (Apple Git-81) (reproduced on 1.9.1 non-Apple)

### Description:

**Installing from git via a requirements file with and without `--force-reinstall` results in different behavior with regard to the default git branch. If the branch is specified in the requirements file, that avoids this problem.** The problem may be of somewhat larger scope: I haven't tried without a requirements file, for example.

If the git branch is not specified, `--force-reinstall` from a requirements file with a dependency coming from git seems to assume that the branch should be 'master', even when the default branch of the git repo is not. By contrast, normal install (no `--force-reinstall`) correctly grabs the default branch (in this case, 'develop').

From our side, the problem is solved by specifying the branch in the requirements file. This could trip someone up, though: when I add a 'master' branch to the depended-on project, `--force-reinstall` does install from that branch, which can lead to the wrong code being installed.


### What I've run:

Example in a Virtual Environment using Python2.7:

``` shell
$ source v2/bin/activate
$ pip install -r dev-requirements.txt
<...>
$ cd v2/src/tuf
$ git status
On branch develop
Your branch is up-to-date with 'origin/develop'.
$ cd ../../..
$ pip uninstall tuf
<...>
$ pip install --force-reinstall -r dev-requirements.txt
<...>
Obtaining tuf from git+git://github.com/awwad/tuf.git#egg=tuf (from -r dev-requirements.txt (line 7))
  Updating ./v2/src/tuf clone
  Could not find a tag or branch 'origin/master', assuming commit.
fatal: ambiguous argument 'origin/master': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Command "git reset --hard -q origin/master" failed with error code 128 in /Users/s/w/uptane_new/v2/src/tuf
```



If I add a 'master' branch to the repo somewhere, say, far behind the develop branch, I get this behavior:

```shell
$ source venv/bin/activate
$ pip install --force-reinstall -r dev-requirements.txt
<...>
$ cd venv/src/tuf
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
```

```shell
$ source venv/bin/activate
$ pip install -r dev-requirements.txt
<...>
$ cd venv/src/tuf
$ git status
On branch develop
Your branch is up-to-date with 'origin/develop'.
```
pip install -e resets to origin/master
If I do this:

```
pip install -e git://github.com/nimbis/django.git#egg=Django-dev
```

Then pip will grab the default branch from that repository (`1.4.1-patched`) and install it, as you can see in the src directory where it's installed:

```
cd ~/.virtualenvs/nimbis/src/django
[(1.4.1-patched) ~/.virtualenvs/nimbis/src/django]
$ git log | head -n4
commit a5d7c7b3a2d6a729bf5ede2254b5c75be9da94d7
Author: Lorin Hochstein <...>
Date:   Mon Jul 30 21:44:20 2012 -0400
```

If I do it again, the HEAD becomes the same as the remote master branch, although it still uses 1.4.1-patched (default branch) as the branch name.

```
$ pip install -e git://github.com/nimbis/django.git#egg=Django-dev
[(1.4.1-patched) lorin@nibbler ~/.virtualenvs/nimbis/src/django]
$ git log | head -n4
commit e567f439bdfc60e16c465220fdaa8ea8a0fae936
Merge: c0748a6 226a3e7
Author: Alex Gaynor <...>
Date:   Sun Jul 29 17:07:55 2012 -0700
```



Discussion:

which version of pip?

pip 1.1

yes, I see exactly what you describe.  sure looks like a bug.

but know that you can use the '@' parameter to reliably get the `1.4.1-patched` branch on repeated installs.
`pip install -e git://github.com/nimbis/django.git@1.4.1-patched#egg=Django-dev`

Yeah, that's what I'm doing now.

FWIW, I added a pdb.set_trace() to req.py update_editable() (line 380), and traced this down to the point where vcs/git.py(110)obtain() does `if self.check_destination(dest, url, rev_options, rev_display):`.  Upon the second run of install, the values of dest, url, rev_options and rev_display are `('/home/chrism/projects/pip/testenv/issue647/src/django', 'git://github.com/nimbis/django.git', ['origin/master'], '')`. `rev_options` is `['origin/master']` because there is no revision in the URL and that's its hardcoded default in the no-url-revision case. `check_destination` isn't quite as innocuous as its name; it winds up calling `self.update(dest, rev_options)` where `rev_options` is `origin/master`.  I don't quite have a theory yet about how to fix it, but that's where the bug is.

Currently pip is inconsistent because it checks out the HEAD symbolic ref (aka the "default branch") when not instructed to do otherwise, but updates the master (not the default branch) when not instructed to do otherwise.  Its checkout vs. update strategies are inconsistent.

There are two competing actions that could be taken to make pip consistent here:
- pip could always use the default branch (aka "HEAD symbolic ref", see http://feeding.cloud.geek.nz/2008/12/setting-default-git-branch-in-bare.html) of the remote and when
  updating a local checkout
  from the remote, it should fetch from the same default branch.
- pip should always use the master of the remote when first checking out the branch, and if a branch
  or other rev is not specified, it would then assume it needs to update from master.

The former is harder than the latter.  I don't know any way to get the HEAD symbolic ref from a remote repository without actually just re-checking-out the entire repository and seeing what its initial branch name is.  The latter is easier, but it might be confusing for people that use gitflow or some other model where the default branch is not master.

If there's a way to get the HEAD symbolic ref from a remote repo, we can easily issue a command to get it and use it.  @lorin, do you know how to do that by any chance?  If not, we'll likely either need to both check out and update master when otherwise unspecified, or close this as a wontfix.

@mcdonc I think you can use "git remote show" to do this, and it will tell you what the remote head branch is:

For example:

```
$ git remote show origin | head -n4
* remote origin
  Fetch URL: git@github.com:nimbis/django.git
  Push  URL: git@github.com:nimbis/django.git
  HEAD branch: 1.4.1-patched
```

Seems, that this bug is still there.

I mean, it is still resets to the given branch or revision even if there is uncommited changes. This is very bad behaviour — yesterday I lost few hours of work this way. 

I think we have this problem, too.

The checkout of "pip install -e git+https:..." is broken.

I looked at the source  vcs/git.py, and I guess this comment says it all:

```
    # refs maps names to commit hashes; we need the inverse
    # if multiple names map to a single commit, this arbitrarily picks one
```

I don't know why pip does a lookup of GIT-HASH-ID --> REV if I do install a git branch.

One GIT-HASH-ID can be referenced in several branches.

If two branches refer to the same hash-id, the guess is a guess and not deterministic.

---

What can I do to fix this? At the moment I don't get what pip does at all.

To set up development environments, we use a pip requirements file with lines like:

`-e git+ssh://host/repository.git#egg=repository`

There are a bunch of repositories that go into the environment. In development I'll use manual git commands to create branches in the particular repositories I'm working on. Every now and then, it's useful to re-run the pip install command that set up the dev environment (to pick up changes like new scripts installed to bin, deleted files, or new dependencies; things that aren't always automatically handled just by changing the files inside an editable-installed package).

If I have any non-master branches still checked out, pip hard resets those to origin/master; this results in losing uncommitted edits as @svetlyak40wt noted, but also any _committed_ changes on the branch that aren't included on another branch (as well clobbering the branch pointer itself).

The way I would have expected pip to behave is one of these three:
1. Error out complaining about the wrong branch being checked out
2. Fetch the remote version of the locally checked out branch, and reset the local branch to that
3. Check out local master, and reset that to the newly fetched origin/master


Resolving patch:
diff --git a/src/pip/_internal/vcs/git.py b/src/pip/_internal/vcs/git.py
--- a/src/pip/_internal/vcs/git.py
+++ b/src/pip/_internal/vcs/git.py
@@ -132,7 +132,7 @@ def obtain(self, dest):
             rev_options = [rev]
             rev_display = ' (to %s)' % rev
         else:
-            rev_options = ['origin/master']
+            rev_options = ['origin/HEAD']
             rev_display = ''
         if self.check_destination(dest, url, rev_options, rev_display):
             logger.info(


Test patch:
diff --git a/tests/functional/test_install_vcs.py b/tests/functional/test_install_vcs.py
--- a/tests/functional/test_install_vcs.py
+++ b/tests/functional/test_install_vcs.py
@@ -253,3 +253,32 @@ def test_git_works_with_editable_non_origin_repo(script):
     assert "Error when trying to get requirement" in result.stderr
     assert "Could not determine repository location" in result.stdout
     assert "version-pkg==0.1" in result.stdout
+
+
+@pytest.mark.network
+def test_reinstalling_works_with_editible_non_master_branch(script):
+    """
+    Reinstalling an editable installation should not assume that the "master"
+    branch exists. See https://github.com/pypa/pip/issues/4448.
+    """
+    version_pkg_path = _create_test_package(script)
+
+    # Switch the default branch to something other than 'master'
+    script.run('git', 'branch', '-m', 'foobar', cwd=version_pkg_path)
+
+    script.pip(
+        'install', '-e',
+        '%s#egg=version_pkg' %
+        ('git+file://' + version_pkg_path.abspath.replace('\\', '/')),
+    )
+    version = script.run('version_pkg')
+    assert '0.1' in version.stdout
+
+    _change_test_package_version(script, version_pkg_path)
+    script.pip(
+        'install', '-e',
+        '%s#egg=version_pkg' %
+        ('git+file://' + version_pkg_path.abspath.replace('\\', '/')),
+    )
+    version = script.run('version_pkg')
+    assert 'some different version' in version.stdout