Repository: pypa/pip

Issue:
pip >=19.2 cannot install packages via custom URLs with "one element" auth
**Environment**

* pip version: >= 19.2
* Python version: 2.7.10 and 3.7.2
* OS: macOS Mojave

<!-- Feel free to add more information about your environment here -->

**Description**
<!-- A clear and concise description of what the bug is. -->
When trying to install a package from a custom URL that requires authentication (e.g. from GitHub private repo releases), pip doesn't properly handle parsing of "one element" auth info e.g. GitHub access tokens (i.e. without `:` like the usual `user:password` auth info).

**Expected behavior**
<!-- A clear and concise description of what you expected to happen. -->
I want to be able to install my package from a private GitHub repo with an Auth Token like this:
```
pip install https://myaccesstoken@github.com/myorganization/myprivaterepo/archive/myrelease.zip#egg=mypackage
```
this works fine with `pip==19.1.1`

**How to Reproduce**
<!-- Describe the steps to reproduce this bug. -->

1. Create a private repo on GitHub with one release (can be empty).
2. Create an access token at https://github.com/settings/tokens/new for private repos
3. Try to install that release package (.zip) using the created token with `pip>=19.2` like in the section above.

The output should mention a successful *download* of the package and a "`setup.py` not found" error message (if the release was empty).

**Workaround**

As a workaround one can put `:` before or after the access token so the URL looks like:
```
https://:myaccesstoken@github.com/...
```

**Output**
Instead of installing the package, pip fails with "ERROR: HTTP error 404 while getting https://...".

I've created an empty private repo, with an empty release to test that. The output for `pip install -vvv ...` can be found here:
- [for pip 19.2](http://dpaste.com/2XGKP51),
- [for pip 19.1.1](http://dpaste.com/1EZ7JQW) (with expected `setup.py` not found error).

The issue/difference can be observed around `Starting new HTTPS connection (1): github.com:443`.

**Cause**
I've run a `git bisect` session for this issue and it pointed to this commit c63ee61027fe

This issue was also discussed at https://github.com/pypa/pip/issues/6775 (see https://github.com/pypa/pip/issues/6775#issuecomment-514284653 and https://github.com/pypa/pip/issues/6775#issuecomment-514299653).

WIP: Fix - Cannot access private repository using token
This PR addresses a potential bug from 19.2 as discussed [here](https://github.com/pypa/pip/issues/6775#issuecomment-514278956). 

Embedding a token in an `index_url` is not working. 

For example:

`pip install SomeProject --index-url="https://TOKEN@some.where/somerepo/"`

throws an exception: `pip._internal.exceptions.DistributionNotFound` 
 
  _Any guidance/feedback would be appreciated!_



Discussion:

Resolving patch:
diff --git a/src/pip/_internal/download.py b/src/pip/_internal/download.py
--- a/src/pip/_internal/download.py
+++ b/src/pip/_internal/download.py
@@ -307,7 +307,7 @@ def _get_new_credentials(self, original_url, allow_netrc=True,
                 logger.debug("Found credentials in keyring for %s", netloc)
                 return kr_auth
 
-        return None, None
+        return username, password
 
     def _get_url_and_credentials(self, original_url):
         """Return the credentials to use for the provided URL.
@@ -324,15 +324,29 @@ def _get_url_and_credentials(self, original_url):
         # Use any stored credentials that we have for this netloc
         username, password = self.passwords.get(netloc, (None, None))
 
-        # If nothing cached, acquire new credentials without prompting
-        # the user (e.g. from netrc, keyring, or similar).
-        if username is None or password is None:
+        if username is None and password is None:
+            # No stored credentials. Acquire new credentials without prompting
+            # the user. (e.g. from netrc, keyring, or the URL itself)
             username, password = self._get_new_credentials(original_url)
 
-        if username is not None and password is not None:
-            # Store the username and password
+        if username is not None or password is not None:
+            # Convert the username and password if they're None, so that
+            # this netloc will show up as "cached" in the conditional above.
+            # Further, HTTPBasicAuth doesn't accept None, so it makes sense to
+            # cache the value that is going to be used.
+            username = username or ""
+            password = password or ""
+
+            # Store any acquired credentials.
             self.passwords[netloc] = (username, password)
 
+        assert (
+            # Credentials were found
+            (username is not None and password is not None) or
+            # Credentials were not found
+            (username is None and password is None)
+        ), "Could not load credentials from url: {}".format(original_url)
+
         return url, username, password
 
     def __call__(self, req):


Test patch:
diff --git a/tests/unit/test_download.py b/tests/unit/test_download.py
--- a/tests/unit/test_download.py
+++ b/tests/unit/test_download.py
@@ -490,18 +490,53 @@ def test_insecure_host_cache_is_not_enabled(self, tmpdir):
         assert not hasattr(session.adapters["https://example.com/"], "cache")
 
 
-def test_get_credentials():
+@pytest.mark.parametrize(["input_url", "url", "username", "password"], [
+    (
+        "http://user%40email.com:password@example.com/path",
+        "http://example.com/path",
+        "user@email.com",
+        "password",
+    ),
+    (
+        "http://username:password@example.com/path",
+        "http://example.com/path",
+        "username",
+        "password",
+    ),
+    (
+        "http://token@example.com/path",
+        "http://example.com/path",
+        "token",
+        "",
+    ),
+    (
+        "http://example.com/path",
+        "http://example.com/path",
+        None,
+        None,
+    ),
+])
+def test_get_credentials_parses_correctly(input_url, url, username, password):
     auth = MultiDomainBasicAuth()
     get = auth._get_url_and_credentials
 
     # Check URL parsing
-    assert get("http://foo:bar@example.com/path") \
-        == ('http://example.com/path', 'foo', 'bar')
-    assert auth.passwords['example.com'] == ('foo', 'bar')
+    assert get(input_url) == (url, username, password)
+    assert (
+        # There are no credentials in the URL
+        (username is None and password is None) or
+        # Credentials were found and "cached" appropriately
+        auth.passwords['example.com'] == (username, password)
+    )
+
 
+def test_get_credentials_uses_cached_credentials():
+    auth = MultiDomainBasicAuth()
     auth.passwords['example.com'] = ('user', 'pass')
-    assert get("http://foo:bar@example.com/path") \
-        == ('http://example.com/path', 'user', 'pass')
+
+    got = auth._get_url_and_credentials("http://foo:bar@example.com/path")
+    expected = ('http://example.com/path', 'user', 'pass')
+    assert got == expected
 
 
 def test_get_index_url_credentials():