Repository: pypa/pip

Issue:
AttributeError: 'NoneType' object has no attribute 'netloc'
**Environment**

* pip version: 18.1
* Python version: 3.6
* OS: linux

**Description**
I am using a fresh conda environment (most packages from conda-forge), pip 18.1, python 3.6, and linux. I am utilizing the new PEP508 syntax for installing private packages from our internal gitlab server (anonymized below). I added some print statements that I hope will be helpful:

```
$ pip install -e .
Obtaining file:///nas/home/broot/Programs/tools/catutils
req: shapely
comes_from: catutils==0.15.dev1 from file:///nas/home/broot/Programs/tools/catutils
req: netCDF4
comes_from: catutils==0.15.dev1 from file:///nas/home/broot/Programs/tools/catutils
req: aershp>=0.10
comes_from: catutils==0.15.dev1 from file:///nas/home/broot/Programs/tools/catutils
req: numpy!=1.10,!=1.11.0
comes_from: catutils==0.15.dev1 from file:///nas/home/broot/Programs/tools/catutils
req: scipy
comes_from: catutils==0.15.dev1 from file:///nas/home/broot/Programs/tools/catutils
req: GDAL
comes_from: catutils==0.15.dev1 from file:///nas/home/broot/Programs/tools/catutils
Requirement already satisfied: shapely in /rd22/scratch/broot/miniconda/envs/py3k/lib/python3.6/site-packages (from catutils==0.15.dev1) (1.6.4.post1)
Requirement already satisfied: netCDF4 in /rd22/scratch/broot/miniconda/envs/py3k/lib/python3.6/site-packages (from catutils==0.15.dev1) (1.4.1)
req: numpy>=1.7
comes_from: netCDF4 in /rd22/scratch/broot/miniconda/envs/py3k/lib/python3.6/site-packages (from catutils==0.15.dev1)
req: cftime
comes_from: netCDF4 in /rd22/scratch/broot/miniconda/envs/py3k/lib/python3.6/site-packages (from catutils==0.15.dev1)
Requirement already satisfied: aershp>=0.10 in /nas/home/broot/Programs/tools/aershp/lib (from catutils==0.15.dev1) (0.16.dev1)
req: nose
comes_from: aershp>=0.10 in /nas/home/broot/Programs/tools/aershp/lib (from catutils==0.15.dev1)
req: numpy
comes_from: aershp>=0.10 in /nas/home/broot/Programs/tools/aershp/lib (from catutils==0.15.dev1)
req: matplotlib<2.2.0
comes_from: aershp>=0.10 in /nas/home/broot/Programs/tools/aershp/lib (from catutils==0.15.dev1)
req: Shapely
comes_from: aershp>=0.10 in /nas/home/broot/Programs/tools/aershp/lib (from catutils==0.15.dev1)
req: GDAL
comes_from: aershp>=0.10 in /nas/home/broot/Programs/tools/aershp/lib (from catutils==0.15.dev1)
req: pyyaml
comes_from: aershp>=0.10 in /nas/home/broot/Programs/tools/aershp/lib (from catutils==0.15.dev1)
req: jinja2
comes_from: aershp>=0.10 in /nas/home/broot/Programs/tools/aershp/lib (from catutils==0.15.dev1)
req: scripttest
comes_from: aershp>=0.10 in /nas/home/broot/Programs/tools/aershp/lib (from catutils==0.15.dev1)
req: netCDF4
comes_from: aershp>=0.10 in /nas/home/broot/Programs/tools/aershp/lib (from catutils==0.15.dev1)
req: six
comes_from: aershp>=0.10 in /nas/home/broot/Programs/tools/aershp/lib (from catutils==0.15.dev1)
req: atomic_context@ git+ssh://git@***********.com/common/atomic_context.git@release
comes_from: aershp>=0.10 in /nas/home/broot/Programs/tools/aershp/lib (from catutils==0.15.dev1)
Exception:
Traceback (most recent call last):
  File "/home/broot/scratch/miniconda/envs/py3k/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 143, in main
    status = self.run(options, args)
  File "/home/broot/scratch/miniconda/envs/py3k/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 318, in run
    resolver.resolve(requirement_set)
  File "/home/broot/scratch/miniconda/envs/py3k/lib/python3.6/site-packages/pip/_internal/resolve.py", line 102, in resolve
    self._resolve_one(requirement_set, req)
  File "/home/broot/scratch/miniconda/envs/py3k/lib/python3.6/site-packages/pip/_internal/resolve.py", line 318, in _resolve_one
    add_req(subreq, extras_requested=available_requested)
  File "/home/broot/scratch/miniconda/envs/py3k/lib/python3.6/site-packages/pip/_internal/resolve.py", line 275, in add_req
    wheel_cache=self.wheel_cache,
  File "/home/broot/scratch/miniconda/envs/py3k/lib/python3.6/site-packages/pip/_internal/req/constructors.py", line 290, in install_req_from_req
    if req.url and comes_from.link.netloc in domains_not_allowed:
AttributeError: 'NoneType' object has no attribute 'netloc'
```
A bit of explanation of my dependency tree. I am installing a package called "catutils". It has a dependency on another private package called "aershp". That package, in turn, has a dependency upon two other private, pure-python, packages called "ncxml" and "atomic_context". I installed "aershp" via "pip install -e .", and it fetched the other two packages from our gitlab server. This failure happened when I tried doing "pip install -e ." for the catutils package.

Some additional insights. I have gone back and reinstalled (`pip install -e .`) the "atomic_context" and the "ncxml" packages. I then `pip uninstall aershp`, and then did `pip install -e .`. This failed for some odd reason:
```  Could not find a version that satisfies the requirement aershp>=0.10 (from catutils==0.15.dev1) (from versions: )```
So, I then did a `pip uninstall catutils`: which said that it was skipping it because it is not installed (as expected), but then went back to the "aershp" package and redid `pip install -e .`, and instead of the unsatisfiable error, I got the same error as originally posted above.


Discussion:
Correction, when I reinstalled "aershp" after fake-uninstalling "catutils", it installed fine, but installing "catutils" still failed.
It looks like the line causing the exception was introduced in PR #5571.
This should be fixed in #5788, there's a test in this comment: https://github.com/pypa/pip/pull/5788#issuecomment-427596606.
Yeah, looking at that comment, it seems to make sense (although, my knowledge of pip internals is extremely limited). Any thoughts on the incomplete cleanup from the failed install?
@benoit-pierre -- your fixes for this bug (which has bitten me) look good.  But the PRs are now closed.  Did you find a way around this issue?
I also seem to have the same problem:

- pip 19.0.3
- OS Windows 10 (using git bash)
- python 3.6

My situation seems analogous to the situation reported here.  I am installing a private package called deep-nlp-embedding (`pip install -e .`) that depends on another private package called deep-nlp-core that was previously installed with `pip install -e .`  

```
$ pip install -e .
Obtaining file:///E:/Repos/flamingo-nlp/deep-nlp-embedding
Requirement already satisfied: deep-nlp-core in e:\repos\flamingo-nlp\deep-nlp-core (from deep-nlp-embedding==0.0.2) (0.0.2)
Exception:
Traceback (most recent call last):
  File "C:\Anaconda\envs\flamingo_nlp\lib\site-packages\pip\_internal\cli\base_command.py", line 179, in main
    status = self.run(options, args)
  File "C:\Anaconda\envs\flamingo_nlp\lib\site-packages\pip\_internal\commands\install.py", line 315, in run
    resolver.resolve(requirement_set)
  File "C:\Anaconda\envs\flamingo_nlp\lib\site-packages\pip\_internal\resolve.py", line 131, in resolve
    self._resolve_one(requirement_set, req)
  File "C:\Anaconda\envs\flamingo_nlp\lib\site-packages\pip\_internal\resolve.py", line 357, in _resolve_one
    add_req(subreq, extras_requested=available_requested)
  File "C:\Anaconda\envs\flamingo_nlp\lib\site-packages\pip\_internal\resolve.py", line 314, in add_req
    use_pep517=self.use_pep517
  File "C:\Anaconda\envs\flamingo_nlp\lib\site-packages\pip\_internal\req\constructors.py", line 328, in install_req_from_req_string
    if req.url and comes_from.link.netloc in domains_not_allowed:
AttributeError: 'NoneType' object has no attribute 'netloc'
```

Note the first private package was installed using [PEP 508 URLS](https://www.python.org/dev/peps/pep-0508/), specfically:
```
torch@ https://download.pytorch.org/whl/cu90/torch-1.0.0-cp36-cp36m-win_amd64.whl from https://download.pytorch.org/whl/cu90/torch-1.0.0-cp36-cp36m-win_amd64.whl in c:\anaconda\envs\flamingo_nlp\lib\site-packages (from deep-nlp-core==0.0.2) (1.0.0)
```

Resolving patch:
diff --git a/src/pip/_internal/req/constructors.py b/src/pip/_internal/req/constructors.py
--- a/src/pip/_internal/req/constructors.py
+++ b/src/pip/_internal/req/constructors.py
@@ -325,7 +325,8 @@ def install_req_from_req_string(
         PyPI.file_storage_domain,
         TestPyPI.file_storage_domain,
     ]
-    if req.url and comes_from.link.netloc in domains_not_allowed:
+    if (req.url and comes_from and comes_from.link and
+            comes_from.link.netloc in domains_not_allowed):
         # Explicitly disallow pypi packages that depend on external urls
         raise InstallationError(
             "Packages installed from PyPI cannot depend on packages "


Test patch:
diff --git a/tests/unit/test_req_install.py b/tests/unit/test_req_install.py
--- a/tests/unit/test_req_install.py
+++ b/tests/unit/test_req_install.py
@@ -2,8 +2,11 @@
 import tempfile
 
 import pytest
+from pip._vendor.packaging.requirements import Requirement
 
-from pip._internal.req.constructors import install_req_from_line
+from pip._internal.req.constructors import (
+    install_req_from_line, install_req_from_req_string,
+)
 from pip._internal.req.req_install import InstallRequirement
 
 
@@ -42,3 +45,50 @@ def test_forward_slash_results_in_a_link(self, tmpdir):
         )
 
         assert requirement.link is not None
+
+
+class TestInstallRequirementFrom(object):
+
+    def test_install_req_from_string_without_comes_from(self):
+        """
+        Test to make sure that install_req_from_string succeeds
+        when called with URL (PEP 508) but without comes_from.
+        """
+        # Test with a PEP 508 url install string:
+        wheel_url = ("https://download.pytorch.org/whl/cu90/"
+                     "torch-1.0.0-cp36-cp36m-win_amd64.whl")
+        install_str = "torch@ " + wheel_url
+        install_req = install_req_from_req_string(install_str)
+
+        assert isinstance(install_req, InstallRequirement)
+        assert install_req.link.url == wheel_url
+        assert install_req.req.url == wheel_url
+        assert install_req.comes_from is None
+        assert install_req.is_wheel
+
+    def test_install_req_from_string_with_comes_from_without_link(self):
+        """
+        Test to make sure that install_req_from_string succeeds
+        when called with URL (PEP 508) and comes_from
+        does not have a link.
+        """
+        # Test with a PEP 508 url install string:
+        wheel_url = ("https://download.pytorch.org/whl/cu90/"
+                     "torch-1.0.0-cp36-cp36m-win_amd64.whl")
+        install_str = "torch@ " + wheel_url
+
+        # Dummy numpy "comes_from" requirement without link:
+        comes_from = InstallRequirement(
+            Requirement("numpy>=1.15.0"), comes_from=None
+        )
+
+        # Attempt install from install string comes:
+        install_req = install_req_from_req_string(
+            install_str, comes_from=comes_from
+        )
+
+        assert isinstance(install_req, InstallRequirement)
+        assert install_req.comes_from.link is None
+        assert install_req.link.url == wheel_url
+        assert install_req.req.url == wheel_url
+        assert install_req.is_wheel