Repository: napalm-automation/napalm

Issue:
[2.0] "pip3 install napalm" doesn't install requirements
Debian 9.2 (Stretch) with python3 v3.5.3, pip3 v9.0.1

With v1.2.0 a `pip3 install napalm==1.2.0` installs also the required modules (MarkupSafe, jinja2, netaddr, pyYAML, pyeapi, future, pynacl, bcrypt, paramiko, pyFG, scp, netmiko, lxml, pyIOSXR, ncclient, pyserial, junos-eznc, urllib3, idna, certifi, chardet, requests, pynxos, pan-python, requests-toolbelt, xmltodict, pyPluribus, chainmap, librouteros, vyattaconfparser).

With Napalm v2.0.0 no required module is installed with `pip3 install napalm`, so napalm won't work.


Discussion:
Hello,
couldn't reproduce your problem:

```
➜  ~ mkvirtualenv -p /usr/local/opt/python3/bin/python3.6 test_napalm
Running virtualenv with interpreter /usr/local/opt/python3/bin/python3.6
Using base prefix '/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6'
New python executable in /Users/dbarroso/.virtualenvs/test_napalm/bin/python3.6
Also creating executable in /Users/dbarroso/.virtualenvs/test_napalm/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /Users/dbarroso/.virtualenvs/test_napalm/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/dbarroso/.virtualenvs/test_napalm/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/dbarroso/.virtualenvs/test_napalm/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/dbarroso/.virtualenvs/test_napalm/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/dbarroso/.virtualenvs/test_napalm/bin/get_env_details
(test_napalm) ➜  ~ pip --version
pip 9.0.1 from /Users/dbarroso/.virtualenvs/test_napalm/lib/python3.6/site-packages (python 3.6)
(test_napalm) ➜  ~ pip install napalm
Collecting napalm
  Downloading napalm-2.0.0.tar.gz (138kB)
    100% |████████████████████████████████| 143kB 3.9MB/s
Building wheels for collected packages: napalm
  Running setup.py bdist_wheel for napalm ... done
  Stored in directory: /Users/dbarroso/Library/Caches/pip/wheels/0f/23/09/b4396827a7e03ad12eb2e3e81925850cbdeaed84c7d2e5fed7
Successfully built napalm
Installing collected packages: napalm
Successfully installed napalm-2.0.0
(test_napalm) ➜  ~ pip freeze
asn1crypto==0.23.0
bcrypt==3.1.4
certifi==2017.11.5
cffi==1.11.2
chardet==3.0.4
cryptography==2.1.3
future==0.16.0
idna==2.6
Jinja2==2.10
jtextfsm==0.3.1
junos-eznc==2.1.7
lxml==4.1.1
MarkupSafe==1.0
napalm==2.0.0
ncclient==0.5.3
netaddr==0.7.19
netmiko==1.4.3
paramiko==2.3.1
pyasn1==0.3.7
pycparser==2.18
pyeapi==0.8.1
pyIOSXR==0.52
PyNaCl==1.2.0
pynxos==0.0.3
pyserial==3.4
PyYAML==3.12
requests==2.18.4
scp==0.10.2
six==1.11.0
urllib3==1.22
```

Note that napalm 2.0.0 installs dependencies silently so could you send us the output of `pip3 install --verbose napalm` and a traceback with the error you are getting?

Thanks
I'm doing it in a debian docker container `docker run -ti debian:stretch`.

First I'm installing python3 and some few modules:
```
apt-get update
apt-get install python3 python3-pip python3-setuptools python3-wheel python3-cryptography python3-cffi
```
Then I'm installing napalm, see the attached log file
[napalm-pip-install.txt](https://github.com/napalm-automation/napalm/files/1466916/napalm-pip-install.txt)

The `pip3 install --verbose` shows that pip tries to install the dependencies, but fails with "ValueError: underlying buffer has been detached". That's strange, because I create a lot of docker images and use pip in them without any issues. And as I said, napalm v1.2.0 installs without issues.

Here the part where pip tries to install one dependency (future), for the full log see the attached file.

```
  Collecting future
    Downloading future-0.16.0.tar.gz (824kB)
  Exception:
  Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main
      status = self.run(options, args)
    File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 353, in run
      wb.build(autobuilding=True)
    File "/usr/lib/python3/dist-packages/pip/wheel.py", line 749, in build
      self.requirement_set.prepare_files(self.finder)
    File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 380, in prepare_files
      ignore_dependencies=self.ignore_dependencies))
    File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 620, in _prepare_file
      session=self.session, hashes=hashes)
    File "/usr/lib/python3/dist-packages/pip/download.py", line 821, in unpack_url
      hashes=hashes
    File "/usr/lib/python3/dist-packages/pip/download.py", line 659, in unpack_http_url
      hashes)
    File "/usr/lib/python3/dist-packages/pip/download.py", line 882, in _download_http_url
      _download_url(resp, link, content_file, hashes)
    File "/usr/lib/python3/dist-packages/pip/download.py", line 585, in _download_url
      progress_indicator = DownloadProgressBar(max=total_length).iter
    File "/usr/lib/python3/dist-packages/pip/utils/ui.py", line 158, in __init__
      super(WindowsMixin, self).__init__(*args, **kwargs)
    File "/usr/lib/python3/dist-packages/pip/utils/ui.py", line 82, in __init__
      super(InterruptibleMixin, self).__init__(*args, **kwargs)
    File "/usr/lib/python3/dist-packages/pip/utils/ui.py", line 118, in __init__
      super(DownloadProgressMixin, self).__init__(*args, **kwargs)
    File "/usr/share/python-wheels/progress-1.2-py2.py3-none-any.whl/progress/helpers.py", line 58, in __init__
      if self.file.isatty() and self.hide_cursor:
  ValueError: underlying buffer has been detached
```
Tested it in a virtualenv directly in debian and it works fine. But in docker it doesn't install properly. I've tried it also within a privileged container (docker run -ti --privileged) and in a virtualenv within docker. Nothing helps, it doesn't like docker.
Were you running python3.5 or some other version when it worked?
The working virtualenv was in a debian stretch 9.2 VM with python 3.5.3 and pip v9.0.1
The non-working docker container was using the same debian stretch v9.2, so it also uses python 3.5.3 and pip v9.0.1. All other software has also the same version.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -5,11 +5,12 @@
 from distutils.core import Command
 from setuptools import setup, find_packages
 from setuptools.command import install
+from subprocess import check_call
 
 
 from pip.req import parse_requirements
 
-import pip
+import pip  # noqa: test pip is installed
 import sys
 
 __author__ = 'David Barroso <dbarrosop@dravetech.com>'
@@ -26,7 +27,9 @@ def process_requirements(dep):
     print("PROCESSING DEPENDENCIES FOR {}".format(dep))
     u = uuid.uuid1()
     iter_reqs = parse_requirements("requirements/{}".format(dep), session=u)
-    [pip.main(['install', (str(ir.req))]) for ir in iter_reqs]
+
+    for ir in iter_reqs:
+        check_call([sys.executable, '-m', 'pip', 'install', str(ir.req)])
 
 
 def custom_command_driver(driver):


Test patch: