Repository: readthedocs/readthedocs.org

Issue:
Always install the latest pip version
Currently, the pip version comes from our docker images, since we don't release then too often, the pip version gets outdated very fast.

Here @humitos proposed to install the latest pip as a build step https://github.com/rtfd/readthedocs.org/issues/4808#issuecomment-433740753

I already have some code that makes that real (a description here https://github.com/rtfd/readthedocs.org/issues/4808#issuecomment-434119310)

Are you agree with this? Should we add a feature flag first? I don't think any project depends on a specific pip version, or at least to install the packages if they depend on a pip version to build their docs, specifying that on their requirements will work.


Discussion:
> I don't think any project depends on a specific pip version

I agree with this.

> if they depend on a pip version to build their docs, specifying that on their requirements will work.

This is not true. If *I need* (for some reason) `pip==9.0.0` and I pin it in my requirements, all the requirements will be installed with the `pip` version already installed in the virtualenv (`18.0.0`, for example).

At the end of the command execution, all my packages were installed with `pip==18.0.0` and now I have `pip==9.0.0` available on the virtualenv, but it was not used to install anything.
> Should we add a feature flag first?

I don't see too much value here but a precaution. I mean, depending on a not pinned package is kind of dangerous because it's something that we don't control and could be anything.

Also, if the new release of `pip` breaks, all of our builds will break. I think it's preferable to add a Feature flag here that will install the latest `pip` by default for all the projects. If we notice that this fails for some reason, we can easily go back by picking those packages that are failing.

> At the end of the command execution, all my packages were installed with pip==18.0.0 and now I have pip==9.0.0 available on the virtualenv, but it was not used to install anything.

What I mean, the project itself may depend on pip, as a dependency, not to install their packages 
> Also, if the new release of pip breaks, all of our builds will break. I think it's preferable to add a Feature flag here that will install the latest pip by default for all the projects. If we notice that this fails for some reason, we can easily go back by picking those packages that are failing.

What about pin pip and then updated it on each release?
> What about pin pip and then updated it on each release?

Yes. I think this is the best approach.

It has been easier updating some Python code than the docker image (I would like to release more frequently, though).

So, I propose to run

    pip install -U pip==18.1

as the first step before creating the virtualenv (this can be added in the VirtualEnvironment class as a method and run it first). That `18.1` can be updated just by changing the code (similar to other reqs that we install in the venv after creating it)

Also, I'd add a feature flag around that pinning so we can mark some project to always upgrade to the latest without pinning for example.

Once there, we can take another different approach. Maybe better, but we have something already working that does not require tons of work.
> as the first step before creating the virtualenv

I wasn't able to do that https://github.com/rtfd/readthedocs.org/issues/4808#issuecomment-434119310

I wasn't able to install the latest version of pip before the virtualenv creation, because the command will need to be run with sudo (we don't have sudo anyway...). We could use this option https://virtualenv.pypa.io/en/stable/userguide/#the-extra-search-dir-option or more easy, create the virtual env with the old pip version, but update the pip version after the virtualenv creation, that way we are able to install the core and user requirements using the latest pip.
Either way works for me.

Using `--extra-search-dir` is more robust to me, but there is a small problem with it that is we will need to download first the new version of `pip`/`setuptools` first. So, it will be,

    pip download pip setuptools

and then,

    python -m virtualenv --extra-search-dir .

Upgrading `pip` after creating the virtualenv env is easier, but if the first `pip` is broken for whatever reason, we won't be able to upgrade it.
> Upgrading pip after creating the virtualenv env is easier, but if the first pip is broken for whatever reason, we won't be able to upgrade it.

If the first pip is broken all docs will be broken anyway and `--extra-search-dir` wouldn't work because `pip download` shouldn't work either p:

I can say that it will be easier to see what's going on for the user if we install pip after the env creation.

Resolving patch:
diff --git a/readthedocs/doc_builder/python_environments.py b/readthedocs/doc_builder/python_environments.py
--- a/readthedocs/doc_builder/python_environments.py
+++ b/readthedocs/doc_builder/python_environments.py
@@ -2,16 +2,21 @@
 """An abstraction over virtualenv and Conda environments."""
 
 from __future__ import (
-    absolute_import, division, print_function, unicode_literals)
+    absolute_import,
+    division,
+    print_function,
+    unicode_literals,
+)
 
+import copy
 import itertools
 import json
 import logging
 import os
 import shutil
-from builtins import object, open
 
 import six
+from builtins import object, open
 from django.conf import settings
 
 from readthedocs.doc_builder.config import load_yaml_config
@@ -223,6 +228,24 @@ def setup_base(self):
 
     def install_core_requirements(self):
         """Install basic Read the Docs requirements into the virtualenv."""
+        pip_install_cmd = [
+            'python',
+            self.venv_bin(filename='pip'),
+            'install',
+            '--upgrade',
+            '--cache-dir',
+            self.project.pip_cache_path,
+        ]
+
+        # Install latest pip first,
+        # so it is used when installing the other requirements.
+        cmd = pip_install_cmd + ['pip']
+        self.build_env.run(
+            *cmd,
+            bin_path=self.venv_bin(),
+            cwd=self.checkout_path
+        )
+
         requirements = [
             'Pygments==2.2.0',
             # Assume semver for setuptools version, support up to next backwards
@@ -255,14 +278,7 @@ def install_core_requirements(self):
                 'readthedocs-sphinx-ext<0.6'
             ])
 
-        cmd = [
-            'python',
-            self.venv_bin(filename='pip'),
-            'install',
-            '--upgrade',
-            '--cache-dir',
-            self.project.pip_cache_path,
-        ]
+        cmd = copy.copy(pip_install_cmd)
         if self.config.python.use_system_site_packages:
             # Other code expects sphinx-build to be installed inside the
             # virtualenv.  Using the -I option makes sure it gets installed


Test patch:
diff --git a/readthedocs/rtd_tests/tests/test_doc_building.py b/readthedocs/rtd_tests/tests/test_doc_building.py
--- a/readthedocs/rtd_tests/tests/test_doc_building.py
+++ b/readthedocs/rtd_tests/tests/test_doc_building.py
@@ -1193,7 +1193,7 @@ def test_install_core_requirements_sphinx(self, checkout_path):
         ]
         requirements = self.base_requirements + requirements_sphinx
         args = self.pip_install_args + requirements
-        self.build_env_mock.run.assert_called_once()
+        self.assertEqual(self.build_env_mock.run.call_count, 2)
         self.assertArgsStartsWith(args, self.build_env_mock.run)
 
     @patch('readthedocs.projects.models.Project.checkout_path')
@@ -1212,7 +1212,7 @@ def test_install_core_requirements_mkdocs(self, checkout_path):
         ]
         requirements = self.base_requirements + requirements_mkdocs
         args = self.pip_install_args + requirements
-        self.build_env_mock.run.assert_called_once()
+        self.assertEqual(self.build_env_mock.run.call_count, 2)
         self.assertArgsStartsWith(args, self.build_env_mock.run)
 
     @patch('readthedocs.projects.models.Project.checkout_path')