Repository: certbot/certbot

Issue:
Fix test_sdists test farm test
`tests/letstest/scripts/test_sdists.sh` which we currently run as part of the release process is now failing. I think this should be fixed quickly so it doesn't cause us problems whenever we go to do our next release. The cause of the failure is that newer versions of `pytest` cannot be installed using old versions of `setuptools`.

To resolve this, we either need to use a newer version of `setuptools` or an older version of `pytest`. My thought is to upgrade `setuptools` because it should prevent problems with other packages in the future and our tests should arguably work with the latest version of every package installed, however, either approach should work.

If we upgrade `setuptools`, we need to decide where to do it. My thought would be to do it somewhere in `tools/_venv_common.py` or the scripts it calls so other code that uses `tools/_venv_common.py` would benefit. (It also looks like `tests/letstest/scripts/test_sdists.sh` still uses the old name `tools/_venv_common.sh` and needs to be updated.) `setuptools>=30.3` seems to be [required](https://github.com/pytest-dev/pytest/commit/d4351ac5a287eb96521f09618bbbe1c73f4e6884#diff-2eeaed663bd0d25b7e608891384b7298).

If we decided to pin back `pytest` (and maybe other packages) instead, I don't have an opinion on the approach.

@ohemorange or @sydneyli, are you interested in taking this? I initially planned on doing it myself but haven't managed to get to it yet and think it's important. It shouldn't be much work.


Discussion:
I'll add it to the highest-priority queue
Thank you!
I had to manually update boto3 to even get to this failure, is that a problem with my setup or should that be fixed while we're here too?
I responded to this out of band.

Resolving patch:
diff --git a/tools/_venv_common.py b/tools/_venv_common.py
--- a/tools/_venv_common.py
+++ b/tools/_venv_common.py
@@ -149,13 +149,15 @@ def main(venv_name, venv_args, args):
     command.extend(shlex.split(venv_args))
     subprocess_with_print(command)
 
-    # We execute the two following commands in the context of the virtual environment, to install
+    # We execute the following commands in the context of the virtual environment, to install
     # the packages in it. To do so, we append the venv binary to the PATH that will be used for
     # these commands. With this trick, correct python executable will be selected.
     new_environ = os.environ.copy()
     new_environ['PATH'] = os.pathsep.join([get_venv_bin_path(venv_name), new_environ['PATH']])
     subprocess_with_print('python {0}'.format('./letsencrypt-auto-source/pieces/pipstrap.py'),
                           env=new_environ, shell=True)
+    subprocess_with_print("python -m pip install --upgrade 'setuptools>=30.3'",
+                          env=new_environ, shell=True)
     subprocess_with_print('python {0} {1}'.format('./tools/pip_install.py', ' '.join(args)),
                           env=new_environ, shell=True)
 


Test patch:
diff --git a/tests/letstest/scripts/test_sdists.sh b/tests/letstest/scripts/test_sdists.sh
--- a/tests/letstest/scripts/test_sdists.sh
+++ b/tests/letstest/scripts/test_sdists.sh
@@ -10,7 +10,7 @@ VERSION=$(letsencrypt-auto-source/version.py)
 export VENV_ARGS="-p $PYTHON"
 
 # setup venv
-tools/_venv_common.sh --requirement letsencrypt-auto-source/pieces/dependency-requirements.txt
+tools/_venv_common.py --requirement letsencrypt-auto-source/pieces/dependency-requirements.txt
 . ./venv/bin/activate
 
 # build sdists