Repository: zulip/zulip

Issue:
virtualenv 20.0.1 breaks zulip installer
When virtualenv 20.0.1 is installed on a system, the zulip installer fails to complete due to an error in `scripts/lib/setup_path_on_import.py`:

```
Successfully installed Yarn 1.19.1!
+ mkdir -p /etc/zulip
+ cat
+ '[' -n '' ']'
+ dpkg-query --showformat '${Status}\n' -W rabbitmq-server
+ grep -vq ' not-installed$'
+ cat
+ '[' -n '' ']'
+ case ",$PUPPET_CLASSES," in
+ '[' -z '' ']'
+ cp -a /tmp/tmp.yliyQLjARE/zulip-server-2.1.2/zproject/prod_settings_template.py /etc/zulip/settings.py
+ '[' -n zulip ']'
+ sed -i 's/^EXTERNAL_HOST =.*/EXTERNAL_HOST = '\''zulip'\''/' /etc/zulip/settings.py
+ '[' -n root@localhost.localdomain ']'
+ sed -i 's/^ZULIP_ADMINISTRATOR =.*/ZULIP_ADMINISTRATOR = '\''root@localhost.localdomain'\''/' /etc/zulip/settings.py
+ ln -nsf /etc/zulip/settings.py /tmp/tmp.yliyQLjARE/zulip-server-2.1.2/zproject/prod_settings.py
+ /tmp/tmp.yliyQLjARE/zulip-server-2.1.2/scripts/setup/generate_secrets.py --production
Traceback (most recent call last):
  File "/tmp/tmp.yliyQLjARE/zulip-server-2.1.2/scripts/setup/generate_secrets.py", line 11, in <module>
    import scripts.lib.setup_path_on_import
  File "/tmp/tmp.yliyQLjARE/zulip-server-2.1.2/scripts/lib/setup_path_on_import.py", line 14, in <module>
    if not os.path.exists(activate_locals["site_packages"]):
KeyError: 'site_packages'

Zulip installation failed (exit code 1)!

The install process is designed to be idempotent, so you can retry after resolving whatever issue caused the failure (there should be a traceback above). A log of this installation is available in /var/log/zulip/install.log
root@zulip-2001:/tmp/tmp.yliyQLjARE/zulip-server-2.1.2#
```

Something about how the virtualenv at `/srv/zulip-venv-cache/$hash` is being created is causing issues (note that I was able to recreate this with Zulip 2.0.7 as well).

## Steps to recreate

I provisioned an Ubuntu 18.04 container with LXD and performed the following:

1. `pip3 install virtualenv==20.0.1`, then followed the [installation guide](https://zulip.readthedocs.io/en/stable/production/install.html#installer-details):

2. `cd $(mktemp -d); wget https://www.zulip.org/dist/releases/zulip-server-latest.tar.gz; tar -xf zulip-server-latest.tar.gz`

3. `zulip-server-*/scripts/setup/install --self-signed-cert --email=root@localhost.localdomain --hostname=zulip`

When version 20.0.1 is substituted with 15.1.0 or 16.7.10 on an identical container, the install process finishes without any errors. If virtualenv 20.0.1 isn't supported, I would think a check in the installer to ensure the correct version (or a note on the installation page) would be useful, since I imagine it would be common for people to want the latest version of virtualenv installed on systems.

I can provide more logs if needed. Thanks!


Discussion:
Certainly sounds like a bug.  

@andersk can you look into this?
Hello @zulip/server-production members, this issue was labeled with the "area: production installer" label, so you may want to check it out!

<!-- areaLabelAddition -->
@zulipbot claim
@timabbott @andersk  This is so because in virtualenv>20.0.0 [activate_this.py](https://github.com/pypa/virtualenv/blob/master/src/virtualenv/activation/python/activate_this.py) is modified and it now does not use the variable` 'site_package'` thereby giving key error. I guess we have to use 'path' instead of 'site_package' or any other method to verify that the virtual environment is set up or not.

Resolving patch:
diff --git a/scripts/lib/setup_path.py b/scripts/lib/setup_path.py
--- a/scripts/lib/setup_path.py
+++ b/scripts/lib/setup_path.py
@@ -12,5 +12,8 @@ def setup_path() -> None:
         activate_this = os.path.join(venv, "bin", "activate_this.py")
         activate_locals = dict(__file__=activate_this)
         exec(open(activate_this).read(), activate_locals)
-        if not os.path.exists(activate_locals["site_packages"]):
+        # Check that the python version running this function
+        # is same as python version that created the virtualenv.
+        python_version = "python{}.{}".format(*sys.version_info[:2])
+        if not os.path.exists(os.path.join(venv, 'lib', python_version)):
             raise RuntimeError(venv + " was not set up for this Python version")


Test patch: