Repository: zulip/zulip

Issue:
4.9: dev failing provisioning due to orjson==3.6.6 requiring rust toolchain
crazy! orjson seems to have a hidden dependency?!

```
    default:     Preparing wheel metadata: finished with status 'error'
    default:     ERROR: Command errored out with exit status 1:
    default:      command: /srv/zulip-venv-cache/b4a767a2f298218a7feee0f52190b8ade6698b74/zulip-py3-venv/bin/python3 /srv/zulip-venv-cache/b4a767a2f298218a7feee0f52190b8ade6698b74/zulip-py3-venv/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp7f5pjtbs
    default:          cwd: /tmp/pip-install-__o230mo/orjson
    default:     Complete output (6 lines):
    default:     Checking for Rust toolchain....
    default:
    default:     Cargo, the Rust package manager, is not installed or is not on PATH.
    default:     This package requires Rust and Cargo to compile extensions. Install it through
    default:     the system's package manager or via https://rustup.rs/
    default:
    default:     ----------------------------------------
    default: ERROR: Command errored out with exit status 1: /srv/zulip-venv-cache/b4a767a2f298218a7feee0f52190b8ade6698b74/zulip-py3-venv/bin/python3 /srv/zulip-venv-cache/b4a767a2f298218a7feee0f52190b8ade6698b74/zulip-py3-venv/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp7f5pjtbs Check the logs for full command output.
    default: WARNING: You are using pip version 20.3.4; however, version 21.3.1 is available.
    default: You should consider upgrading via the '/srv/zulip-venv-cache/b4a767a2f298218a7feee0f52190b8ade6698b74/zulip-py3-venv/bin/python3 -m pip install --upgrade pip' command.
```

running standalone I get the same error:

```
$ pip3 install orjson==3.6.6
Collecting orjson==3.6.6
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/99/6c/cb0dcbc2500c004764cfadb63a51701c328f45f8689ffecbc8906df0f0de/orjson-3.6.6.tar.gz (550kB)
    100% |████████████████████████████████| 552kB 1.9MB/s
  Installing build dependencies ... error
  Complete output from command /usr/bin/python3 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-6jn9rmj7 --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- maturin>=0.12.6,<0.13:
  Collecting maturin<0.13,>=0.12.6
    Cache entry deserialization failed, entry ignored
    Downloading https://files.pythonhosted.org/packages/fb/16/d401cb37d3adb33c6e56cb4fc3916626afd75f4a57a315895abea4837c95/maturin-0.12.6.tar.gz (139kB)
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
  Collecting toml~=0.10.0 (from maturin<0.13,>=0.12.6)
    Using cached https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl
  Building wheels for collected packages: maturin
    Running setup.py bdist_wheel for maturin: started
    Running setup.py bdist_wheel for maturin: finished with status 'error'
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-of8zdmcq/maturin/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-fxd1ov71 --python-tag cp37:
    running bdist_wheel
    running build
    installing to build/bdist.linux-aarch64/wheel
    running install
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-of8zdmcq/maturin/setup.py", line 135, in <module>
        zip_safe=False,
      File "/tmp/pip-build-env-2xerxhnc/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/tmp/pip-build-env-2xerxhnc/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 335, in run
        self.run_command('install')
      File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-of8zdmcq/maturin/setup.py", line 58, in run
        "cargo not found in PATH. Please install rust "
    RuntimeError: cargo not found in PATH. Please install rust (https://www.rust-lang.org/tools/install) and try again

    ----------------------------------------
    Failed building wheel for maturin
    Running setup.py clean for maturin
  Failed to build maturin
  Installing collected packages: toml, maturin
    Running setup.py install for maturin: started
      Running setup.py install for maturin: finished with status 'error'
      Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-of8zdmcq/maturin/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-cv6n90um/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-6jn9rmj7 --compile:
      running install
      Traceback (most recent call last):
        File "<string>", line 1, in <module>
        File "/tmp/pip-install-of8zdmcq/maturin/setup.py", line 135, in <module>
          zip_safe=False,
        File "/tmp/pip-build-env-2xerxhnc/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
          self.run_command(cmd)
        File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/tmp/pip-install-of8zdmcq/maturin/setup.py", line 58, in run
          "cargo not found in PATH. Please install rust "
      RuntimeError: cargo not found in PATH. Please install rust (https://www.rust-lang.org/tools/install) and try again

      ----------------------------------------
  Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-of8zdmcq/maturin/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-cv6n90um/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-6jn9rmj7 --compile" failed with error code 1 in /tmp/pip-install-of8zdmcq/maturin/
```

cc @alexmv 

(vagrant/docker Engine 20.10.12 (desktop 4.4.2) - host OS is Mac OSX 12.1 Monterey)



Discussion:
haha, not getting those hours back.  tl;dr: there's something very screwy with OSX 12.1 and docker... anyway, I also wanted to dev on ubuntu 20.04 to match my prod instance, so I decided to nail both at once... below is the diff. Notes:
- curl crashed so I replaced with wget, which... also crashed!  wget2 seems to work.
- `pip3 install orjson=3.6.6` fails on needing rust and `python-is-python3`, so I added those.
- `pip3 install --upgrade pip` might be unnecessary... 

hope this helps!
cc @alexmv 

```
$ git diff
diff --git a/scripts/lib/install-yarn b/scripts/lib/install-yarn
index c28dc12cf6..6c5cb2b211 100755
--- a/scripts/lib/install-yarn
+++ b/scripts/lib/install-yarn
@@ -18,7 +18,7 @@ if ! check_version; then
     tmpdir="$(mktemp -d)"
     trap 'rm -r "$tmpdir"' EXIT
     cd "$tmpdir"
-    curl -fLO "https://registry.npmjs.org/yarn/-/$tarball"
+    wget2 "https://registry.npmjs.org/yarn/-/$tarball"
     sha256sum -c <<<"$sha256 $tarball"
     rm -rf /srv/zulip-yarn
     mkdir /srv/zulip-yarn
diff --git a/tools/setup/dev-vagrant-docker/Dockerfile b/tools/setup/dev-vagrant-docker/Dockerfile
index 98fa714888..a8cb403d8f 100644
--- a/tools/setup/dev-vagrant-docker/Dockerfile
+++ b/tools/setup/dev-vagrant-docker/Dockerfile
@@ -1,4 +1,4 @@
-FROM debian:10
+FROM ubuntu:20.04

 ARG DEBIAN_MIRROR

@@ -10,17 +10,22 @@ RUN echo locales locales/default_environment_locale select C.UTF-8 | debconf-set
     && apt-get install --no-install-recommends -y \
            ca-certificates \
            curl \
+          wget2 \
+          rustc cargo \
            locales \
            lsb-release \
            openssh-server \
            python3 \
+          python-is-python3 \
            sudo \
            systemd \
     && rm -rf /var/lib/apt/lists/*

 ARG VAGRANT_UID

-RUN \
+RUN pip3 install --upgrade pip \
     # We use https://github.com/gdraheim/docker-systemctl-replacement
     # to make services we install like PostgreSQL, Redis, etc. normally
     # managed by systemd start within Docker, which breaks normal
```
@alexmv is there any follow-up we need to do here, or should we close the issue?

Resolving patch:
diff --git a/version.py b/version.py
--- a/version.py
+++ b/version.py
@@ -48,4 +48,4 @@
 #   historical commits sharing the same major version, in which case a
 #   minor version bump suffices.
 
-PROVISION_VERSION = "173.2"
+PROVISION_VERSION = "173.3"


Test patch: