Repository: google-deepmind/meltingpot
Issue: Issues installing meltingpot with macos
Root cause: Environment Incompatibility

Hi, I'm trying write a shell script which will clone this repo and then install dmlab2d and meltingpot, but I'm having issues. I tried having a very simple script which simply clones the repo and runs the `install_dmlab2d.sh` and `install_meltingpot.sh`, but got errors about not having bazel installed. I tested that the code below successfully installs bazel on linux and macos, but now I'm running into issues with the `install_dmlab2d.sh` component.

Running the individual shell scripts worked when I did it manually (rather than with one single shell script).

```
#!/bin/sh
set -eu

# Install bazel (Linux or MacOS)
if [[ "$(uname -s)" == 'Linux' ]]; then
  apt-get install -y apt-transport-https curl gnupg
  curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor > bazel.gpg
  mv bazel.gpg /etc/apt/trusted.gpg.d/
  echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list
  apt-get update && apt-get install -y bazel

elif [[ "$(uname -s)" == 'Darwin' ]]; then
  which -s brew
  if [[ $? != 0 ]] ; then
    echo "Homebrew not installed, run: `ruby -e '$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)'`"
    exit
  else
    brew install bazel
  fi
else
  echo "Install script does not support windows, melting pot and dmlab2d must be installed manually and are not natively supported"
  exit 1
fi

# Build
if [ ! -d "meltingpot" ]; then
  git clone https://github.com/deepmind/meltingpot.git
else
  echo "directory already exists"
fi
cd meltingpot
chmod +x *.sh
#echo "Installing dmlab2d..."
./install-dmlab2d.sh
echo "Installing melting pot..."
./install-meltingpot.sh
cd ..
rm -rf meltingpot
echo "Finished installing melting pot"
```
Sorry for the long paste but this is what my output is 

> directory already exists
> Installing melting pot...
> + main
> + check_setup
> + echo -e '\nChecking python version...'
> 
> Checking python version...
> + python --version
> Python 3.9.0
> + python --version
> + check_version_gt 3.9
> + local required=3.9
> + local input lowest
> ++ grep -Eo '[0-9]+\.[0-9]+' /dev/stdin
> ++ head -n 1
> + input=3.9
> ++ printf '3.9\n3.9'
> ++ sort -V
> ++ head -n 1
> + lowest=3.9
> + [[ 3.9 == \3\.\9 ]]
> + echo -e '\nChecking dmlab2d is installed...'
> 
> Checking dmlab2d is installed...
> + python -c 'import dmlab2d'
> Traceback (most recent call last):
>   File "<string>", line 1, in <module>
> ModuleNotFoundError: No module named 'dmlab2d'
>  !  ◰³ shimmy  ~/D/G/Shimmy   *~…  bash bin/install_melting_pot.sh       (shimmy) 3425ms  Sun Mar 19 23:15:16 2023
> Warning: bazel 6.1.1 is already installed and up-to-date.
> To reinstall 6.1.1, run:
>   brew reinstall bazel
> directory already exists
> + main
> + check_setup
> + echo -e '\nChecking OS is Linux or macOS...'
> 
> Checking OS is Linux or macOS...
> + uname -a
> Darwin Elliots-MacBook-Pro.local 21.5.0 Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:22 PDT 2022; root:xnu-8020.121.3~4/RELEASE_X86_64 x86_64
> ++ uname -s
> + [[ Darwin =~ (Linux|Darwin) ]]
> + echo -e '\nChecking python version...'
> 
> Checking python version...
> + python --version
> Python 3.9.0
> + python --version
> + check_version_gt 3.9
> + local required=3.9
> + local input lowest
> ++ grep -Eo '[0-9]+\.[0-9]+' /dev/stdin
> ++ head -n 1
> + input=3.9
> ++ printf '3.9\n3.9'
> ++ sort -V
> ++ head -n 1
> + lowest=3.9
> + [[ 3.9 == \3\.\9 ]]
> + echo -e '\nChecking pip version...'
> 
> Checking pip version...
> + pip --version
> pip 23.0.1 from /Users/elliottower/anaconda3/envs/shimmy/lib/python3.9/site-packages/pip (python 3.9)
> + pip --version
> + check_version_gt 20.3
> + local required=20.3
> + local input lowest
> ++ grep -Eo '[0-9]+\.[0-9]+' /dev/stdin
> ++ head -n 1
> + input=23.0
> ++ printf '20.3\n23.0'
> ++ sort -V
> ++ head -n 1
> + lowest=20.3
> + [[ 20.3 == \2\0\.\3 ]]
> + echo -e '\nChecking gcc version ...'
> 
> Checking gcc version ...
> + gcc --version
> Apple clang version 13.1.6 (clang-1316.0.21.2.5)
> Target: x86_64-apple-darwin21.5.0
> Thread model: posix
> InstalledDir: /Library/Developer/CommandLineTools/usr/bin
> + gcc --version
> + check_version_gt 8
> + local required=8
> + local input lowest
> ++ grep -Eo '[0-9]+\.[0-9]+' /dev/stdin
> ++ head -n 1
> + input=13.1
> ++ printf '8\n13.1'
> ++ sort -V
> ++ head -n 1
> + lowest=8
> + [[ 8 == \8 ]]
> + echo -e '\nChecking bazel version...'
> 
> Checking bazel version...
> + bazel --version
> bazel 6.1.1-homebrew
> + bazel --version
> + check_version_gt 5.2
> + local required=5.2
> + local input lowest
> ++ grep -Eo '[0-9]+\.[0-9]+' /dev/stdin
> ++ head -n 1
> + input=6.1
> ++ printf '5.2\n6.1'
> ++ sort -V
> ++ head -n 1
> + lowest=5.2
> + [[ 5.2 == \5\.\2 ]]
> + install_dmlab2d
> + echo -e '\nCloning dmlab2d...'
> 
> Cloning dmlab2d...
> + git clone https://github.com/deepmind/lab2d
> Cloning into 'lab2d'...
> remote: Enumerating objects: 1262, done.
> remote: Counting objects: 100% (348/348), done.
> remote: Compressing objects: 100% (198/198), done.
> remote: Total 1262 (delta 216), reused 198 (delta 144), pack-reused 914
> Receiving objects: 100% (1262/1262), 683.59 KiB | 1.53 MiB/s, done.
> Resolving deltas: 100% (687/687), done.
> + echo -e '\nInstalling dmlab2d requirements...'
> 
> Installing dmlab2d requirements...
> + pip install --upgrade pip packaging
> Requirement already satisfied: pip in /Users/elliottower/anaconda3/envs/shimmy/lib/python3.9/site-packages (23.0.1)
> Requirement already satisfied: packaging in /Users/elliottower/anaconda3/envs/shimmy/lib/python3.9/site-packages (23.0)
> + echo -e '\nBuilding dmlab2d wheel...'
> 
> Building dmlab2d wheel...
> ++ uname -s
> + [[ Darwin == \L\i\n\u\x ]]
> ++ uname -s
> + [[ Darwin == \D\a\r\w\i\n ]]
> + local -r LUA_VERSION=lua5_2
> + pushd lab2d
> ~/Documents/GitHub/Shimmy/meltingpot/lab2d ~/Documents/GitHub/Shimmy/meltingpot
> + C=clang
> + CXX=clang++
> + bazel build --compilation_mode=opt --dynamic_mode=off --config=lua5_2 --subcommands --verbose_failures --experimental_ui_max_stdouterr_bytes=-1 --sandbox_debug //dmlab2d:dmlab2d_wheel
> INFO: Analyzed target //dmlab2d:dmlab2d_wheel (28 packages loaded, 267 targets configured).
> INFO: Found 1 target...
> Target //dmlab2d:dmlab2d_wheel up-to-date:
>   bazel-bin/dmlab2d/dmlab2d-1.0-cp39-cp39-macosx_12_0_x86_64.whl
> INFO: Elapsed time: 1.495s, Critical Path: 0.01s
> INFO: 1 process: 1 internal.
> INFO: Build completed successfully, 1 total action
> + popd
> ~/Documents/GitHub/Shimmy/meltingpot
> + echo -e '\nInstalling dmlab2d...'
> 
> Installing dmlab2d...
> + pip install -vvv --find-links=lab2d/bazel-bin/dmlab2d dmlab2d
> Using pip 23.0.1 from /Users/elliottower/anaconda3/envs/shimmy/lib/python3.9/site-packages/pip (python 3.9)
> Non-user install because site-packages writeable
> Created temporary directory: /private/var/folders/5y/hjb1rykx58xcbwg6r04m76tr0000gn/T/pip-build-tracker-42zykkbt
> Initialized build tracking at /private/var/folders/5y/hjb1rykx58xcbwg6r04m76tr0000gn/T/pip-build-tracker-42zykkbt
> Created build tracker: /private/var/folders/5y/hjb1rykx58xcbwg6r04m76tr0000gn/T/pip-build-tracker-42zykkbt
> Entered build tracker: /private/var/folders/5y/hjb1rykx58xcbwg6r04m76tr0000gn/T/pip-build-tracker-42zykkbt
> Created temporary directory: /private/var/folders/5y/hjb1rykx58xcbwg6r04m76tr0000gn/T/pip-install-j8cy_j8z
> Created temporary directory: /private/var/folders/5y/hjb1rykx58xcbwg6r04m76tr0000gn/T/pip-ephem-wheel-cache-20_9ju2h
> Looking in links: lab2d/bazel-bin/dmlab2d
> 1 location(s) to search for versions of dmlab2d:
> * https://pypi.org/simple/dmlab2d/
> Fetching project page and analyzing links: https://pypi.org/simple/dmlab2d/
> Getting page https://pypi.org/simple/dmlab2d/
> Found index url https://pypi.org/simple
> Looking up "https://pypi.org/simple/dmlab2d/" in the cache
> Request header has "max_age" as 0, cache bypassed
> Starting new HTTPS connection (1): pypi.org:443
> https://pypi.org:443 "GET /simple/dmlab2d/ HTTP/1.1" 404 13
> Status code 404 not in (200, 203, 300, 301, 308)
> Could not fetch URL https://pypi.org/simple/dmlab2d/: 404 Client Error: Not Found for url: https://pypi.org/simple/dmlab2d/ - skipping
> Skipping link: not a file: https://pypi.org/simple/dmlab2d/
> Skipping link: not a file: file:///private/var/tmp/_bazel_elliottower/375bd521004fa747049b154d1535910c/execroot/org_deepmind_lab2d/bazel-out/darwin-opt/bin/dmlab2d/lib
> Skipping link: unsupported archive format: .txt: file:///private/var/tmp/_bazel_elliottower/375bd521004fa747049b154d1535910c/execroot/org_deepmind_lab2d/bazel-out/darwin-opt/bin/dmlab2d/dmlab2d_wheel_target_wrapped_inputs.txt
> Skipping link: unsupported archive format: .name: file:///private/var/tmp/_bazel_elliottower/375bd521004fa747049b154d1535910c/execroot/org_deepmind_lab2d/bazel-out/darwin-opt/bin/dmlab2d/dmlab2d_wheel.name
> Skipping link: unsupported archive format: .txt: file:///private/var/tmp/_bazel_elliottower/375bd521004fa747049b154d1535910c/execroot/org_deepmind_lab2d/bazel-out/darwin-opt/bin/dmlab2d/dmlab2d_wheel.metadata.txt
> Skipping link: unsupported archive format: .cppmap: file:///private/var/tmp/_bazel_elliottower/375bd521004fa747049b154d1535910c/execroot/org_deepmind_lab2d/bazel-out/darwin-opt/bin/dmlab2d/dmlab2d_pybind.so.cppmap
> Skipping link: unsupported archive format: .params: file:///private/var/tmp/_bazel_elliottower/375bd521004fa747049b154d1535910c/execroot/org_deepmind_lab2d/bazel-out/darwin-opt/bin/dmlab2d/dmlab2d_pybind.so-2.params
> Skipping link: unsupported archive format: .so: file:///private/var/tmp/_bazel_elliottower/375bd521004fa747049b154d1535910c/execroot/org_deepmind_lab2d/bazel-out/darwin-opt/bin/dmlab2d/dmlab2d_pybind.so
> Skipping link: none of the wheel's tags (cp39-cp39-macosx_12_0_x86_64) are compatible (run pip debug --verbose to show compatible tags): file:///private/var/tmp/_bazel_elliottower/375bd521004fa747049b154d1535910c/execroot/org_deepmind_lab2d/bazel-out/darwin-opt/bin/dmlab2d/dmlab2d-1.0-cp39-cp39-macosx_12_0_x86_64.whl
> Skipping link: not a file: file:///private/var/tmp/_bazel_elliottower/375bd521004fa747049b154d1535910c/execroot/org_deepmind_lab2d/bazel-out/darwin-opt/bin/dmlab2d/_objs
> Given no hashes to check 0 links for project 'dmlab2d': discarding no candidates
> ERROR: Could not find a version that satisfies the requirement dmlab2d (from versions: none)
> ERROR: No matching distribution found for dmlab2d
> Exception information:
> Traceback (most recent call last):
>   File "/Users/elliottower/anaconda3/envs/shimmy/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 348, in resolve
>     self._add_to_criteria(self.state.criteria, r, parent=None)
>   File "/Users/elliottower/anaconda3/envs/shimmy/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
>     raise RequirementsConflicted(criterion)
> pip._vendor.resolvelib.resolvers.RequirementsConflicted: Requirements conflict: SpecifierRequirement('dmlab2d')
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "/Users/elliottower/anaconda3/envs/shimmy/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
>     result = self._result = resolver.resolve(
>   File "/Users/elliottower/anaconda3/envs/shimmy/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
>     state = resolution.resolve(requirements, max_rounds=max_rounds)
>   File "/Users/elliottower/anaconda3/envs/shimmy/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 350, in resolve
>     raise ResolutionImpossible(e.criterion.information)
> pip._vendor.resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('dmlab2d'), parent=None)]
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/Users/elliottower/anaconda3/envs/shimmy/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
>     status = run_func(*args)
>   File "/Users/elliottower/anaconda3/envs/shimmy/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
>     return func(self, options, args)
>   File "/Users/elliottower/anaconda3/envs/shimmy/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 419, in run
>     requirement_set = resolver.resolve(
>   File "/Users/elliottower/anaconda3/envs/shimmy/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 101, in resolve
>     raise error from e
> pip._internal.exceptions.DistributionNotFound: No matching distribution found for dmlab2d
> 

Your script is checking pypi.org for dmlab2d which means it's not finding the wheel you built. 

You build `~/Documents/GitHub/Shimmy/meltingpot/lab2dbazel-bin/dmlab2d/dmlab2d-1.0-cp39-cp39-macosx_12_0_x86_64.whl` which I guess it's rejecting as that's x86 and not arm64. So I guess something in your toolchain isn't the arm64 version (e.g. gcc, bazel).

> Your script is checking pypi.org for dmlab2d which means it's not finding the wheel you built.
> 
> You build `~/Documents/GitHub/Shimmy/meltingpot/lab2dbazel-bin/dmlab2d/dmlab2d-1.0-cp39-cp39-macosx_12_0_x86_64.whl` which I guess it's rejecting as that's x86 and not arm64. So I guess something in your toolchain isn't the arm64 version (e.g. gcc, bazel).

Good catch about the x86 vs arm versions, will test and see if that solves the issue.