Repository: google-deepmind/meltingpot
Issue: ModuleNotFoundError: No module named 'meltingpot.python'
Root cause: Configuration Error

I'm trying to run Melting Pot. I followed the instructions in the readme, creating the virtualenv, installing `dmlab2d` and installing `meltingpot` from the repo root. When I try to run this line:

```
python3 meltingpot/python/human_players/play_clean_up.py
```

I get this error:

```
(meltingpot_venv) ramrachum@cornholio:~/Desktop/meltingpot$ python3 meltingpot/python/human_players/play_clean_up.pyTraceback (most recent call last):
  File "/usr/local/google/home/ramrachum/Desktop/meltingpot/meltingpot/python/human_players/play_clean_up.py", line 25, in <module>
    from meltingpot.python.configs.substrates import clean_up as mp_clean_up
ModuleNotFoundError: No module named 'meltingpot.python'
```

Trying to troubleshoot this issue, it seems like the `meltingpot` package exists, but it doesn't have any `python` package inside it: 

```
(meltingpot_venv) ramrachum@cornholio:~/Desktop/meltingpot$ ll /usr/local/google/home/ramrachum/meltingpot_venv/lib/python3.9/site-packages/meltingpot/
total 32K
drwxr-x---   5 ramrachum primarygroup 4.0K Feb 20 10:35 .
drwxr-x--- 159 ramrachum primarygroup  12K Feb 20 10:35 ..
drwxr-x---   2 ramrachum primarygroup 4.0K Feb 20 10:35 __pycache__
drwxr-x---   4 ramrachum primarygroup 4.0K Feb 20 10:35 lua
drwxr-x---   3 ramrachum primarygroup 4.0K Feb 20 10:35 assets
-rw-r-----   1 ramrachum primarygroup  595 Feb 20 10:35 __init__.py
```

Did I do anything wrong?

I worked around this issue: Instead of running `pip3 install .` I ran `export PYTHONPATH=$(pwd)`, and this made the demo work. (You'd also need to install all the dependencies first.)

Hi, you can also run 

```
python -m meltingpot.python.human_players.play_clean_up
```

It can also work too. In the newly updated version, this issue will not occur.


just to chime in, I just had this issue too, where after running `pip install .` and then `python3 setup.py test`, I got `ImportError: Failed to import test module: meltingpot.python`

@cool-RR @GoingMyWay could you please elaborate a bit on how your installation methods worked?

From what I understand, `python -m meltingpot.python.human_players.play_clean_up` only runs a piece of code, it doesn't load up an environment such that new learning code can be applied, and `export PYTHONPATH=$(pwd)` doesn't allow for the installation of the library?

@jzleibo @jagapiou @duenez would any of you guys be so kind as to take a look into this? I'd love to use your library for a paper I'm trying to get soon and I'd really like to get this working in the immediate future, however unless I've completely lost my mind this seems like a fairly large open issue that's gone over 2 weeks without a reply?

@jkterry1 I followed steps 1-4 here: https://github.com/deepmind/meltingpot/#installation except instead of running `pip3 install .`, I ran `export PYTHONPATH=$(pwd)`. This makes Python import the meltingpot library from the repo files rather than `site-packages`. Try it to see whether it works for you. 

@cool-RR I just ran export PYTHONPATH=$(pwd) and at least got a slightly different error?

```
jkterry0@instance-3:~/meltingpot$ export PYTHONPATH=$(pwd)
jkterry0@instance-3:~/meltingpot$ python3 setup.py test
running test
WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.
/opt/conda/lib/python3.7/site-packages/setuptools/installer.py:30: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
  SetuptoolsDeprecationWarning,
running egg_info
writing dm_meltingpot.egg-info/PKG-INFO
writing dependency_links to dm_meltingpot.egg-info/dependency_links.txt
writing requirements to dm_meltingpot.egg-info/requires.txt
writing top-level names to dm_meltingpot.egg-info/top_level.txt
reading manifest file 'dm_meltingpot.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'dm_meltingpot.egg-info/SOURCES.txt'
running build_ext
harvest_finished (unittest.loader._FailedTest) ... ERROR
play_harvest (unittest.loader._FailedTest) ... ERROR
python (unittest.loader._FailedTest) ... ERROR
meltingpot.python (unittest.loader._FailedTest) ... ERROR

======================================================================
ERROR: harvest_finished (unittest.loader._FailedTest)

ImportError: Failed to import test module: meltingpot.python
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/unittest/loader.py", line 470, in _find_test_path
    package = self._get_module_from_name(name)
  File "/opt/conda/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/home/jkterry0/meltingpot/meltingpot/python/__init__.py", line 17, in <module>
    from meltingpot.python import scenario
  File "/home/jkterry0/meltingpot/meltingpot/python/scenario.py", line 20, in <module>
    import chex
  File "/home/jkterry0/meltingpot/.eggs/chex-0.1.1-py3.7.egg/chex/__init__.py", line 17, in <mod
ule>
    from chex._src.asserts import assert_axis_dimension
  File "/home/jkterry0/meltingpot/.eggs/chex-0.1.1-py3.7.egg/chex/_src/asserts.py", line 26, in 
<module>
    from chex._src import asserts_internal as _ai
  File "/home/jkterry0/meltingpot/.eggs/chex-0.1.1-py3.7.egg/chex/_src/asserts_internal.py", lin
e 32, in <module>
    from chex._src import pytypes
  File "/home/jkterry0/meltingpot/.eggs/chex-0.1.1-py3.7.egg/chex/_src/pytypes.py", line 18, in 
<module>
    import jax
  File "/home/jkterry0/meltingpot/.eggs/jax-0.3.1-py3.7.egg/jax/__init__.py", line 137, in <modu
le>
    from jax import nn as nn
  File "/home/jkterry0/meltingpot/.eggs/jax-0.3.1-py3.7.egg/jax/nn/__init__.py", line 21, in <mo
dule>
    from jax._src.nn.functions import (
  File "/home/jkterry0/meltingpot/.eggs/jax-0.3.1-py3.7.egg/jax/_src/nn/functions.py", line 28, 
in <module>
    from jax.scipy.special import expit
  File "/home/jkterry0/meltingpot/.eggs/jax-0.3.1-py3.7.egg/jax/scipy/__init__.py", line 16, in 
<module>
    from jax.scipy import interpolate as interpolate
  File "/home/jkterry0/meltingpot/.eggs/jax-0.3.1-py3.7.egg/jax/scipy/interpolate/__init__.py", 
line 19, in <module>
    from jax._src.third_party.scipy.interpolate import (
  File "/home/jkterry0/meltingpot/.eggs/jax-0.3.1-py3.7.egg/jax/_src/third_party/scipy/interpola
te.py", line 2, in <module>
    import scipy.interpolate as osp_interpolate
  File "/opt/conda/lib/python3.7/site-packages/scipy/interpolate/__init__.py", line 174, in <mod
ule>
    from ._rbfinterp import *
  File "/opt/conda/lib/python3.7/site-packages/scipy/interpolate/_rbfinterp.py", line 11, in <mo
dule>
    from ._rbfinterp_pythran import _build_system, _evaluate, _polynomial_matrix
ImportError: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required 
by /opt/conda/lib/python3.7/site-packages/scipy/interpolate/_rbfinterp_pythran.cpython-37m-x86_6
4-linux-gnu.so)
----------------------------------------------------------------------
Ran 4 tests in 0.000s
FAILED (errors=4)
Test failed: <unittest.runner.TextTestResult run=4 errors=4 failures=0>
error: Test failed: <unittest.runner.TextTestResult run=4 errors=4 failures=0>
```



A few thoughts:

1. When you run `export PYTHONPATH=$(pwd)` you should be in the repo root.
2. I see you're in Conda, which isn't in a virtualenv. (I can't find "env" anywhere in your snippet.) Maybe that's the issue?
3. I also installed the dependencies manually: `pip3 install dm_env ml_collections tensorflow immutabledict chex rx pygame`
4. Instead of running the tests, could you try to run `python3 meltingpot/python/human_players/play_clean_up.py`? Note that since it's a graphical program, you should run it when you can access the GUI interface, not through an SSH connection.

Hi @jkterry1 , 

I was able to install it in the way documented in this [very messy colab](https://colab.research.google.com/drive/1XpUY6MjhkN7az3c5eyqn2Aw8OglHFhzg?usp=sharing). I don't think this is the preferred way of doing things though..

Hi, apologies, we have been quite swamped the last few weeks.

I am making sure that the commands in the documentation work as listed. There is, indeed, an issue with the way things are set up at the moment.

My suggestion is that you clone the repo at a location of your preference, and then, instead of following the instruction in the README, you run the `install.sh` script that is at the root of meltingpot: https://github.com/deepmind/meltingpot/blob/main/install.sh

That sets up the VEnv, installs lab2d, and then meltingpot.

At the moment, running commands like `python3 meltingpot/python/human_players/play_clean_up.py` seems broken, but a workaround is to just pipe the script into a python interactive session. That is: `python3 <  meltingpot/python/human_players/play_clean_up.py` (notice the `<` in the command.)

Your latest error appears to be, like Ram said, an issue with dependencies... particularly _glibc_. So maybe make sure you have the right packages installed?

In particular, after running `install.sh`, I run this (on the directory including `install.sh`), and it works:

```
$ source venv/bin/activate
$ python3 < meltingpot/python/human_players/play_clean_up.py
```


@duenez 

So I was able to get the `setup.py test` script working with `pip install .`, the issue was some sort of ghost in the GCP deep learning image and I created one myself by installing the cuda stack on a fresh ubuntu image. Trying to call the library via third party like I had wanted to still didn't work though, with the same `ModuleNotFoundError: No module named 'meltingpot.python'` error.

As far as ./install.sh goes, I can't use that directly for my work because of the automatic venv creation which would conflict with my learning code, so I sat down and read it. I already have a working and tested install of lab2d. If I'm reading correctly, the install.sh file still only installs meltingpot via `pip install .`, just with extra testing afterwords and automated lab2d installation?

Ram's suggestion to export the path was able to finally get it to work on my new VM

I am closing this issue since there is a workaround. We are updating the documentation to advise other users how to fix it.