Repository: google-deepmind/meltingpot
Issue: Can't run examples/pettingzoo/sb3_train.py
Root cause: External Dependency Breakage

Hi,
I installed the environment according to the examples/requirements.txt, but an error occurred.
`stable_baselines3            2.3.2
SuperSuit                    3.7.2
PettingZoo                   1.22.3
`

Traceback (most recent call last):
  File "/root/autodl-tmp/Benchmarking-Meltingpot/examples/pettingzoo/sb3_train.py", line 29, in <module>
    import supersuit as ss
  File "/root/miniconda3/envs/mp/lib/python3.10/site-packages/supersuit/__init__.py", line 1, in <module>
    from supersuit.generic_wrappers import clip_actions_v0  # NOQA
  File "/root/miniconda3/envs/mp/lib/python3.10/site-packages/supersuit/generic_wrappers/__init__.py", line 12, in <module>
    from .delay_observations import delay_observations_v0  # NOQA
  File "/root/miniconda3/envs/mp/lib/python3.10/site-packages/supersuit/generic_wrappers/delay_observations.py", line 4, in <module>
    from .utils.shared_wrapper_util import shared_wrapper
  File "/root/miniconda3/envs/mp/lib/python3.10/site-packages/supersuit/generic_wrappers/utils/shared_wrapper_util.py", line 4, in <module>
    from pettingzoo.utils import BaseParallelWrapper
ImportError: cannot import name 'BaseParallelWrapper' from 'pettingzoo.utils' (/root/miniconda3/envs/mp/lib/python3.10/site-packages/pettingzoo/utils/__init__.py)


I can solve the problem by installing lower versions of SuperSuit, stable_baselines3, and PettingZoo.
`PettingZoo==1.14.0
stable_baselines3==2.4.0a7
SuperSuit==3.3.2
`
But When I run the code, It seems that multi-core CPUs and multiple environments have little effect. 12 CPUs and 12 envs almost have the same performance with 1 CPU and 1 env.
Could you please provide more complete requirements.txt?

By the way, I am trying to use sbx(sb3 + jax) to make the pettingzoo framework to be faster, I don't know whether you would do the similar work. Thank you so much.

Even as I use the same libraries as the user above I'm not able to run the examples due to:

```
  File "dev/meltingpot/meltingpot/configs/substrates/__init__.py", line 60, in get_config
    if substrate not in SUBSTRATES:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: unhashable type: 'ConfigDict'
```


I've fixed the previous error but now got:

```
  File ".../python3.11/site-packages/supersuit/generic_wrappers/frame_stack.py", line 17, in modify_obs_space
    assert False, "Stacking is currently only allowed for Box and Discrete observation spaces. The given observation space is {}".format(obs_space)
AssertionError: Stacking is currently only allowed for Box and Discrete observation spaces. The given observation space is Box(0, 255, (88, 88, 3), uint8)
```

Which seems like a much more broken state than I was expecting.
It seems like the best way to go about meltingpot is through the layer of compatibility, Shimmy. However, that also offers its own problems, for example, it seems far to being completely integrated with the rest of the PettingZoo ecosystem. 

Do we know of anyone who ever successfully used pettingzoo? We've never tried it or tested it ourselves. My inclination would simply be to remove support for it, and maybe for all the training examples. It's not clear what value they add, we've never used them.

Hey @jzleibo, it works for me through the Shimmy layer of compatibility.
But not through the examples in this repo.
I also just ran the RLlib example and I think it doesn't work anymore. I've updated a few of the deprecated functions to current ones but fell into more complicated errors as well.
I'm currently trying to figure out if I should use the PettingZoo layer at all in my project or if I should just stick to the vanilla MeltingPot + my own implementation of PPO, etc. It seems like it would be nice to use the helper functions implemented in SuperSuit but I'm not sure they're worth all the trouble of having to patch up the functionalities that are not covered by PettingZoo e.g. [observations aren't propagated properly](https://github.com/Farama-Foundation/Shimmy/issues/128).
Let me know if you have any inputs here! That'd be very helpful.
Thanks!

Sounds like a much better idea to stick to the vanilla MeltingPot + your own implementation of PPO. 

It was probably a bad idea on our part in the first place to have included the training examples. They seem to be more trouble than they are worth, given that we don't use them ourselves (we have our own RL code internally). 

For PettingZoo in particular, I have no idea if the PettingZoo example ever really worked. I only looked at it once when it was first added, and was very skeptical at that time that it could work well, since it looked to me like the only RL library it was connected to on the other side was one that did not support recurrent networks (no idea if this is still true or not). And no one ever showed me an example training curve produced with PettingZoo... So I never got any evidence that it was working in all this time. That, combined with the fact that it's apparently broken now, seems like a good enough reason to delete it I think.


Aha well, that's helpful!
Thank you @jzleibo. Let me know, I'm happy to remove them and update the requirements files/installation instructions for them.

thanks @marimeireles, it's probably easiest if we delete it on our end. Thank you for bringing this to our attention!