Repository: google-deepmind/meltingpot
Issue: PettingZoo and Stable-Baselines3 example code does not work
Root cause: Validation Gap

Hi,

When I tried to run `python3 sb3_train.py`, there were several errors in the original code.

- The `env_name` here should have two underscores between the environment name and suffix "open", which should be `commons_harvest__open`
https://github.com/deepmind/meltingpot/blob/bb11b0d999483bccdfb8c3488660b63821e42e50/examples/pettingzoo/sb3_train.py#L86

- The `__init__` function of the class `_MeltingPotPettingZooEnv` uses a wrong `build` function. I believe it should be `build_from_config` instead of `build`, and the correct argument should be `self._env = substrate.build_from_config(self.env_config,roles=self.env_config.default_player_roles)`
https://github.com/deepmind/meltingpot/blob/bb11b0d999483bccdfb8c3488660b63821e42e50/examples/pettingzoo/utils.py#L53

Even if I fixed mentioned typos, I still got errors as shown below. Any help on this will be appreciated!
> /home/denso/anaconda3/envs/meltingpot/lib/python3.9/site-packages/pettingzoo/utils/conversions.py:196: UserWarning: The base environment `_ParallelEnv` does not have a `render_mode` defined.
  warnings.warn(
Traceback (most recent call last):
  File "/home/denso/yuxin/meltingpot/./examples/pettingzoo/sb3_train.py", line 185, in <module>
    main()
  File "/home/denso/yuxin/meltingpot/./examples/pettingzoo/sb3_train.py", line 120, in main
    env = ss.frame_stack_v1(env, num_frames)
  File "/home/denso/anaconda3/envs/meltingpot/lib/python3.9/site-packages/supersuit/generic_wrappers/frame_stack.py", line 42, in frame_stack_v1
    return shared_wrapper(env, FrameStackModifier)
  File "/home/denso/anaconda3/envs/meltingpot/lib/python3.9/site-packages/supersuit/utils/wrapper_chooser.py", line 31, in __call__
    return self.parallel_wrapper(env, *args, **kwargs)
  File "/home/denso/anaconda3/envs/meltingpot/lib/python3.9/site-packages/supersuit/generic_wrappers/utils/shared_wrapper_util.py", line 81, in __init__
    self.add_modifiers(self.env.possible_agents)
  File "/home/denso/anaconda3/envs/meltingpot/lib/python3.9/site-packages/supersuit/generic_wrappers/utils/shared_wrapper_util.py", line 96, in add_modifiers
    self.observation_space(agent)
  File "/home/denso/anaconda3/envs/meltingpot/lib/python3.9/site-packages/supersuit/generic_wrappers/utils/shared_wrapper_util.py", line 85, in observation_space
    return self.modifiers[agent].modify_obs_space(self.env.observation_space(agent))
  File "/home/denso/anaconda3/envs/meltingpot/lib/python3.9/site-packages/supersuit/generic_wrappers/frame_stack.py", line 19, in modify_obs_space
    assert (
AssertionError: Stacking is currently only allowed for Box and Discrete observation spaces. The given observation space is Box([[[0 0 0]
  [0 0 0]
  [0 0 0]
  ...
  [0 0 0]
  [0 0 0]
  [0 0 0]]
 [[0 0 0]
  [0 0 0]
  [0 0 0]
  ...
  [0 0 0]
  [0 0 0]
  [0 0 0]]
 [[0 0 0]
  [0 0 0]
  [0 0 0]
  ...
  [0 0 0]
  [0 0 0]
  [0 0 0]]
 ...
 [[0 0 0]
  [0 0 0]
  [0 0 0]
  ...
  [0 0 0]
  [0 0 0]
  [0 0 0]]
 [[0 0 0]
  [0 0 0]
  [0 0 0]
  ...
  [0 0 0]
  [0 0 0]
  [0 0 0]]
 [[0 0 0]
  [0 0 0]
  [0 0 0]
  ...
  [0 0 0]
  [0 0 0]
  [0 0 0]]], [[[255 255 255]
  [255 255 255]
  [255 255 255]
  ...
  [255 255 255]
  [255 255 255]
  [255 255 255]]
 [[255 255 255]
  [255 255 255]
  [255 255 255]
  ...
  [255 255 255]
  [255 255 255]
  [255 255 255]]
 [[255 255 255]
  [255 255 255]
  [255 255 255]
  ...
  [255 255 255]
  [255 255 255]
  [255 255 255]]
 ...
 [[255 255 255]
  [255 255 255]
  [255 255 255]
  ...
  [255 255 255]
  [255 255 255]
  [255 255 255]]
 [[255 255 255]
  [255 255 255]
  [255 255 255]
  ...
  [255 255 255]
  [255 255 255]
  [255 255 255]]
 [[255 255 255]
  [255 255 255]
  [255 255 255]
  ...
  [255 255 255]
  [255 255 255]
  [255 255 255]]], (88, 88, 3), uint8)

Yes, unfortunately we don't use PettingZoo internally, and only just recently we added tests to the (RLLib) examples. It is on my radar to get this updated to Melting Pot 2.0, but if you know Petting Zoo well enough and feel like submitting a pull request with the fix, I'd be happy to review. The changes should be relatively straight-forward. You can base it upon the RLLib change: https://github.com/deepmind/meltingpot/commit/9e473f7eb449ab5dc9335a2fa36dd6fe9fff4f27

Also, parameterising the test like in https://github.com/deepmind/meltingpot/commit/5bb5c48b9ce286f76accc700d344e545a1b863b6 would be good so that it doesn't run for a hard-coded amount of time.

> Yes, unfortunately we don't use PettingZoo internally, and only just recently we added tests to the (RLLib) examples. It is on my radar to get this updated to Melting Pot 2.0, but if you know Petting Zoo well enough and feel like submitting a pull request with the fix, I'd be happy to review. The changes should be relatively straight-forward. You can base it upon the RLLib change: [9e473f7](https://github.com/deepmind/meltingpot/commit/9e473f7eb449ab5dc9335a2fa36dd6fe9fff4f27)

Has Petting Zoo with SB3 example code been tested? The RLLib examples works fine on my side, but the Petting Zoo example (I believe it is not using RLLib, but Stable Baseline 3 instead) is not functioning at this time. The error messages I received suggested that the Melting Pot wrapper for the Petting Zoo may be broken.

No, the example has not been updated to MeltingPot 2.0, that's why I'm linking the changes on RLLib that updated that example to MP 2.0. The PettingZoo & SB3 example was contributed, so I haven't gotten around to updating it.

Now I got it. Thanks for your reply!

I am curious, what do you use internally, RLLib, or is there another way ?

> Yes, unfortunately we don't use PettingZoo internally, and only just recently we added tests to the (RLLib) examples. It is on my radar to get this updated to Melting Pot 2.0, but if you know Petting Zoo well enough and feel like submitting a pull request with the fix, I'd be happy to review. The changes should be relatively straight-forward. You can base it upon the RLLib change: [9e473f7](https://github.com/deepmind/meltingpot/commit/9e473f7eb449ab5dc9335a2fa36dd6fe9fff4f27)



Internally we have our own distributed training system, custom made for MARL.

Sorry this was closed but the issue is still relevant, correct?
I can't run it because of `TypeError: unhashable type: 'ConfigDict'`.

I think one workaround to use [Shimmy](https://shimmy.farama.org/environments/meltingpot/)?