zhiminy/EvalEng huggingface dataset:
Repository: google-deepmind/meltingpot
Issue: ModuleNotFoundError: No module named 'examples'
Root cause: Documentation Deficiency

**Description:**
I am trying to get familiar with the Melting Pot environment in order to create my own environment. However, I encountered an issue while trying to run the rllib example provided in the repository.

**Steps to reproduce:**
1. Clone the Melting Pot repository.
2. Navigate to the root directory of Melting Pot: `cd <meltingpot_root>`
3. Install the required dependencies for rllib: `pip3 install -e .[rllib]`
4. Navigate to the rllib example directory: `cd <meltingpot_root>/examples/rllib`
5. Run the self_play_train.py script: `python3 self_play_train.py`

**Expected behavior:**
The self_play_train.py script should run successfully without any errors.

**Actual behavior:**
When running the self_play_train.py script, I encounter the following error:
```
File ".../meltingpot/examples/rllib/self_play_train.py", line 24, in <module>
    from examples.rllib import utils
ModuleNotFoundError: No module named 'examples'
```

**Additional information:**
It appears that the self_play_train.py script is unable to find the 'examples' module, leading to the ModuleNotFoundError. 

**Possible solution:**
A fix I found is running `export PYTHONPATH=$(pwd)` in the parent directory

**Further improvements:**
I think it is not clear in the Readme that this is needed for the rllib example as well. Or maybe there is a more convenient fix

The examples are set up to be run from the meltingpot root directory, so instead of:

```sh
cd <meltingpot_root>/examples/rllib
python3 self_play_train.py
```

You should run with:

```sh
cd <meltingpot_root>
python3 examples/rllib/self_play_train.py
```

If that doesn't work either, you can also take a look at [test-extras.sh](https://github.com/deepmind/meltingpot/blob/main/test-extras.sh) for how we run it. The issue is fundamentally that we don't install the extras into your python directory, and we don't assume anything about your general dev setup. Instead, we provide an example that could be run from your usual project root.
Repository: google-deepmind/meltingpot
Issue: Meltling pot Readme link sends to empty page

Hi !
In this part of the Readme : 
"PettingZoo and Stable-Baselines3
This example uses a PettingZoo wrapper with a fully parameter shared PPO agent from SB3.

The PettingZoo wrapper can be used separately from SB3 and can be found at [meltingpot_env.py](https://github.com/deepmind/meltingpot/blob/main/examples/pettingzoo/meltingpot_env.py)"

The last link point to a file that doesn't exist anymore.

The link should probably be replaced by : https://github.com/deepmind/meltingpot/blob/main/examples/pettingzoo/utils.py

But I think the complete example is outdated and doesn't work with Meltingpot 2.0, I don't yet poses the skills to fix it.

It has just been fixed 👍🏿
Repository: google-deepmind/meltingpot
Issue: Update documentation regarding dmlab2d
Root cause: Documentation Deficiency

> If there is no appropriate wheel (e.g. M1 chipset) you will need to install [dmlab2d](https://github.com/deepmind/lab2d) and build the wheel yourself (see [install-dmlab2d.sh](https://github.com/deepmind/meltingpot/blob/main/install-dmlab2d.sh) for an example installation script that can be adapted to your setup).

Hey, the new release of dmlab2d has wheels for M1 chipset too, can I update the documentation and remove the `install-dmlab2d.sh` script? 

On a similar note, would it be possible to build wheels for `meltingpot`?

Hello, we are in the process of cleaning this up on our end. Thanks for your patience.

Thanks for the heads up. I've updated the README.md. Not sure we can yet get rid of the `./install-dmlab2d.sh` since you currently need to know which wheel to install (you can't point `pip --find-links` or `pip --index-url` at a GitHub release).

We're looking into getting the dmlab2d wheels added to PyPi and then we'd be able to do likewise with MeltingPot.

I think, the install-dmlab2d.sh is still necessary for some versions of ubuntu.

Since `meltingpot` (and `dmlab2d`) is now `pip` installable I have removed the info on installing `dmlab2d` and moved `install-dmlab2d.sh` to be a `install.sh` script in `dmlab2d` itself.

If there are any problems with `dmlab2d` please raise them at https://github.com/deepmind/lab2d/issues
Repository: google-deepmind/meltingpot
Issue: Support pip installation of meltingpot
Root cause: Environment Incompatibility

Once dmlab2d is pip installable (https://github.com/deepmind/lab2d/issues/20), we can make Melting Pot pip installable.

Just FWIW I tried the install dmlab2d script just now and it failed on my intel mac due to some xcode error, but I have dmlab2d installed in a previous virtualenv so I know it's possible to install. Also if you re-run the script it fails because the directory already exists, I think you can do git clone -p or something similar to clone only if it doesn't exist. Pip installation would make these both irrelevant though so I think that's probably the best thing to do, just letting you know in case there are blockers and it can't get done.

Tested out the pip wheels on my machine and with shimmy and they seem to have some python path issues: 
```
import meltingpot.python
E   ModuleNotFoundError: No module named 'meltingpot.python'
``` 
(https://github.com/Farama-Foundation/Shimmy/actions/runs/5618125745/job/15223240313)

I am assuming there are some bugs since the pypi wheels are set for pre-release, just letting you know in case this info helps. I'm also not sure if this is a bug or if I'm just using it incorrectly, but after installing dm_meltingpot if I try to use it with python I get ModuleNotFound errors when trying to do `import meltingpot` or `import dm_meltingpot`. 

Installing it through our [dockerfile](https://github.com/Farama-Foundation/Shimmy/blob/70791c883ce72023c765eac5eb1897002694a091/bin/meltingpot.Dockerfile) allows us to do `import meltingpot` but not `import meltingpot.python` (as shown above). 

The bizarre thing is, two days ago when we ran CI it all worked fine, but now suddenly things aren't working. I think I may just pin the git clone command in the dockerfile to a specific commit from a few days ago and test that out, but maybe you would know specifically what recent changes may have caused this issue. Python 3.10 seems to be required now but I've changed the versions in that dockerfile accordingly.

Hi, I recently made it so it's just `import meltingpot` so you shouldn't need `import meltingpot.python`. I plan to drop a new version soon and document all this before pushing a non-dev release to PyPI. Perhaps I should try to keep `import meltingpot.python` working for backwards compatibility though...

Done! See https://pypi.org/project/dm-meltingpot. Please reopen this if there are any problems.

NOTE: you might need `SYSTEM_VERSION_COMPAT=0` on macOS.
```
SYSTEM_VERSION_COMPAT=0 pip install dmlab2d
```

Hi, I seemed to have minor python path issues. I passed all the `pytest --pyargs meltingpot` test cases, so the manual install should be OK. 

My issue was with running the RLlib example: `python self_play_train.py` gives a `ModuleNotFoundError: No module named 'meltingpot'` error. Not too sure if I just needed to set PYTHONPATH/tweak the import or if this was suggestive of a bigger issue? The same sort of thing happens when I try the human_players scripts.

`pip list` also shows that `meltingpost v1.0.0` is installed.
Repository: google-deepmind/meltingpot
Issue: devcontainer installation
Root cause: Environment Incompatibility

When installing `meltingpot` using .devcontainer, an error pops:
`ERROR: dmlab2d-1.0-cp39-cp39-manylinux_2_35_x86_64.whl is not a supported wheel on this platform.`
After some googling, I found out that `2_35` wheel tag is compatible with ubuntu22.04 and later ([source](https://github.com/mayeut/pep600_compliance)). 

Is there a way to use pip wheels in ubuntu20.04, or version in the dockerfile should be upgraded to 22.04?


Thanks for the heads up, I'll update the Dockerfile.
Repository: google-deepmind/meltingpot
Issue: Accessing environment state
Root cause: Documentation Deficiency

Hi, 

I have managed to train a population of agents using your Rllib tutorial as a basis (many thanks for that). During the evaluation, I would like to collect some more detailed information about an environment's state, so I can better compare the agent's behaviours outside of the base return. Is there some way I can access the state of individual tiles + other environment properties via [`MeltingPotEnv`](https://github.com/deepmind/meltingpot/blob/main/examples/rllib/utils.py#L34)?

Thanks!

A concrete example problem I would like to solve: in the [Capture the Flag](https://github.com/deepmind/meltingpot/blob/main/docs/substrate_scenario_details.md#capture-the-flag) environment, how can I count the number of tiles coloured red? 

Hello,

The best way to extract (custom) information from a substrate is to push them as events. This is assuming you don't need them for agent observation (as observations require more machinery). This means you need to edit the Lua (unless we already have a component in Lua that exposes what you want).

In the case of your example for CTF, probably the easiest thing to do would be to add a component to the Scene object, and in that component you have an `update` function iterates over all objects (checking if they) that contain a `Ground` component (https://github.com/deepmind/meltingpot/blob/22f0a448fdcc19f0f9a1eb115c95f923900a0f8c/meltingpot/lua/levels/paintball/shared_components.lua#L80), then you emit an event counting how many of those are in state `blue` and how many in state `red`. 

Something like:

```python
local Ground = class.Class(component.Component)

function GroundCounter:__init__(kwargs)
  kwargs = args.parse(kwargs, {
      {'name', args.default('GroundCounter')},
  })
  GroundCounter.Base.__init__(self, kwargs)
end

function GroundCounter:update()
  local grounds = self.gameObject.simulation:getAllGameObjectsWithComponent('Ground')
  local counts = {
    clear = 0,
    red = 0,
    blue = 0,
  }
  for _, ground in pairs(grounds) do
    counts[ground:getState()] = counts[ground:getState()] + 1
  end
  events:add('grounds', 'dict',
               'red', counts['red'], 
               'blue', counts['blue'])
end
...

local allComponents = {
    ...
    GroundCounter = GroundCounter,
    ...
}
```

Then, you would add this component to the Scene object config: https://github.com/deepmind/meltingpot/blob/22f0a448fdcc19f0f9a1eb115c95f923900a0f8c/meltingpot/python/configs/substrates/paintball__capture_the_flag.py#L529

And you can now get the event by checking the substrate (as a python object) `events()` call, or by registering an observable on the `events` channel (similar to how you can register renderers for the third person view `WORLD.RGB`)

Hey @duenez,

Thanks for the explanation and sample code. I was kind of afraid that I would have to dig into the lua code. I would still prefer to add this to the `info` returned by the environment, but I'll see if I can bake the observable into the `MeltingPotEnv`.
Repository: google-deepmind/meltingpot
Issue: How to handle local communication
Root cause: Documentation Deficiency

Hello, 

I'm using a previous version of Meltingpot and in the substrate configuration file, there was the following code:
```  
config.individual_observation_names = [
      "RGB",
      "READY_TO_SHOOT",
      # Debug only (do not use the following observations in policies).
      "POSITION",
      "ORIENTATION",
  ]
  config.global_observation_names = [
      "WORLD.RGB",
  ]
```
I saw that you removed the debug observations to speed up the stepping. 
I would like to use an RL algorithm with local communication, and to do so, I need to check the relative position/distance between two agents.
Is there a proper way to obtain this information?

Is what you really want a list of players near the focal player on each timestep? You can directly report that data as an observation  without using the absolute positions. The way to do it is to add a component on the avatar that runs a query for avatar objects in a rectangle (or whatever shape you like) around the focal avatar on each step. Then add an observation that reports this list out of the environment.

If you do want the absolute positions then just add the component that reports them and turn on the observations (basically undoing the commit that removed them).

Ok everything clear, thanks for the answer
Repository: google-deepmind/meltingpot
Issue: Proposal: do something about redundantly large observations
Root cause: Architectural Constraint

The observations are large images due to spriteSize=8. However, most of this information is redundant. In the rllib example, a convolutional layer is applied with stride 8 to leave only the first pixel out of 8x8=64 pixels, meaning that observations are x64 bigger than needed. This might be fine for on-policy methods, but for methods that use replay buffers, this transforms several gigabytes of memory into several hundred gigabytes. I had to change spriteSize=8 to spriteSize=1 in the config files of specific environments which is fine but not the most elegant (or intended) solution.

As far as I understand, spriteSize=8 is only useful for rendering. So a good solution to this issue could be to use spriteSize=1 everywhere but transform small images into bigger images as a subroutine during rendering. A worse but maybe easier solution would be to allow users to change spriteSize in the config (and also explain this parameter in the examples).

(this is based on the state of the repo a couple of weeks ago, I don't know if anything changed)

Because the test scenarios have pretrained agents with spriteSize 8, we shouldn't change that. I agree that for many solutions, collapsing the 8x8 sprites to 1x1 is reasonable. That is what we are recommending as an entry-level approach at training agents in the MeltingPot Competition coming up. However, changing this in the config isn't the best. So, what we think is the best option is to write a substrate wrapper that just averages (i.e. with numpy convolutions) the 8x8 sprite.
Repository: google-deepmind/meltingpot
Issue: Incorrect `is_focal` in focal-vs-background scenarios.
Root cause: Algorithmic Error

Affected scenarios:
-   `capture_the_flag_0`
-   `capture_the_flag_1`
-   `king_of_the_hill_0`
-   `king_of_the_hill_1`
-   `king_of_the_hill_2`

Previously, we set `is_focal = (True,) * 4 + (False,) * 4` for focal-vs-background scenarios. This mistakenly lead to focal players filling 2 player slots on one team and 2 player slots on the other. Since CTF is a zero-sum game this mean that the focal per-capita return was guaranteed to be zero.

This was fixed in 01e2ab857d6b455e8f078154d9fe576c7fed6858. There we set `is_focal = (True, False,) * 4` to ensured that a team of 4 focal players plays against a team of 4 background bots, as intended.

(NOTE: this didn't affect our experiments run for the Melting Pot paper since these weren't run using Scenario.)
Repository: google-deepmind/meltingpot
Issue: Collaborative cooking pseudoreward
Root cause: Architectural Constraint

As mentioned in #39 it is difficult to get learning off the ground in collaborative cooking without pseudorewards. I see how to set the cooking pot pseudoreward to a non-zero value by editing the default config: https://github.com/deepmind/meltingpot/blob/f7905f0d41e4c351913c8b9af446ef4aaf12f680/meltingpot/configs/substrates/collaborative_cooking.py#L908 

Two questions though:
1. Is it possible to set this value without editing files inside the meltingpot package, e.g. can I pass it in through an environment configuration dictionary or similar?
2. Is it possible to change this value after the environment has been created? That would be useful for instance to implement curriculum learning using a callback from rllib.

Thanks so much!

**TL;DR** We don't want to expose parameterised configs, but there are easy ways to get what you want.

Unfortunately with the current way we do things, this is not directly possible. The reason being that we have substrate factories that don't forward arguments to individual substrates `get_config` function. So, we could certainly offer a parameterised collaborative cooking `get_config`, but it would not be used by the factory.

There's good reason for this level of indirection, though: We want it to be very hard to get the wrong config if you are using our factories.... nothing to configure, so you are very sure you are having the canonical config and all comparisons of results are valid.

If you know what you are doing, we still expose everything out so you can simply use the configs and substrates directly for research purposes.

I suggest you create a custom function that bypasses the factory, and that overwrites the value of the pseudorewards directly. That way, things are completely explicit. For example:

```python
from meltingpot.configs.substrates import collaborative_cooking__ring
from meltingpot import substrate

def get_collab_cooking_with_pseudorewards():
  config = collaborative_cooking__ring.get_config()
  with config.unlocked():
    config.cooking_pot_pseudoreward = 1.0
    config.lab2d_settings_builder = collaborative_cooking__ring.build
  return config

config = get_collab_cooking_with_pseudorewards()
roles = config.default_player_roles
env = substrate.build_from_config(config, roles)
```

Hope that helps.



Yes, that does help, thank you! Is there a way of changing the pseudoreward after the environment has been created? 

No, once the environment is created, the game objects that calculate the rewards (and pseudorewards) have been finalised. In principle it would be possible to use the `properties` library to change this in Lua from Python but I would _strongly_ advise against it.

Got it, thank you! And not a huge problem, I think we can just periodically re-create a new environment during training if we want to decay the pseudoreward. Thank you!
Repository: google-deepmind/meltingpot
Issue: Running ACB Baseline
Root cause: Unimplemented Feature Gap

How can I run the ACB baseline? self_play_train.py runs PPO and says "The ACB models used as baselines in the meltingpot paper were not run using rllib". I would like to run ACB on a modified version of cleanup. Thanks

We don't have an open source implementation of the ACB baseline, and it would be hard to open source, as it relies on significant internal tooling and infrastructure. We anticipate some async actor critic methods will be provided as part of the [Melting Pot Competition](https://twitter.com/coop_ai/status/1664270352435015683). Those will not be our internal implementation, but will be good reference points for open source RL baselines on Melting Pot. So, stay tuned! :)
Repository: google-deepmind/meltingpot
Issue: Changing the reward spec of an existing environment? (Clean Up)
Root cause: Architectural Constraint

I am trying to manipulate clean up to have a vectorised reward and have been able to do so by changing `meltingpot/lua/levels/clean_up/components.lua` and `meltingpot/python/configs/substrates/clean_up.py` . I made the reward type `tableType` but the issue is that I need to change the reward spec (since it crashes with `Check failed: double_tensor != nullptr [observation] - Must return a contiguous DoubleTensor or number while reading: '1.REWARD'`) but changing the shape in `meltingpot/python/utils/substrate/specs.py` has no effect. Please advise. 

This is tricky, we tend not to support rewards that are not scalar. The assumption that rewards are scalar is baked into the `Avatar` component. However, what you want to achieve is easy in principle. You are able to expose any custom observation, for instance as a vector (or even a matrix or tensor). Take a look at how we handle inventories in `*_in_the_matrix` substrates, for instance. Then just use that observation as your vectorised reward in your algorithm.

Hope that helps.

Also note that any custom reward won't be available at test time (i.e. in the scenarios) so best to not condition your policy on that (though your critic can do whatever it likes).

Thank you! Making the vectorised reward a custom observation worked :D
Repository: google-deepmind/meltingpot
Issue: Field of view visualisation
Root cause: Documentation Deficiency

Hi!

I am working to replicate the results from the Hidden Agenda paper, and was wondering if there was the best way to add field of view visualisation to the renders? I think this would be very useful for evaluation, to see what the agents can focus on.

Currently, my best guess is to modify the WORLD.RGB observation to add borders or a tint to the agents' FOV. Is there a better way?

Thanks!

Hello,

I'm not sure I completely understand what you mean. Do you mean that you want to see what agents can see, but only on the `"WORLD.RGB"` observation, and not for learning? If that's the case, you could achieve this with direct post processing. Since you can extract position and orientation of all agents (using debug observations, disabled by default), you can then take the WORLD.RGB and add borders or tint, like you suggest.

At some point we were thinking about implementing a view system that would respect walls (i.e. not being able to see beyond a wall bend), but I don't think that's what you are talking about, right?

Yes, I would like to be able to highlight what the agents can see on the `WORLD.RBG` observation, similar to the figure below.

![example](https://github.com/deepmind/meltingpot/assets/5785856/cee86441-4b3b-45d0-a919-f403ce2d2f8d)

Thank you for the pointers with the debug observations! I was hoping there would be an easier way to do this, to not slow down the simulation by having more observations being added.

And yes, the walls are not an issue for me.

Thanks you!
Repository: google-deepmind/meltingpot
Issue: test method depricated
Root cause: Documentation Deficiency

When I run the command to test my installation, I got this error:

```
jkterry0@instance-3:~/meltingpot$ python3 setup.py test
running test
WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.
/opt/conda/lib/python3.7/site-packages/setuptools/installer.py:30: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
```

This should likely be changed accordingly

Thanks for the heads up.

I'll update the `README.md` to use:
```
pip install nose
nosetests meltingpot
```
As we do in `install.sh`.
Repository: google-deepmind/meltingpot
Issue: Modifying the 'commons_harvest' Scenario During Training Episodes
Root cause: Documentation Deficiency

Hello, I'm working with Melting Pot, specifically with the `commons_harvest` scenario. I'd like to know if there's a way to modify the scenario during the training stage? I mean, is there a way for the agent to see a different map than the initial one in each training episode?

Sure, you can modify the substrate during training. The changes won't be there at test time when you use the 'scenario' though. 

If all you want to do is change the substrate during training then it should be fine. You can change anything you like in the substrate config between episodes, this includes the map.

Thank you for the response @jzleibo. I'd like to know if there's an example in the repository's documentation on how to carry out this procedure. Is there already something implemented where the agent sees different maps during training?

Somewhere in your experiment you'll be calling the 'build' function in one of these files: 
https://github.com/google-deepmind/meltingpot/blob/main/meltingpot/configs/substrates/commons_harvest__open.py
(that's the one for commons_harvest__open). Most likely you'll be calling it on every episode. 

You can take the dict d returned from that function and override d['simulation']['map'] with your map.