zhiminy/EvalEng huggingface dataset:
Repository: google-deepmind/meltingpot
Issue: Possible Bug: Negative Probability with default config in cleanup
Root cause: Algorithmic Error

In https://github.com/google-deepmind/meltingpot/blob/main/meltingpot/lua/levels/clean_up/components.lua#L64 the probability is negative due to the interpolation calculation so apples never spawn.

I guess technically apples would never grow until the level of dirt is decreased so it serves its purpose

Yes, that's the idea.
Repository: google-deepmind/meltingpot
Issue: Help Request: Decreasing size of local observations
Root cause: Documentation Deficiency

I am working in a scaled-down version of cleanup to generate quick baselines. Please advise on how I can decrease the local observation size to be smaller than an 11x11 window

I was able to do this via clean_up.py config

For posterity, you can change the avatar view parameters: https://github.com/google-deepmind/meltingpot/blob/main/meltingpot/configs/substrates/clean_up.py#L696

However, doing so will make your agent incompatible with test scenarios. A more principled way would be to add a wrapper to your agent so that it would do this post-processing on its observations, that way it could also be placed in a scenario without problems. Something like meltingpot/utils/substrates/wrappers/base.py for a substrate at train time and an equivalent policy wrapper at test time.
Repository: google-deepmind/meltingpot
Issue: Use extras in setup.py
Root cause: Architectural Constraint

Right now, setup.py includes tensorflow as a main dependency. The problem with specifying tensorflow as a main dependency is that if you're trying to use jax or pytorch with the environments alone its an unnecessary dependency that can be problematic to install (which is what I'm doing), and even if you want to use tensorflow you often want to custom install tensorflow-gpu or build it from source or something. This in turn means that in these reasonably common cases, you have to use --no-deps on installation and do everything manually, which is bad practice and impedes reproducibility. The standard way other libraries handle situations like this (gym, rlcard, etc) is to use the extras feature in setup.py files, such that only the dependencies files for the environments are installed by default, and extra arguments can be passed for other decencies (you could call them examples, tests, etc.)

Hello, thank your for you message and suggestion. Yes, we could make tensorflow an `extras` dependency, but it is likely required for SavedModel, and thus, to run the scenarios. You are right that we likely don't need it just for training on substrates.

We'll evaluate this and get back to you soon.

Thanks!

`tensorflow` is required in `install_requires` as it is needed for the bots that are part of the scenarios.

As a workaround I suggest editing/removing "tensorflow" in the `install_requires` of your local copy of setup.py.

Sorry meant to ask if that workaround would work for your use case?

We'd prefer both substrates (training envs) and scenarios (test envs) to work on a simple install, but if you have a substrate-only use case perhaps we can have some optional minimal install without tensorflow?
Repository: google-deepmind/meltingpot
Issue: Why `python_requires>=3.10`? 
Root cause: Environment Incompatibility

Hi, is there a particular reason you only support >=3.10?
What is the limiting factor for lower versions?

Thanks, mark

We cannot test 3.9 or below internally, so we don't have a way to ensure compatibility anymore. You are free to try it on your side, but you'd have to build from source.

Fair, thanks for answering quickly
Repository: google-deepmind/meltingpot
Issue: Disable common.log call on env.reset()
Root cause: Configuration Error

Currently, env.reset() calls:
https://github.com/deepmind/lab2d/blob/main/dmlab2d/__init__.py#L77
which in turn calls:
https://github.com/deepmind/meltingpot/blob/main/meltingpot/lua/modules/base_simulation.lua#L432
which prints the entire grid to the console.

This is making learning on the meltingpot environments extremely hard to read and debug. Is there any way to disable this or move it to a lower logging level? I'd be happy to create a PR for the same.

Closing issue in favour of #20
Repository: google-deepmind/meltingpot
Issue: How to unifiedly detect sprites in the observation overall substrates?
Root cause: Documentation Deficiency

Hi, I am planning to generate a text-based description for RGB-based observation. To do this, I need to know the type of all sprites. For example, given a [88, 88, 3] observation, which contains 11x11=121 sprites, I wanna know what type it is for each sprite, i.e. classify it as "agent", "wall", "water", "apple", "sand", etc.  Can you give me some advice for achieving my goal?  Thanks a lot!

You probably would be better off using the LAYERS observation for this. This was discussed before https://github.com/google-deepmind/meltingpot/issues/67. However, sometimes we put some special and invisible objects in the world, so this method would violate those assumptions that they are invisible.

Another possibility would be to just add a custom observation in plain text. This is possible creating some Lua components, and we've had some success doing this internally.

Yeah, don't use the LAYERS observation, we have not documented its dynamics (and they could be extremely confusing).\

As Edgar said, the best way to do this is by adding custom observations.  You'll have to read through the tutorial docs on substrate creation to see how to do this. If you still have questions after that then let us know, we would be happy to help.

Thanks a lot! I will read tutorial docs on substrate creation and try to add custom observations to generate plain text for the original observations. And i will ask you if i still meet problems.

Hi @jzleibo @duenez , recently I tried to read the source code for the Melting Pot. I am curious about the spacial relation of prefabs, i.e. the definition of "upperPhysical", "overlay" and "supperOverlay" and their mechanism for aggregating RGBA pixels. I wanna calculate the combined pixel results for the given two prefabs. For example, in Clean Up substrate, I can directly calculate "AVATAR_W + SAND" since the RGB color of AVATAR_W will entirely overwrite the background prefab's color so that one can get the sprite if one knows their ASCII map and corresponding color Platte. However, I cannot calculate the correct result for "SAND + SHADOW_N + SHADOW_E" since some prefabs' layer combination rule is "upperPysical" which seems like I need to know the fusion mechanism with some RGBA pixels. In addition, I don't find the relative code about this. Can you tell me how to obtain the combined image given two or more prefabs' ASCII maps with color platte( which is defined in shape.py and xxx_substrate.py) and layer relation?

The question of how precisely to fuse the RGBA values for sprites on different layers at the same x,y position is really a question about the underlying engine: dmlab2d. Maybe try asking it there https://github.com/google-deepmind/lab2d
Repository: google-deepmind/meltingpot
Issue: Jax-related
Root cause: Unimplemented Feature Gap

Are there any ways to natively wrap an environment to be suitable for running a rl algorithm with jax (make the environment stateless), or is it planned to add such support?

Thank you!

Hello,

No, it would be very challenging to make the environments stateless and implemented exclusively in JAX. That said, there's no reason why you couldn't use JAX for the agent side, and that's what we do internally. We use something similar to the A3C setup where there are processes stepping the environment in CPU only, and the agent is doing inference and learning on the accelerator (GPU/TPU).

By the looks of things, it the only feasible variant right now. Thank you for the response. Closing the issue.
Repository: google-deepmind/meltingpot
Issue: Environment with more social interactions

Dear developer team,

Thank you very much for the effort to maintain this wonderful repository. It is very useful for social AI research. I am wondering if there exists a single environment that can contain all kinds of social interactions. such as cooperation, conflict, competition, coercion, and exchange).

Hello, there is a matrix with the aspects we think each substrate has in the Melting Pot 2.0 tech report: https://arxiv.org/pdf/2211.13746 Figure 2

Closing. Feel free to re-open if you have more questions.
Repository: google-deepmind/meltingpot
Issue: pickling error when multiprocessing for batched continuation of environments
Root cause: External Dependency Breakage

Hi All!

Thanks for the amazing melting pot environments. I am trying to train agents using jax on the environments. To enable this, I keep a batch of environments - which is just a list of let's say 32 environments - and I call the step function on all of them in a for loop. To make it a bit faster I wanted to call the step function for each one using multiprocessing. But I got `cannot pickle 'dmlab2d.dmlab2d_pybind.Lab2d' object`. I am wondering if you have suggestions for me for what I want to do. 

No good suggestions sorry.

Looks like you'll have to find a way to avoid pickling `Lab2d` objects. Perhaps try just pickling the substrate name and have the processes spin up the environment themselves?

If you want `Lab2d` to support pickling you'll need to make a feature request at https://github.com/google-deepmind/lab2d. I expect they won't have time for this I'm afraid.

Oh, I see. I just cannot make the process spin up its own environment as I want agents to interact with a batch of environments back and forth. Thanks for the swift reply. 

Haven't played with this myself, but I think maybe you can set up one process for each environment that constructs the `Lab2d` itself and is communicated with by the main process only via its `env.step(actions)`. Then I think there will be no need to serialize the `Lab2d` (only the actions/timestep).
Repository: google-deepmind/meltingpot
Issue: Conversion from dm_env to gym space
Root cause: Algorithmic Error

Hi all!

I've spotted a small issue in `examples/rllib/multiagent_wrapper.py` and specifically in the `_spec_to_space` function.

https://github.com/deepmind/meltingpot/blob/ef3353dc4eee23d8a4b473b63dd189ac4612bd48/examples/rllib/multiagent_wrapper.py#L53-L56

In this part of the code, `spec` is first checked if it's an instance of a `BoundedArray` and then `DiscreteArray`. However, from dm_env, DiscreteArray is a subclass of BoundedArray:
```python 
class DiscreteArray(BoundedArray):
```
That means when comparing a DiscreteArray, the first check will be true, and the function returns a Box (instead of Discrete). 

I encountered this when converting the action space of `allelopathic_harvest` but I assume that's the case for the rest of the environments.

The obvious solution would be to swap the if statement, but I am not using rllib so I can't say for sure if it would break something. I am surprised it works in the first place. Maybe rllib knows to convert an integer Box space to a discrete space (?).



Thank you, this should be fixed at head, now!
Repository: google-deepmind/meltingpot
Issue: evaluate library is missing
Root cause: Documentation Deficiency

Hi All! Thanks for the great repository. 
I wanted to point out that the link in the readme file pointing to the evaluation script is broken and I cannot find how to evaluate a saved model. Would you mind checking this out?

Thanks! I'm updating the link to point to the right place: https://github.com/google-deepmind/meltingpot/blob/main/meltingpot/utils/evaluation/evaluation.py
Repository: google-deepmind/meltingpot
Issue: Accessing background returns for evaluation results
Root cause: Unimplemented Feature Gap

I am interested in looking at the collective rewards achieved by the exploiter MAPLA agents that were trained for the Melting Pot 2.0 report. In the `evaluation_results.ipynb` notebook, the results are downloaded from the path `https://storage.googleapis.com/dm-meltingpot/meltingpot-results-2.1.1.feather`, which contains the focal per capita returns for the exploiters, but not the background per capita returns for the agents they were trained against. Would it be possible to access the data so that I can make evaluations of the collective returns achieved in these scenarios?

This would be good to have. Unfortunately, I don't think we actually saved this information.
Repository: google-deepmind/meltingpot
Issue: Alternative input representations to RGB
Root cause: Documentation Deficiency

Is it possible to get an alternative input representation? For example, rather than an RBG input, to have one-hot matrices encoding features such as square contains:
- Nothing
- Apple
- Agent facing N
- Agent facing S
- etc

I am not familiar with lab2d so I wouldn't know how to do this.

You would have to add custom observations in the Lua for whatever information you are looking to pull out. There are lots of examples of how to add custom observations in the existing substrates. You can follow those examples.  Take a look at Allelopathic Harvest, I believe that one has quite a few implemented.  You'll probably find this tutorial helpful: https://github.com/google-deepmind/meltingpot/blob/main/docs/substrate_tutorial/index.md

Btw, this is a meltingpot question, not a lab2d question. Also: if you do look into lab2d (the underlying engine) you may come across the "layers" representation, which will sound like what you want. But for meltingpot it will give misleading and wrong answers. Don't use it. The right way to get new custom observations is to add them in the same way we add all the custom observations.

Closing. Feel free to re-open if you have more questions.
Repository: google-deepmind/meltingpot
Issue: new pip release with chex fix
Root cause: External Dependency Breakage

Could you make a new meltingpot release, 2.2.1 with the [chex version requirement removal](https://github.com/google-deepmind/meltingpot/commit/3019726e6d1ce203ac7c0229360df2e8ae3a8296), thanks

Fixed in v2.2.2

Thanks, much appreciated
Repository: google-deepmind/meltingpot
Issue: Lua 5.2 causes tests to fail
Root cause: External Dependency Breakage

e.g. https://github.com/deepmind/meltingpot/actions/runs/1999939516

Until this is fixed better to use Lua 5.1 rather than Lua 5.2. This may have an impact on building Melting Pot on Apple M1 chips, see https://github.com/deepmind/meltingpot/issues/13.

Specifically the error is:

```
self = <dmlab2d.Environment object at 0x15f0bfa90>
    def reset(self):
      """See base class."""
      self._reset_next_step = False
>     self._env.start(self._next_episode, seed=self._rng.randint(0, 2**31))
E     ValueError: Failed to start: 
E     ERROR: ./meltingpot/lua/modules/component_library.lua:261: Error: Must call Transform:start(...) before Transform:getPiece().
E     stack trace-back:
E     ERROR: [C]: in function 'assert'
E     ERROR: @./meltingpot/lua/modules/component_library.lua:261:
E     ERROR: @/Users/runner/work/meltingpot/meltingpot/meltingpot/lua/levels/commons_harvest/components.lua:143: in function 'start'
E     ERROR: @./meltingpot/lua/modules/game_object.lua:423: in function 'func'
E     ERROR: @./meltingpot/lua/modules/game_object.lua:160: in function '_doOnAllComponents'
E     ERROR: @./meltingpot/lua/modules/game_object.lua:420: in function 'start'
E     ERROR: @./meltingpot/lua/modules/base_simulation.lua:465: in function 'start'
E     ERROR: @./meltingpot/lua/modules/api_factory.lua:100:
```
Repository: google-deepmind/meltingpot
Issue: Guidance for Creating Avatars with Different Goals
Root cause: Documentation Deficiency

Forgive me for the beginner question as I am still pretty new to MeltingPot in general. My aim is to create two agents that have their own respective targets that they have to reach. Currently, it seems that all agents copy one specific avatar to move towards the same goal, thus the possible actions of every agent are also the same. How do I set up agents such that each agent has a different goal and maybe even different actions than other agents? Is this possible?

As a very simple example, I would like to take the harvest tutorial game, make some of the apples red, initialize two agents, and have one agent collect red apples while the other collects green. 

Hello,

It sounds like you are creating a custom config for your substrate (that's perfectly fine and supported :) ). If that's the case, you probably are using `meltingpot.utils.substrates.substrate.build_substrate` (https://github.com/google-deepmind/meltingpot/blob/main/meltingpot/utils/substrates/substrate.py#L107). If that's the case, you can create your avatars directly as game objects in the lab2d_settings you pass to the builder. Then you have full control over what their goal is (or any other configuration like changing capabilities, actions, observations, etc.

So, this would look like:

```python
from meltingpot.configs.substrates import commons_harvest__open
from meltingpot.utils.substrates import substrate

config = commons_harvest__open.get_config()
substrate_def = commons_harvest__open.build(['red', 'green'], config)

# Add custom avatars to substrate_def['simulation']['game_objects']
substrate_def['simulation']['game_objects'] += [red_avatar, green_avatar]

substrate_def

env = substrate.build_substrate(
    substrate_def
    action_table=config.action_set,
    individual_observations=config.individual_observation_names,
    global_observations=config.global_observation_names,
)
```

Let me know if this doesn't make sense.

Closing. Feel free to re-open if you have more questions.
Repository: google-deepmind/meltingpot
Issue: Scenarios clean_up_20 and territory__rooms_6 are wrong [v2.3]
Root cause: Validation Gap

Two scenarios used for the Melting Pot Competition have issues:

* `clean_up_20` is using the wrong (puppet) bot: `clean_up__puppet_sanctioning_alternator_nice_0` instead of `clean_up__puppet_sanctioning_alternator_0`.
* `territory__rooms_6` is identical to `territory__rooms_5`.

The effective outcome is that the weight given to `territory__rooms_5` and `clean_up_19` are doubled.

At head, `clean_up_20` has been fixed. The tagged version 2.3.0 contains the mistake so it can be used to reproduce the state for the Melting Pot Competition 2023.

Fixed and will be released in v2.4
Repository: google-deepmind/meltingpot
Issue: Request: Support for vectorised environments
Root cause: Unimplemented Feature Gap

Hi All

Is it possible to run MeltingPot with vectorised environments? I am using CleanRL + Shimmy to run MeltingPot but when I try to use `concat_vec_envs_v1` I get error: 'TypeError: cannot pickle `dmlab2d.dmlab2d_pybind.Lab2d object`. Is there any other way? Any advice would be appreciated.

I need to train faster as I have limited compute walltime and the easiest way is vectorised environments. Any other methods for optimisation?

Maybe try raising this on the issues page for the underlying engine: [dmlab2d](https://github.com/google-deepmind/lab2d)

I have done so but have received no response
Repository: google-deepmind/meltingpot
Issue: Qustion about Episode Length Settings in Evaluation Scenarios
Root cause: Documentation Deficiency

Hello,

Firstly, I’d like to express my appreciation for your fantastic work on this gym. It’s been incredibly helpful and insightful.

I have a question regarding the evaluation scenarios: Given that the substrate can have a random length for an episode, are the episode lengths in the evaluation scenario also randomized? Additionally, how do you ensure that the evaluation of different algorithms is conducted under fair and consistent conditions concerning episode length?

Thank you for your assistance.

Yes, the episode lengths are also randomised. We just evaluate for many episodes and take average. It's important that the evaluation is _not_ on a finite horizon, because backwards induction is then possible, which changes the dynamics.
Repository: google-deepmind/meltingpot
Issue: ERROR: No matching distribution found for dmlab2d
Root cause: External Dependency Breakage

**Describe the problem:** Installing meltingpot (all latest versions, tested from v2.2.0 to 2.3.1) fails due to an error.

**Error message:**
```
pip install dm-meltingpot==2.3.1
Collecting dm-meltingpot==2.3.1
  Using cached dm_meltingpot-2.3.1.tar.gz (390 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting absl-py (from dm-meltingpot==2.3.1)
  Using cached absl_py-2.1.0-py3-none-any.whl.metadata (2.3 kB)
Collecting chex (from dm-meltingpot==2.3.1)
  Using cached chex-0.1.86-py3-none-any.whl.metadata (17 kB)
Collecting dm-env (from dm-meltingpot==2.3.1)
  Using cached dm_env-1.6-py3-none-any.whl.metadata (966 bytes)
INFO: pip is looking at multiple versions of dm-meltingpot to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement dmlab2d (from dm-meltingpot) (from versions: none)
ERROR: No matching distribution found for dmlab2d
```

**Attempted solutions:**
- change meltingpot version (v2.2.0 --> v2.3.1 tested)
- manually install `dmlab2d` via `pip install dmlab2d` following [lab2d github repo instructions](https://github.com/google-deepmind/lab2d)

**Python / pip version:** tested with both Python 3.11.9 and Python 3.9.12 / pip 24.0

**Additional info:** I did not trying building from source as I am constrained to have the package installed via `pip`.
Repository: google-deepmind/meltingpot
Issue: Request: tutorial for basic functions

Hi there,

I'm trying to build a substrate ('clean_up') but the documentation, in my opinion, is not very clear about what the functions or their arguments are specifically. For instance, 'roles' in clean_up is only shown to be a sequence of strings, not what those strings should be. If this information is located somewhere in the document or in dmlab2d, it is not easy to find. For ease of usage, I think some worked examples of building the substrates in Python and showcase the differences in usage would be very beneficial.



clean_up has no roles. Check out a a substrate like predator_prey to see how to use roles.

I'll have a look, thanks! My point is generally that functions like build, get_config etc should be directly explained rather than having to comb through each file to see what they do.