zhiminy/EvalEng huggingface dataset:
Repository: HumanCompatibleAI/overcooked_ai
Issue: Error Cloning Repository: Unexpected Disconnect and fetch-pack Failure
Root cause: Environment Incompatibility

Hi, 

New to coding. I encountered an issue while attempting to clone a repository. The cloning process fails partway through with an unexpected disconnect and fetch-pack errors.

Steps:
`git clone https://github.com/HumanCompatibleAI/overcooked_ai.git`

Error:
`Cloning into 'overcooked_ai'...
remote: Enumerating objects: 6988, done.
remote: Counting objects: 100% (1146/1146), done.
remote: Compressing objects: 100% (156/156), done.
error: 8085 bytes of body are still expectedB | 215.00 KiB/s  
fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: fetch-pack: invalid index-pack output
`

Environment:
Windows 11 Pro 24H2
Git  2.47.1.windows.2

Attempts at resolving:
`git config --global http.postBuffer 2147483648
git clone --depth 1 <repo-url>
git config --global core.compression 0 
git config --global core.compression 9
git config --global http.lowSpeedLimit 0
git config --global http.lowSpeedTime 999999`

- updating git
- turning off firewall

Any help is appreciated. Thank you. 

I think this is mostly likely due to an issue with your network or a temporary disconnection. I'd just try to re-clone the repository. I was able to clone the repository fine just now.
Repository: HumanCompatibleAI/overcooked_ai
Issue: Error while running ./up.sh - Issue with Python version and TensorFlow version mismatch
Root cause: External Dependency Breakage


### Description
When running the overcooked_demo ./up.sh script, I encounter the following error messages:

```
ERROR: Ignored the following versions that require a different python version: 0.3.8 Requires-Python >=3.8; ...
9.815 ERROR: Could not find a version that satisfies the requirement tensorflow==2.0.3 (from versions: 2.10.0rc0, 2.10.0rc1, 2.10.0rc2, 2.10.0rc3, 2.10.0, 2.10.1, 2.11.0rc0, 2.11.0rc1, 2.11.0rc2, 2.11.0)
9.815 ERROR: No matching distribution found for tensorflow==2.0.3
```

### Error Messages:

ERROR: Ignored the following versions that require a different python version: ...
ERROR: Could not find a version that satisfies the requirement tensorflow==2.0.3
### Steps to Reproduce:

Clone the repository and navigate to the project directory.
Run the ./up.sh script.
### Environment:

OS: macOS (M3 Chip)
Python version: 3.12.7
Pip version: 24.2
### Attempted Solutions:

Checked the Python version and found that it needs to be upgraded to 3.8 or higher.
Tried upgrading pip to the latest version by running pip install --upgrade pip, but the issue persists.
Tried installing a compatible version of TensorFlow, but tensorflow==2.0.3 cannot be installed.

### Expected Behavior: 
The ./up.sh script should run successfully, installing all dependencies without issues.
Repository: HumanCompatibleAI/overcooked_ai
Issue: Visualize Env
Root cause: Documentation Deficiency

Hi, is there any simple way to visualize the state of the environment by doing something like `env.render()`, I tried `env.render()` but it says that render is not defined for the env object. I only ask this since environments following the gym api generally have a render method, right?
The following is the example code that I am trying to run - 


```python

mdp = OvercookedGridworld.from_layout_name("asymmetric_advantages_tomato")

env = OvercookedEnv.from_mdp(mdp, horizon=400)
# for a certain number of timesteps, run random actions for both of the agents
num_time_steps = 200

env.reset()

for i in range(num_time_steps):
    # sample random action
    joint_action = random_joint_action()
    # step the env with that action
    state, rewards, dones, info = env.step(joint_action)
    env.render()

```
This gives an error - 
`AttributeError: 'OvercookedEnv' object has no attribute 'render'`

I mean I know this is an obvious error, but I am curious to know as to why doesn't the env have a render method?

Thanks again for the great environment and package!

Megh Bhalerao

I forget the exact reason we didn't do it. We only added the visualization functionality quite late, and we may have simply forgotten to plug it into the standard gym `render` functionality. If you want to do so and add a PR, feel free to!

I actually noticed the other day that [we do have a render method implemented](https://github.com/HumanCompatibleAI/overcooked_ai/blob/739950a079cdaed5a44fcc662efc40244c205d06/src/overcooked_ai_py/mdp/overcooked_env.py#L911). Let me know if it doesn't work for you
Repository: HumanCompatibleAI/overcooked_ai
Issue: Difference between the raw data in google drive and the data on the github repo?

Hi, what is the difference between the raw data in google drive and the data on the github repo? Are they exactly the same or is there some difference in the actual datapoints? Or are they just different formats, say csv vs pickle?
Please let me know and thanks!

Sorry, which data are you referring to exactly? If you could share the links that would be helpful

Hi, I am talking about the data here - https://drive.google.com/drive/folders/1aGV8eqWeOG5BMFdUcVoP2NHU_GFPqi57 and https://github.com/HumanCompatibleAI/overcooked_ai/tree/master/src/human_aware_rl/static/human_data/cleaned


they seem to be the same data but just csv vs pickle?

thanks for your response and please let me know if anything is unclear.


--- Megh Bhalerao

Yes I think that's right!
Repository: HumanCompatibleAI/overcooked_ai
Issue: Bonus Rewards
Root cause: External Dependency Breakage

Hi,

I have been working on integrating symbolic AI agents with the environment and I noticed something:
-Whenever you complete a bonus order the game completely freezes

For example, if you open the tutorial3 layout and complete the bonus reward, the game will just stop

Is this for overcooked-demo data collection? It's been a while since we've tested that, probably some package versions have changed and are causing issues. I'm trying to make package installation more reliable, and while I doubt it will fully fix this problem, if you were to get to the bottom of it, the fix would probably be long-lasting (so feel free to make a PR with the changes!) 

This is whenever I run the Overcooked-demo. The exact issue is as follows: you complete a bonus reward and the game freezes. To me that would only make sense if the bonus order was the last one to complete, so the game "ends". However, we should be able to complete multiple bonus rewards AND you can't even serve that order as the game freezes on serve. I will try to fix it when I have time next week, and will submit a PR if I find the issue:)
Repository: HumanCompatibleAI/overcooked_ai
Issue: [Looking for contributors] Fixing BC and RL training
Root cause: External Dependency Breakage

Hi there, I'm the original creator of the repo. Since the original release, the original packages we were using for BC and RL training have had major updates, people have transitioned from tensoflow to pytorch to jax, etc. 

This has broken the BC and RL training. Over the years, I've tried to fix things locally without having to spend too much time on it, but I don't think it's worth doing that anymore: this is investing in a stack (mainly built on tensorflow) which is outdated, and should be re-written. Good news is that writing this stack today is much easier than it once was.

What would this look like?
* For BC, you could quickly get the current tensorflow implementation in `human_aware_rl` to work (I was able to at least), but it's probably best to start from scratch and write it in pytorch (or even better, JAX). This shouldn't be too hard. If you do so, please make a PR!
* For RL training, I'd recommend using [JaxMARL](https://github.com/FLAIROx/JaxMARL): it's _much_ faster than our training was. I don't think salvaging our rllib-based training is worth it.
* For evaluating/visualizing/interacting with the trained RL policies, this would require some (hopefully minimal) infrastructure to port the JaxMARL-trained policies back to the overcooked-ai environment.



### Old instructions from the README about testing whether human_aware_rl is working correctly

To check whether the `humam_aware_rl` is installed correctly, you can run the following command **from the `src/human_aware_rl` directory**:

```
$ ./run_tests.sh
```

1. btw, I have a quick and dirty pytorch implementation for the BC training algorithm, but I don't think it is yet ready to make a PR - but I'll let you know.
2. Regarding this I think RL Lib is pretty good too, right? - https://docs.ray.io/en/latest/rllib/index.html - they claim industry standard RL etc. Plus I think JaxMARL already has an overcooked environment, although with different graphics.


PS - I am new to RL so my knowledge might be incorrect but just putting down my thoughts.

Rllib with a pytorch backend would be good, but using JaxMARL would still lead to faster training as far as I understand. That said, adding new functionality in Jax is more annoying because it requires one to vectorize everything. Indeed, the JaxMARL implementation of overcooked (which they based off of this repo) has less features, e.g. no tomato ingredients IIRC. 

While JaxMARL does have the basic version of overcooked implemented, the graphics are less nice/intuitive, and they don't have associated demo code to be able to deploy agents with humans to e.g. collect human data. So I think this repo still has some value for that, and for ease of implementation of new features relative to JAX.
Repository: HumanCompatibleAI/overcooked_ai
Issue: Error while running ./up.sh - Missing setuptools distribution version
Root cause: External Dependency Breakage

### Description
When running the overcooked demo ./up.sh script, I see the following error message after a while of running:
`
failed to solve: process "/bin/sh -c pip install -e '/overcooked_ai[harl]'" did not complete successfully: exit code: 1
`
### Error
When looking at docker logs, I get the following error:
`
ERROR: No matching distribution found for setuptools>=77.0.3
`

### Steps to Reproduce:
Clone the repository and navigate to the project directory.
Run the ./up.sh script.

### Environment:
OS: Windows (Intel)
Python version: 3.10.0
Pip version: 24.0

Hi, I'm encountering the exact same issue when trying to install `overcooked_ai[harl]` in a Docker container based on `python:3.7-buster`. The build fails requesting `setuptools>=77.0.3` for `overcooked-ai`, even though its `pyproject.toml` specifies `setuptools>=61`.

I've tried several approaches in my `Dockerfile`, including:

*   Pre-installing `setuptools==67.8.0`, `wheel`, and `Cython==0.29.x`.
*   Using `pip install --no-build-isolation -e '/overcooked_ai[harl]'`.
*   Attempting to install the `[harl]` dependencies (`tf_agents[reverb]==0.12.0`, `pygame==2.0.1`) separately before installing `overcooked_ai` itself.

Unfortunately, none of these have resolved the `setuptools>=77.0.3` requirement error stemming from `overcooked-ai`.

Has anyone found a workaround or a solution for this on Python 3.7? Any insights would be greatly appreciated!
Repository: HumanCompatibleAI/overcooked_ai
Issue: Where to find Human-AI gameplay data?
Root cause: Documentation Deficiency

Dear authors/developers, 

There is a sentence in the README of the repo saying "You can find some human-human and human-AI gameplay data already collected [here](https://github.com/HumanCompatibleAI/overcooked_ai/tree/master/src/human_aware_rl/static/human_data)."

However, I see that provided 2020 and 2019 data is all human-human. May I know where to find the human-AI (preferably PPO) data that you may already have?

Thank you,
Vidyasagar.
Repository: HumanCompatibleAI/overcooked_ai
Issue: Should probably merge this repo with overcooked-demo

There seems to be little reason not to merge this repo with overcooked-demo. Currently there is no way to visualize trajectories directly in this repo that isn't terminal graphics in python (which takes some time to get used to). 

In this sense, having an extra repo just for interactive play with agents (trained or non-trained) seems like just an additional barrier to entry. One reason why we did the division in the first place was that the hard-coded agents in overcooked-demo are resulting from DRL runs of the paper, so it seemed more tied to the paper than not. However, I feel like once we make the demo more agent agnostic (having random and custom agents too), then this shouldn't be an issue anymore.

This is rehashed (in better shape) in #58.
Repository: HumanCompatibleAI/overcooked_ai
Issue: Will the trained models (BC and H_proxy) be publicly available?
Root cause: Documentation Deficiency

I want to use these models to be the baseline in my work.
Can I have the access to these models? or is there any way I can make sure that my implementation of the baseline (BC and H_proxy) is correct? (e.g. training loss)

edit: I found this [file](https://github.com/HumanCompatibleAI/human_aware_rl/blob/master/human_aware_rl/imitation/behavioural_cloning.py) but it still use GAIL model to train behaviour cloning. Is this the version that used in the paper?

Sorry for the wait, I didn't have notifications turned on for some reason. We didn't include the models because there are a lot of them and are each 10 MB or so (but if you want I can send them to you somehow).

However, I would recommend training your own: you can just follow the first couple cells of [this](https://github.com/HumanCompatibleAI/human_aware_rl/blob/master/human_aware_rl/NeurIPS%20Experiments%20and%20Visualizations.ipynb) Jupyter Notebook. Let me know if you have any issue with this.

The file that you linked is the correct file (which the Jupyter Notebook is also importing): if you look closely, you will see that we only use the "pretrain" method (not the GAIL learn method itself), that is just behaviour cloning.

--

Edit: the link to the notebook is now [this](https://github.com/HumanCompatibleAI/human_aware_rl/blob/neurips2019/human_aware_rl/NeurIPS%20Experiments%20and%20Visualizations.ipynb)

@micahcarroll Thanks for the reply. After looking into the experiment repo, I found this [folder](https://github.com/HumanCompatibleAI/human_aware_rl/tree/master/human_aware_rl/data/bc_runs) which contains the trained BC and H_proxy model.

And this [block](https://github.com/HumanCompatibleAI/human_aware_rl/blob/master/human_aware_rl/ppo/ppo.py#L235) can load those models as "OTHER_AGENT" in the environment.
```
def configure_other_agent(params, gym_env, mlp, mdp):
    if params["OTHER_AGENT_TYPE"] == "hm":
        hl_br, hl_temp, ll_br, ll_temp = params["HM_PARAMS"]
        agent = GreedyHumanModel(mlp, hl_boltzmann_rational=hl_br, hl_temp=hl_temp, ll_boltzmann_rational=ll_br, ll_temp=ll_temp)
        gym_env.use_action_method = True

    elif params["OTHER_AGENT_TYPE"][:2] == "bc":
        best_bc_model_paths = load_pickle(BEST_BC_MODELS_PATH)
        if params["OTHER_AGENT_TYPE"] == "bc_train":
            bc_model_path = best_bc_model_paths["train"][mdp.layout_name]
        elif params["OTHER_AGENT_TYPE"] == "bc_test":
            bc_model_path = best_bc_model_paths["test"][mdp.layout_name]
        else:
            raise ValueError("Other agent type must be bc train or bc test")
```

So I guess I can use them directly as "OTHER_AGENT"  in my experiments without running training code in the mentioned notebook?

Yes, you are correct! I had forgotten that while we had not added the PPO agents, we had added the BC ones!
Repository: HumanCompatibleAI/overcooked_ai
Issue: Add support for Windows
Root cause: Architectural Constraint

Make all path directory code OS agnostic. Would require using `os.path.join` rather than manually adding slashes between directory names in many places.

Hi Micah, I have looked into this on the side, and unfortunately ray 0.8.5 does not support windows

Ah, ok, sounds good – I guess this is simply not going to happen for the time being then.
Repository: HumanCompatibleAI/overcooked_ai
Issue: Add additional ingredients
Root cause: Unimplemented Feature Gap

Adapting the `Recipe` class to handle additional ingredients for a more diverse soup space. Might involve changes to the `OvercookedGridworld` and `OvercookedEnv` classes to handle the new ingredients

Just as a comment – this seems to me like less important than other issues in the to-do list. Adding ingredients will increase the amount of code we have to maintain and test, and we currently have no plans for actually using such code. Instead, visualizations infrastructure improvements would be useful to everyone pretty much immediately, and as such would be very worthwhile to maintain.
Repository: HumanCompatibleAI/overcooked_ai
Issue: Native Python Visualization
GitHub labels: enhancement

Updating the existing JS graphics to provide native python support for overcooked GUI. 

Result of this task would be a way of displaying OvercookedState to produce view like [this](https://raw.githubusercontent.com/HumanCompatibleAI/overcooked_ai/master/images/screenshot.png)

Pygame offers enough features to make visualization of the state (and playing game with AI) work. The only thing it is missing from the ideal solution is integration with Ipython notebook (seems like displaying pygame inside jupyter cell is problematic). Do we care about that a bit, much or not at all?

Possible workarounds include:
- use skulpt, there is [project](https://github.com/Petlja/pygame4skulpt) that has not everything implemented but probably enough
- implement state visualization as js code and open js in web browser/inside jupyter cell (as currently is done with charts https://github.com/HumanCompatibleAI/overcooked_ai/pull/38 )

the only reason we would ever want to use the graphics in a jupyter notebooked would be for static state visualization, not game play. For example, it would be super cool to be able to provide an `OvercookedState` instance, `state` to a function call like `visualize_state(state)` and have it display a JPG like the one you linked above inside a notebook.

There is no use case for playing interactive games inside a notebook

With that in mind, I'll leave it to you to choose the best tool (I'm not particularly familiar with pygame or skulpt) with the only constraint that all visualization (static and interactive) must be able to be invoked from a python main function

By the way, you can still have sliders with ipythonwidgets that enable you to flick between images of the same trajectory – that comes in handy. But we definitely will not have to _play_ with agents through the notebook – in fact, we want the gameplay dynamics to have only one ground truth - overcooked-demo, or things will get kind of confusing (it would be hard and unnecessary work to perfectly match up the smoothness and gameplay experience of overcook-demo in another game environment).

And yes, I agree with Nathan that the main objective here is to have a function `visualize_state` that goes from OvercookedState to JPG.

Thanks for the information.

Alternative approach would be to use overcooked-demo code and put into ipython cell, but it is probably more time consuming and creates additional dependency between python and js code.

I will create task for this and put in to-do list.
EDIT: I've created issue and put it into kanban https://github.com/HumanCompatibleAI/overcooked_ai/issues/63
Repository: HumanCompatibleAI/overcooked_ai
Issue: Code Coverage Monitor
GitHub labels: Continuous Integration

Incorporating code coverage automation reports into the project to better enhancing testing and confidence in code. Example of such a tool [here]( https://codecov.io/)

The `upload coverage to Codecov` portion of the integration test failed when the checks run in the current master branch. I assume this is some sort of connection issue connecting to the codecov client? The actual tests appeared to pass. https://github.com/HumanCompatibleAI/overcooked_ai/runs/995100873

It can happen if there is an issue with uploading the result to codecov. I've clicked "rerun all jobs" button and now it passed without errors so it is not an issue with the code itself. Possible solutions:
 - change flag fail_ci_if_error to false, but then we are not notified if there is fail in sending coverage data into the cloud
- add separate test GitHub workflow for coverage tests 
	- needs to run all the tests one more time
	- it is seen from pull request GUI which workflow failed (and if it is only coverage one or test one too)

Setting `fail_ci_if_error` to False seems fine to me. As long as it succeeds some of the time to upload it, I think we should be good.
Repository: HumanCompatibleAI/overcooked_ai
Issue: Create Trajectory class
GitHub labels: Refactor

Creating a Trajectory class to better abstract away a lot of the complex dictionary parsing in a more Object Oriented fashion.
Moved from https://github.com/HumanCompatibleAI/overcooked_ai/issues/44 as this issue will probably wait a bit.

I have a couple of questions before starting making code here:

1. Do we want this task done considering it was initially part of https://github.com/HumanCompatibleAI/overcooked_ai/issues/44 that probably won't happen (current discussion ended on the decision to not do this issue)? 

2. How much do we care about fully backward compatibility here? How I can solve backward compatibility:
a) Implement a method that returns trajectory dict in its current form
b) Implement magic methods like `__getitem__` to let users use trajectory object exactly like trajectory dict. It is a bit constraining (at least prevent from use magic method in the way different that was implemented in dict) and there is a risk of forgetting some magic method.
c) just don't care about it - edit overcooked-ai and harl repo to work with the changes and that it enough
d) some mix of above

3. Should I create it from the master branch or trajectory visualization PR code https://github.com/HumanCompatibleAI/overcooked_ai/pull/38 (it depends if anyone wants this change before the merge of https://github.com/HumanCompatibleAI/overcooked_ai/pull/38 will happen - currently it waits for review after some refactor and added compatibility with the adjectives)?

@nathan-miller23 & @mesutyang97, I think this is kind of up to you (whether we should do this now or not) – I wouldn't expect either of you to be using this version of the code anytime soon (unless you think it valuable enough to spend time merging it with yours). Moreover, I expect this to require quite a large overhaul of human_aware_rl and other repos, such as your own project ones – so even reviewing this PR and the corresponding ones in other repos (or making those yourselves) would likely be a lot of work.

I personally think that it's probably not worth it right now, and we should punt it to a better time in which there is less pressure on getting actual research done.