SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For Miniworld, the most relevant work is ViZDoom. The ViZDoom research platform [19] is a set of simulation environments based on the popular first-person shooter (FPS) game Doom that enables RL agents to make tactical and strategic decisions. At a high level, the type of environments that can be created are comparable in Miniworld and ViZDoom. However, ViZDoom uses a custom-designed language to create new scenarios, while Miniworld uses a small set of Python functions for environment creation. This makes Miniworld easier to use for the RL research community, which is more familiar with Python. Nevertheless, ViZDoom does support shooting games, depth information, and audio which are not supported by Miniworld. Another 3D simulation library with a similar purpose is DeepMind Lab [3], but given that the game engine is written in C and the levels are written using Lua, there is a steep learning curve for customizing it. 3D simulation environments like Habitat 3D [23, 36] and Unity-based simulation environments [17] also enable the agent to navigate and interact with its surroundings. Compared to Miniworld, they can simulate more complex dynamics and are more photo-realistic. However, for target-reaching and object-collection tasks where the goal is not deployment on embodied AI systems, Miniworld provides a much simpler and lightweight simulation solution that enables faster iteration and evaluation of new research ideas. Open-ended 3D environments like Avalon [1], MineRL [11], Malmo [16], and MineDojo [8] also have similar capabilities. But their focus is more on evaluating the RL agent's ability to generalize on a wide range of tasks, while Miniworld environments mostly focus on a single task.
The Minigrid and Miniworld libraries provide modular and customizable RL environments for goal-oriented tasks. We detailed the design philosophy behind the two libraries and provided a walkthrough of their API along with research areas that already utilize the two libraries. In our case studies, we have shown the unified API among two libraries provides an easy way to study transfer learning between different observation spaces and human decision-making. In future works, we plan to further develop the libraries' capabilities for human-in-the-loop decision-making.
Limitations: The libraries have two main limitations, first, the environment creation process prioritizes simplicity with minimal functions, which limits the type of environments that can be created. Second, the two libraries are implemented in Python, which makes them computationally slower than environments that utilize highly-optimized game engines in  $\mathrm{C + + }$
Societal Impact: Since the libraries have idealized system dynamics, the learned policy might not be directly applicable to real-world applications without introducing safeguard mechanisms.
Minigrid and Miniworld were originally created as part of research work done at Mila - Quebec AI Institute. We thank Manuel Goulão for their contribution to the documentation website.
[34] S. Sukhbaatar, A. Szlam, G. Synnaeve, S. Chintala, and R. Fergus. Mazebase: A sandbox for learning from games. CoRR, abs/1511.07401, 2015.
[35] R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 1998.
[36] A. Szot, A. Clegg, E. Undersander, E. Wijmans, Y. Zhao, J. Turner, N. Maestre, M. Mukadam, D. Chaplot, O. Maksymets, A. Gokaslan, V. Vondrus, S. Dharur, F. Meier, W. Galuba, A. Chang, Z. Kira, V. Koltun, J. Malik, M. Savva, and D. Batra. Habitat 2.0: Training home assistants to rearrange their habitat. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[37] E. Todorov, T. Erez, and Y. Tassa. MuJoCo: A physics engine for model-based control. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal, pages 5026-5033, October 2012.
[38] S. Tunyasuvunakool, A. Muldal, Y. Doron, S. Liu, S. Bohez, J. Merel, T. Erez, T. Lillicrap, N. Heess, and Y. Tassa. dm_control: Software and tasks for continuous control. Software Impacts, 6:100022, 2020.
[39] A. Wachi, Y. Wei, and Y. Sui. Safe policy optimization with local generalized linear function approximations. In Proceedings of Advances in Neural Information Processing Systems 34, Virtual, pages 20759-20771, December 2021.
[40] T. Zhang, H. Xu, X. Wang, Y. Wu, K. Keutzer, J. E. Gonzalez, and Y. Tian. Noveld: A simple yet effective exploration criterion. In Proceedings of Advances in Neural Information Processing Systems 34, Virtual, pages 25217-25230, December 2021.
[41] M. Zhao, Z. Liu, S. Luan, S. Zhang, D. Precup, and Y. Bengio. A consciousness-inspired planning agent for model-based reinforcement learning. In Advances in Neural Information Processing Systems 34, Virtual, pages 1569–1581, December 2021.
[42] W. Zhou and W. Li. A hierarchical bayesian approach to inverse reinforcement learning with symbolic reward machines. In Proceedings of International Conference on Machine Learning, Baltimore, MD, USA, volume 162, pages 27159-27178, July 2022.