Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
Authors: Mikel Malagón
Josu Ceberio
Jose A. Lozano
Venue: ICML 2025 poster (2025)
Keywords: environment
reinforcement learning
agent
embodied AI
open endedness
Primary area: reinforcement_learning
TL;DR: Craftium is an easy-to-use and flexible framework for creating rich 3D single- and multi-agent environments while substantially improving the computational cost of alternatives.

Abstract: Advances in large models, reinforcement learning, and open-endedness have accelerated progress toward autonomous agents that can learn and interact in the real world. To achieve this, flexible tools are needed to create rich, yet computationally efficient, environments. While scalable 2D environments fail to address key real-world challenges like 3D navigation and spatial reasoning, more complex 3D environments are computationally expensive and lack features like customizability and multi-agent support. This paper introduces Craftium, a highly customizable and easy-to-use platform for building rich 3D single- and multi-agent environments. We showcase environments of different complexity and nature: from single- and multi-agent tasks to vast worlds with many creatures and biomes, and customizable procedural task generators. Benchmarking shows that Craftium significantly reduces the computational cost of alternatives of similar richness, achieving +2K steps per second more than Minecraft-based frameworks.

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
Mikel Malagón
Josu Ceberio
Jose A. Lozano 12
Abstract
Advances in large models, reinforcement learning, and open-endedness have accelerated progress to- ward autonomous agents that can Iearn and inter- act in the real world. To achieve this, flexible tools are needed to create rich, yet computationally ef- ficient, enviromments. While scalable 2D environ- ments fail to address key real-world challenges like 3D navigation and spatial reasoning. more complex 3D environments are computationally expensive and lack features like customizability and multi-agent support. This paper introduces Craftium, a highly customizable and easy-to-use platform for building rich 3D single- and multi- agent environments. We showcase environments of different complexity and nature: from single- and multi-agent tasks to vast worlds with many creatures and biomes, and customizable proce- dural task generators. Benchmarking shows that Craftium significantly reduces the computational cost of alternatives of similar richness, achieving +2K steps per second more than Minecraft-basse frameworks.
CRAFTILMM
Figure I. Examples of the diverse single- and multi-agent envi- ronments that can be created Craftium. From surviving in procedurally generated dungcons exploring vast open worlds filled with animals, monsters, and varied biomes.
well-known example is the Arcade Learning Environment (ALE) (Bellemare et al., 2013), which undoubtedly con- tributed to the advancement of the RL field, marking many of its most important milestones. To name a few: the intro- duction of the Deep Q-Networks (Mnih et al., 2013), the "infamously difficult Montezuma's Revenge" (Bellemare et al., 2016) 6) that inspired many exploration strategies (Os- trovski al., 2017; Burda et al., 2019; Badia al., 2020b), and the first time agent outperformed humans in all Atari benchmarks (Badia et al., 2020a).
1. Introduction
However, as observed throughout the literature, research in these areas is bound to the challenges the employed envi- ronments introduce. The researcher often faces a dilemma between computationally efficient but simplistic environ- ments substantially slower but richer environments. For instance, Continual Reinforcement Learning (CRL) (Abel et al., 2023), Unsupervised Environment Design (UED) (Garcin et al., 2024), and Multi-Agent RL (MARL) (Ying et al., 2023), are greatly affected by the efficiency of the employed environments as they require learning from many tasks or agents. Thus, in these works, experiments are of- ten limited to simple environments computational cost of employing more complex alternatives (Rigter et al., 2024; Malagon et al., 2024; Rutherford et al., 2024). For example. Craftax relies on 2D grids (Matthews et al., 2024), while OMNI-EPIC (Faldor et al., 2025) em- ploys 3D environments of substantially limited diversity compared to alternatives like MineDojo (Fan et al., 2022)
Progress Reinforcement Learning (RL) (Sutton Barto, 2018), embodied AI (Paolo et al.. 2024), and open-ended agents (Hughes et al., 2024) is inherently tied to the envi- ronments where agents trained, evaluated, and analyzed. Each new insight or advancement in the field is supported by an environment that enables its emergence and study. A
'Department of Computer Scienee and Artificial Intelli- gence, University of the Basque Country UPV/EHU, Donostia- San Sebastian, Spain 'Basque Center for Applied Mathematics (BCAM), Bilbao, Spain. Correspondence to: Mikel Malagón <mikel.malagon@e @ch..css.
consequence the
Proceedings of the 4nnd International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Code available at https: wtt.ttuu...mm/mikeemmmmm craft fum.

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
or Habitat 3.0 (Puig et al., 2024).
e.g., Roblox, World of Wareraft, and Neovim. In Craftium, Lua is used expose the Luanti engine, allowing vast pos- sibilities for developing custom environments. Moreover, Luanti is open-source and has a vibrant community that has created many games and assets that can be used in Craftium environments (Ward, 2025a), significantly reducing the de- velopment cost of complex scenarios. For instance, all the environments shown in Figure have been implemented in less than 160 lines of code (comments and whitespace in- cluded). These environments, later described in Section 3.5, showcase the versatility of the presented framework, from RL and MARL tasks of different nature, customizable proce- dural environment generators for CRL, UED, and meta-RL (Yu et al., 2020; Rimon et al., 2024) to gigantic procedu- rally generated open worlds 644Kx64Xx64K blocks) for research on embodied AI (Paolo et al., 2024) and open- ended agents (Wang et al., 2023). Beyond being flexible, feature-rich, and developer-friendly we show that Craftium environments run 38x faster than alternatives based on the original Minecraft game, the only platforms that offer simi- lar complexity and richness. Craftium also supports running asynchronous environments in parallel, achieving more than 12K steps per second in this setup. Furthermore, Craftium is the first framework that allows the creation of vast 3D open worlds while supporting multi-agent settings, opening the door to new research lines. Finally, Craftium implements the popular Gymnasium (Towers et al., 2024) and Petting- Zoo (Terry et al., 2021) interfaces, the modern standard for RL and MARL research respectively, making it compatible with many other libraries and projects (Raffin al., 2021; Huang al., 2022b; Serrano-Muñoz et al., 2023). Finally, Craftium is fully open source and includes extensive online documentation with many guides, usage examples, tutorials, a detailed reference, and ready-to-use scripts.'
Conversely, works on rich and complex environments (Grbic et al., 2021; Earle et al., 2024; Prasanna et al., 2024; Raad et al., 2024) rely on fully featured video games that have a high computational cost and are closed-source. The best- known of such platforms is Minecraft, which has inspired several single-agent environments and benchmarks over the years (Johnson al., 2016; Guss et al., 2019; Fan et al., 2022). However, Minecraft is a fully featured and complex 3D game, which makes it substantially more inefficient than simpler alternatives (Wydmuch et al., 2019; Matthews et al., 2024). Furthermore, its closed-source nature greatly limits its flexibility, hindering its application to problems beyond "classic" RI., like CRI., MARI., and UED.
Another important issue that especially affects research in these areas is the lack of flexibility in the environments. Commonly used environments offer no customization or limited possibilities, often restricted to a set of predefined parameters, such as difficulty level or the number of enemies. Among others, these environments include: ALE (Machado et al., 2018), MineRL. (Guss et al., 2019), ProeGen (Cobbe et al., 2020), MineDojo (Fan et al., 2022), Crafter (Hafner, 2022), and Craftax (Matthews et al., 2024). The lack of flexibility hinders the ability to analyze specific behavior of agents, obstructing algorithmic comparison beyond pure performance benchmarking, which has been shown to be insufficient for RL (Jordan et al., 2024). Although flexible platforms that allow the creation of new and diverse environ- ments exist, these fall into 2D worlds (Bamford et al., 2020; Chevalier- -Bisvvrtrr al., 2023; Matthews et al., 2024) or depend on complex Domain Specific Languages (DSL) that make their implementation difficult, while still not being 3D, as is the case with VizDoom (Wydmuch et al., 2019) and MiniHack (Samvelyan et al., 2021).
In this paper, we present Craftium, an easy-to-use plat- form for creating rich and efficient 3D environments for autonomous agent research. Unlike most complex envi- ronment platforms, which are based on video games (e.g., VizDoom is based on ZDoom and MiniHack on NetHack), Craftium is based on a game engine: Luanti (Luanti Team, 2025b). The integration with our modified version of the engine (see Appendix A) allows the easy creation of com- plex voxel environments using the powerful and greatly documented Lua Modding API (Luanti Team, 2025a) in- stead of much less popular DSLss, employed in Z.Doom or MiniHack. Lua (lerusalimschy, 2006) is a Python-like, easy-to-use and understand, mature, and efficient program- ming language used in many popular tools and projects,
2. Background: Luanti
Craftium is based on our modified version of the Luanti game engine (refer to Appendix A for details and the list of modifications). Luanti (Luanti Team, 2025b) is well- known open-source voxel game engine launched for the first time in 2011 that is currently being developed by a vibrant community. Unlike most game engines, Luanti sup- ports modding at its core through its Lua API, allowing fine-grained and real-time access and modification of the internal state of the engine (examples can be found in Sec- tion 3.2 and Appendix H). This enables extensive and pro- grammatic customization of its behavior, facilitating the creation, modification, and extension of existing games (en- vironments) using its powerful Modding API (Luanti Team, 2025a; Ward, 2025b). Additionally, Luanti is implemented
Voxel games use 3D blocks (voxels) to construct and represent the world, allowing players to modify the environment by adding removing blocks. Figure 1 shows examples of these types of environments.
'These interfaces are general and can be used for learning paradigms beyond RL (e.g.. evolutionary algorithms).
2

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
Nx
Craftium
Agent
Luanti
Python library
observation
Environment Interface
mod
world
API
Wrappers
init./ua
stee-)
viniovetiin
reward
Server
Aon
Nx
Additional wrappers
action
Client
TCP
#rrarykctooniaaper
Figure 2. Overview of Craftium's 's internal architecture. Components denoted with x.N are repeated according to the number of agents (one more).
in C++, a widely adopted programming language known for its high efficiency. Finally, Luanti is supported by an active community that has created hundreds of open, free- to-use games and mods (Ward, 2025a) that are seamlessly loaded in Craftium. For example, community mods are em- ployed in all the environments from Section 3.5 showcased in Figure and 9.
environments. Moreover, Craftium supports many popular techniques such as frame skipping and frame stacking that are commonly used throughout the literature (Huang et al., 2022a).
Actions. By default, actions are composed of a combination of 21 keyboard actions and a tu- plc that defines the movement of the mouse, which is mainly used to control the camera. Keyboard- related actions are binary vari- ables with value of I if the key is pressed, and 0 otherwise. The movement of the mouse is de- fined by the tuple AA:AA)) E [-1, 112, where A, < 0 moves the mouse the left in the horizontal axis and A₂ > to the right, similarly, Ay 0 moves the mouse downwards in the vertical axis and A,, > 0 moves it upwards. Thus, if A₂ = A, O, the mouse is not moved. See Appendix B. for detailed description of all the possible actions sup- ported in Craftium.
3. Craftium
Craftium follows the architecture illustrated in Figure 2. It consists of two main components: the Luanti engine and the Python environment interface. This interface is the bridge between the environment and the agents. Internally, it han- dles a communication channel per agent, which connects to Luanti, sending and receiving data such as observations, actions, and rewards. On the other hand, the Luanti server executes the logic of the environment, specified by a file characterizing the 3D world and script (i.e., mod) that defines its behavior. The Luanti server also synchronizes its clients (one per agent), which handle rendering and commu- nication tasks with the Python library. Finally, note that the original version of Luanti does not support these features, but its open-souree nature allowed modifying its source code support this architecture (see Appendix A).
Figure 3. Example of 64x64 pixel RGB image observation.
The default action space designed to be versatile, cover- ing as many use cases as possible: from tasks with complex action sequences (e.g., manual inventory control) to simple navigation environments with couple of actions (e.g., for- ward and lateral movement). However, the default action space is overly complex for most tasks: the number of pos- sible keyboard action combinations in the default space is 221, Therefore, Craftium allows reducing the action space to the minimal subset required solve the task hand, substantially simplifying the learning process of the agent (see Appendix B.2).
In the following, Sections 3.1, 3.2, and 3.3 describe Craftium environments, the creation process, and the in- terface to use them. Respectively, Section 3.4 compares the performance of Craftium with other frameworks. Finally, Section 3.5 showcases the presented framework as a general- purpose environment creation tool across variety of use cases and fields concerning autonomous agents.
3.1. Observations, Actions, and Rewards
Observations. Craftium, observations are images from the agent's point of view. An example observation is pro- vided in Figure 3. Observations are highly customizable (c.g., size, number of channels, etc.) and can vary between
Rewards. In Craftium, reward functions are defined us- ing Lua scripts (mods are discussed in the next section). Craftium provides a comprehensive set of tools for this pur- pose, including an extended version of Luanti's Modding
3

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
name = craftium mod
core. dignode (function (ps, block) if string.find (block "aalookk""mme"""" ["name"], "tree") then set ret_reearrd_oo pecce (1.0, 0, 0.
description My environment 3 depends default
end end)
(f core. pere.register. _et_sttr__o_ddeplaerr (function (obj, rn)
Figure 4. Example configuration file of mod implementing a Craftium environment. This ex- ample depends on the de fault mod that pro- vides some basic functionalities to the mod.
set set_termination
end)
Figure 5. Lua script (i.c., mod) implementing basic environment mechanics.
API. This functionality is implemented in a modified version of the engine developed specifically for this work, which incorporates additional functions for setting and retrieving reward values and episode termination flags. Some of these functions are shown in the example mod from Section 3.2, while the additional API functions for defining RL envi- ronments are detailed in Appendix C. The complete list of modifications to the original Luanti engine can be found in Appendix
ping as many trees as possible (presented in Section 3.5.1). Line 1 registers a callback function that is called every time the player (i.e., agent) digs a block. In line 2, this function checks if the dug block is part of tree; if the condition is met, line 3 sets the reward to for that timestep (set (set.rewa www.rrewardocccoo and other RL related functions are described in Appendix C). Line 7 registers another callback function. In this case, the function is run every time the player dies and calls another function that terminates the episode, in line
3.2. Creating Custom Environments
Even basic mods, such as the presented example, can be used to generate wide range of environments. Furthermore, advanced community-made extensions and games can be easily integrated into Craftium, significantly expanding its potential. Section 3.5 highlights some these possibilities. Refer to Appendix and to the online documentation for detailed instructions on creating Craftium environments. Finally, note that the creation of Luanti mods is outside the scope of this paper, comprehensive resources are already available (Luanti Team, 2025a; Ward, 2025b).
Creating a Craftium environment implies two steps: ! gen- erating a world: a database with all the information about the virtual environment where the agent will be placed and will interact with (Figure 1 shows images of variety of worlds); and 2 defining the behavior of the environment, such as the reward function and conditions for episode ter- mination. The following lines describe these steps in detail.
! Luanti offers a vast range of possibilities for generating worlds. However, creating world can be as simple as a few clicks when using one of the many predefined map generators. 4 If finer control over the map generation process is needed, maps can be created using custom scripts. The procedural environment generator presented in Section 3.5.4 is an example of a more complex custom map generation process.
3.3. Interface
Once created, Craftium environments are used via the Gym- nasium (Towers et al., 2024) (single-agent) or PettingZoo (Terry et al., 2021) (multi-agent) interfaces. Both interfaces are open-source and have become the standard interface for RL and MARL environments, providing a unified ab- straction over environments that enables interoperability between environments and methods. Just by implement- ing these interfaces, Craftium is already compatible with many existing tools and projects to train, test, develop, and analyze many algorithms, including but not limited to stable-based iebelineed ines3 (Raffin et al., 2021), Ray RL- lib (Moritz et al., 2018), CleanRL. (Huang et al., 2022b), and skrl CCerrano-Müño et al., 2023).
② The next step define the behavior of the environment. This is done via mods: user-defined scripts that modify and extend the game engine's behavior, allowing for the cre- ation of custom environments, mechanics, and interactions within the 3D world. A mod has minimum of two files: a configuration file and Lua script.
The configuration file contains the mod's metadata. com- monly includes the mod's name, a description, and the list of dependencies (see Figure 4). The Lua script is where the environment's 's mechanics are implemented. Figure 5 illustrates an example script that defines the task of chop-
Figure 6 illustrates example using the Gymnasium (single-agent) interface. Note that, PettingZoo employs very similar interface described in Appendix D. Line 4 loads an example Craftium environment by name (see Section 3.5). Line 6 initiates an episode, obtaining the first observation
MMap generators documented https://dev:
luanti. meanti.org/mmee wrt.../ggg///annn.eeeee

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
Beyond the single-environment setup, running Craftium en- vironments in parallel notably increases their throughput as shown in Appendix E, reaching over 12K steps per sec- ond on the same hardware. By significantly reducing the computational requirements, Craftium enables researchers to conduct large-scale experiments on complex scenarios within their desired domain, supporting advancements in emerging (but especially sensitive to computational cost) areas like CRL, lifelong learning, UED, and open-ended agents. Further details on the benchmark and extended analysis of the results are provided in Appendix E.
import gymnasium as gym import craftium
cnv = gym. make "Craafiiu/Roonnoo0
obs, inf env. resere ()
for in range (5000):
a agent (obs)
cbs, r, tm, tc, inf = env. (a)
10
u if tm. tc:
12 obs, inf reset ()
13
14 env. close() ()
3.5. Illustrative Examples
Figure 6. Python code illustrating the interaction loop between the agent and a Craftium environment using the Gymnasium interface.
Craftium is platform that allows the development of fast and rich 3D environments for all research subfields of au- tonomous agents, such as RL, MARL, embodied AI, meta- learning, continual RL, and open-endedness. Due to the page limitation, this section highlights Craftium's poten- tial across a few of its vast number of possible use cases: single and multi-agent RL tasks (Sections 3.5.1 and 3.5.2), open- world environments for large multimodal model-based embodied agents (Section 3.5.3), and environment gener- ators for CRL (Section 3.5.4). The aim is to demonstrate the framework's capabilities and provide accessible, well- documented foundations for building custom environments tailored to specific research needs. Note that these exam- ples are merely illustrative and are be understood as benchmarks.
and a Python dictionary with additional information (e.g., clapsed time). Lines 7-12 implement the eecrreenvir interaction loop. In line 8, the agent selects an action based on the current observation. The line executes the action specified by the agent, resulting in an observation, reward, truncation flag, a termination flag, and new informa- tion dictionary, respectively. The truncation flag indicates if the maximum number of timesteps allowed by the environ- ment is reached, while the termination flag determines if the episode has reached a terminal state (e.g., the player dies). Both flags are checked in line 11, and one or both of them are true, the episode restarted in line 12. Finally, the last line closes the environment after the main loop ends.
3.4. Performance
3.5.1. EXAMPLE I: SINGLE-AGENT RL
As stated in the intro- duction, computation- ally efficient environ- ments are key for re- search on autonomous agents; as such, it has been a focal point of Craftium's development. Figure 7 compares the steps (i.e., interactions) per second obtained by Craftium to VizDoom and MineDojo, well- known environment cre- ation platforms from the literature. Results show the average of 5 runs in differ- ent environments per framework, on machine with a sin- gle NVIDIA A5000 GPU and an Intel Xeon Silver Y309Y CPU. Craftium achieves very competitive results compared to VizDoom, even though VizDoom is based on ZDoom, which is not 3D per se, discussed in Appendix F. Comparing Craftium's 's performance to MineDojo's, we observe that the presented framework achieves +2670 steps per second more.
This section provides examples of using Craftium to cre- ate single-agent environments for RL. We implement five tasks diverse nature: simple environments for testing RL algorithms, sparse reward and exploration scenarios, and a challenging survival task. For simplicity, all tasks share the same 64 x 64 pixel RGB image observation space. More- over, the default action space described in Section 3.1 is simplified to only use the necessary actions to solve each task (see Appendix B.2). Figures and extended descriptions of the environments are provided in Appendix G.I.
2500
2000
1500
1000
MineDajo
VizDoom
Craftium
To complement this example, Figure demonstrates that environments of varying difficulty levels can be designed within Craftium. The figure shows the results of the Proxi- mal Policy Optimization (PPO) algorithm (Schulman et al., 2017) in two of the presented tasks. Results in Figure 8a indicate that the ChopTree task can be successfully solved, chopping over trees per episode (+1 for every chopped tree). In SpidersAnack, agents are rewarded +1 for every defeated spider, where additional spider appears in every round (until 5 spiders). As can be seen in Figure 8b, the final episodic return in this task is lower than 1.5, showing that the agents only reach the second of five rounds. See
Figure 7. Average steps per sec- ond obtained with MineDojo, Viz- Doom, and Craftium.
5

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
Episodic return
Episodic retur
Episedic return
0.30
0.25
60
62
0.4
1.6
2.5
10
24
Steps
Steps
Steps
(a) ChopTree.
(b) SpidersAtrack:
(c) Multi-Agent Combat.
Figure& 8. Episodic return curves obtained by PPO in the single- agent ChopTree and SpidersAttack tasks, and the multi- MACombat environment. Results aggregate different runs per task: average is denoted with lines and the standard error with the contour.
X7
100
Steps
LLava-Agant
190-15TM
Figure 9. The leftmost picture shows an overview of the map for the open-world environment example. The map shows an area of 1.6K 1.6K blocks from the vast 64K x 64K x64K blocks area that agents explore. The color of each pixel is used denote different biomes, some of which are visualized in the center figures. The rightmost plot shows the results of PPO+LSTM and LLava-Agent (zero-shot) in terms of average and best cumulative reward values across 10 repetitions per method. Refer to Appendix G.3 for further details.
Appendix G.1 for further details and experimental results in the rest of the tasks.
Hunt
Start
3.5.2. EXAMPLE 2: MULTI-AGENT RL.
chicken
steep
P5
This section showcases Craftium's multi-agent capabilities by implementing MARL enviromment: one one multi- agent combat environment. Like the tasks from the previous section, this environment employs an RGB image observa- tion space and a simplified discrete action space. Agents are rewarded (+1) when punching other agents and penalized for damage (-0.1). To illustrate an example, we train the agents using self-play (Crandall & Goodrich, 2005), popu- lar method for this type of competitive scenario (Silver et al., 2017; 2018; Jiang et al., 2024). Results are presented in Figure 8c, where the policy has been trained to play against itself using PPO. The increasing episodic return curve in the figure shows how the policy learns to fulfill the task. Refer Appendix G.2 for additional figures and more details on the environment and the learning method.
Defend
spider
spider
Figure 10. Skills tree of the open-world environment in Sec- tion 3.5.3. The diagram shows the hicrarchical sequence of tasks to be unlocked by the agent. Refer to the Appendix G.3 for details.
also serves as an example of how community- -ade games in Luanti can be integrated into Craftium.
Leftmost and center images in Figure 9 illustrate part of the and diverse virtual world generated for this environ- ment. Figure 10 presents the skills tree developed for this enviromment, showing the hierarchical sequence of skills that the agent can develop to reach more complex goals. Every time the agent unlocks a skill the tool branch (e.g., collect two wood blocks), it receives a reward and new tools (e.g., wood pickaxe and sword), while the objective switches to the next skill (e.g., collect two stone blocks). Regarding the hunt and defend branches, the agent receives a reward according to the difficulty of hunting the animal or defeating the monster (refer to Appendix G.3 for details).
3.5.3. EXAMPLE 3: OPEN- WORLD ENVIRONMENTS
This section introduces an open-world environment as an example of a complex scenario for embodied AI. The envi- ronment employs the open-source VoxeLibre project (Fleck- enstein et al., 2025) for Luanti, which provides a rich and vast enviromment with many complex interactions, different biomes, animals, plants, hostile creatures. This section
To complement this example, the rightmost plot in Figure compares the achievements of PPO+LSTM and an agent
6

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
0.8
Eatodic Best
0.6
0.4
0.2
0.0
0.00
0.25
0.50
0.75
1.00
Step
166
FS
FT
(a)
(b)
Figure 11. Images in (a) illustrate examples of the procedurally generated dungeon environments from Section 3,5.4. From left to right: the dungeon's 's top view, and two observations from the agent's perspective, front of sand monster, and a spider with the diamond the back. The plot in (b) shows the episodic return curves of FS and FT-L2 in the fourth CRL task (see Appendix G.S for details).
based on the open source large multimodal model LLaVa (Liu et al., 2024a) version 1.6 by Liu et al. (2024b) (with no finetuning to this specific task). Results show that the LLa Va-Aent unlocks the collect wood and stone stages, while PPO+LSTM only completes the first one. Both meth- ods successfully hunt animals and fight some monsters. This example demonstrates Craftium's 's usage beyond RL, analyz- ing and evaluating the ability of large multimodal model- based agents to leverage world knowledge to approach com- plex open-world tasks. Additional information and details are provided in Appendix G.3.
To complement this example, we train two agents: one from scratch on each task in the sequence (FS) and another that continuously fine-tunes the previously Iearned model using L2 regularization (FT-L2), a common baseline in CRL (Gaya et al., 2023; Wolezyk et al., 2024). As shown in Figure 11b and Appendix G.4 (complete results in the appendix), FT-L2 significantly outperforms the from-scratch baseline in several environments, demonstrating forward knowledge transfer across the generated tasks.
4. Related Work
3.5.4. EXAMPLE 4: PROCEDURAL ENVIRONMENT GENERATION FOR CRL
Table includes a comparative overview of popular environ- ment frameworks from the literature. 5 The following lines provide more extensive discussion of this analysis.
This section demonstrates Craftium's versatility by imple- menting a procedural environment generator that automat- ically constructs a sequence of increasingly difficult tasks. While such a generator has broad applications, including meta-RL (Dennis al., 2020), open-endedness (Wang et al., 2023), and UED (Rigter al.. 2024), we focus on use case in CRL (Abel al., 2023). In CRL,, agents interact with sequence of environments, each constrained by timestep budget, and are expected to leverage prior experience to solve new tasks efficiently. Existing approaches typically rely on manually designed task sequences, which limits their scalability and diversity, and rely on repetitive patterns to extend them (e.g.. Wolezyk et al. (2021) and Tomilin al. (2023)). In contrast, our generator enables the automatic creation of diverse task sequences with controlled difficulty, showing how Craftium could be used to overcome these limitations.
Most of the environments employed in the literature are adaptations of video games that were not originally designed for research (Bellemare et al., 2013; Wydmuch et al., 2019; Guss et al., 2019; Küttler et al., 2020). As result, they offer limited customization, often restricted predefined param- eters (c.g., number of enemies). Examples include ALE (Machado et al., 2018), MineRL (Guss et al., 2019), and NLE (Küttler et al., 2020). The lack of flexibility hinders their use various research scenarios, such as designing custom environments to study catastrophic forgetting or analyzing specific behaviors of different learning systems. These limitations have long been recognized, and several frameworks have been proposed that allow the creation of completely new environments. For example, VizDoom (Wydmuch et al., 2019) allows defining environments us- ing ZScript, and MiniHack (Samvelyan et al., 2021) em- ploys the des file format for the same purpose. Both, ZScript and the des-file format are Domain Specific Langauges (DSL) tailored to the games they originate from (ZDoom and NetHack, respectively). However, DSLs of- ten purpose-specific and lack the flexibility and functionality of general-purpose programming languages. For instance, the des-file format is not a programming language
Conditioned on some input parameters, the generator proce- durally constructs labyrinthic 3D dungeons populated with hostile enemies. The agent has to navigate these dungeons, survive, and reach its objective, a diamond. Rewards are as- signed as +10 for collecting the diamond, +0.5 for defcating an enemy, and 0 otherwise. In this example, we generate 10 environments of increasing difficulty. Figure 1 la illustrates a generated environment, while Appendix G.4 provides fur- ther details on the generator.
Although the original ProcGen project is unmaintained, the table considers the community rewrite available at ht https:// gi rithub. con/Fa rama- -ammaaaunnniioon Procgen2.
7

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
Table 1. Popular environment frameworks compared by: numher of playable dimensions, procedural generation capahilities, environment creation, whether environments can be programmatically implemented (and not through predefined configuration options), Gymnasium support, multi-agent, and open- capabilities. We specify the language if framework allows programmatic implementation environments, and a red cross otherwise.
FRAMEWORK
DIMS.
PROC. GEN.
ENV. CREAT
PROG. DEF.
GYMNASTUM
MARL
OP. WORLD
ALE (Bellemare et 2013)
2D
X
X
X
V
X
DM LAR (Reattie et al. 2016)
3D
X
Laa
x
X
X
AI2-THOR (Kolve et al., 2017)
3D
X
X
V
VIZDOOM (Wydmuch et al., 2019)
2.5D
X
ZSeript
V
V
MINERI (Guss et 2019)
x
X
NLE (Küitter et al., 2020)
2D
X
X
PROCGEN (Cubbe al., 2020)
2D
x
V
X
MINIHACK (Samvelyan et al., 2021)
2D
des-file format
X
X
X
MINEDOJO (Fan et al., 2022)
3D
x
X
X
HABITAT 3.0 (Puig et al., 2024)
3D
X
X
CRAPTAX (Puig et al., 2024)
2D
X
CRAPTIUM
3D
Lua
per se, just a language to define NetHack levels. Addi- tionally, D often differ significantly from mainstream programming languages, which limits their usability and adoption.
parameters, making scenarios like those Section 3.5 in- feasible to implement (see Appendix H for details) and lacking multi-agent support, which hinders its adoption in this growing field.
Some frameworks offer customization through the program- ming languages in which they are implemented, avoiding the limitations of DSLs. For example, Griddly (Bamford et al., 2020) and MiniGrid (Chevalier-Boisver et al., 2023) offer Python APIs for creating grid-like 2D environments. While grid environments are fast to simulate, they lack the com- plexity and diversity of more advanced environments like MineRL. and VizDoom. Although more complex tasks could be implemented in these frameworks, it would require sig- nificant development effort for researchers. Regarding 3D environments, MiniWorld (Chevalier-Be Shhevalie-Booiseertt et al., 2023) offers similar API to MiniGrid but suffers from the same issues regarding the implementation of richer environments.
5. Conclusion
Designing new environments and modifying existing ones is crucial for advancing research in the different subfields of autonomous agents, such as RL, MARL, CRL, embodied AI, or open-endedness. However, many established envi- ronments provide limited no options for customization (Bellemare et al., 2013; Guss et al., 2019; Küttler et al., 2020; Matthews et al., 2024). Although some works offer tools for developing environments, they rely on restrictive DSLS (Wydmuch et al., 2019; Samvelyan et al., 2021) or simpler 2D worlds (Bamford et al., 2020; Leibo et al., 2021; Chevalier-Boisvent et al., 2023). Conversely, rich and com- plex 3D environments like MineDojo (Fan et al., 2022) allow limited customization, have no multi-agent support, and are built on source and prohibitively computationally expensive games like Minecraft.
On the other hand, the field of embodied AI for robotics has emphasized the importance of visually complex sce- marios (Gan et al., 2021), including popular frameworks such as A12-THOR (Kolve et al., 2017) and Habitat 3.0 (Puig et al., 2024). However, these works focus on accurate physical modeling and photorealism while having limited diversity (mostly including indoor household scenarios) and lack of open-world environments (Deitke et al., 2022; Gu et al., 2023; Wang et al., 2024). For higher-level cognitive tasks that do not require accurate physics modeling or pho- torealism, the field has popularly adopted Minecraft- an extremely popular game with rich content and diverse open worlds. Some examples are Malmo (Johnson et al., 2016) and MineRL (Guss al., 2019), which wrap Minecraft in Python interface. However, they lack support for task customization or the creation of new environments. More recently, MineDojo (Fan et al., 2022) has greatly improved customization within Minecraft-based environments. Never- theless, environment creation is constrained by predefined
This work presents Craftium, an easy-to-use and flexible framework for creating rich and fast 3D environments, Craftium's 's versatility is showcased in Section 3.5, which shows its application to train and analyze single- and multi- agent RI. algorithms, implement open-world environments for complex embodied agent tasks, and procedurally gener- ate environments for CRL. Unlike many alternatives built on top of existing video games, Craftium is based on Luanti, fully-featured open-source game engine. This analogy is also translated to the presented framework, as it is not a benchmark but ments. By leveraging the extensive and well-documented Luanti Modding API (Luanti Team, 2025a), Craftium en- ables nearly limitless possibilities for the development of custom single- and multi-agent environments. Additionally,
tool for creating environ-
8

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
Luanti has a vibrant community that has produced numerous games and extensions (Ward, 2025a), which can be easily integrated into Craftium environments (see Section 3.5.3). Moreover, its efficient implementation significantly reduces the computational cost of alternatives of comparable rich- ness. As shown in Section 3.4, Craftium achieves over 2K timesteps per second more than MineDojo, and performs competitively with VizDoom, even though VizDoom is not fully 3D. Craftium also implements the widely- Gymnasium (Towers et al., 2024) and Petting Zoo (Terry et al., 2021) interfaces, making it compatible with numerous existing tools and projects, such as Moritz et al. (2018); Huang et al. (2022b); Serrano- fin et al. (2021). Finally, Craftium is open source and pro- vides extensive documentation, including many practical examples from which users can build environments for their particular research needs.
Badia, A. P., Piot, B., Kapturowski, S., Sprechmann, P., Vitvitskyi, A., Guo, Z.. D., and Blundell, C. Agent57: Outperforming the atari human benchmark. In Proceed- ings of the 2020 International Conference on Machine Learning (ICML), 2020a.
Badia, A. P., Sprechmann, P., Vitvitskyi, A., Guo, D., Piot, B., Kapturowski, S., Tieleman, O., Arjovsky, M., Pritzel, A., Bolt, A., et al. Never give up: Learning directed explo- ration strategies. In Proceedings the
the 2020 International Conference on Learning Representations (ICLR), 2020b.
Bamford, C., Huang. S., and Lucas, S. Griddly: A platform for ai research in games. arXiv preprint arXiv:2011.06363, 2020.
et al. (2023) and Raf-
Beattie, C., Leibo, J. Z.,, Teplyashin, D., Ward, T., Wain- wright, M., Küttler, H., Lefrancq, A., Green, S., Valdés, V., Sadik, A., et al. Deepmind Lab. arXiv preprint arXiv: 111.::161,.03801, 2016.
Acknowledgements
We are grateful to Jose A. Pascual for the technical sup- port and to Jon Vadillo and Ainhize Barrainkua for reading preliminary versions of the paper. We also thank the Lu- anti developers and community for their ongoing efforts to maintain and continuously improve the engine and its ecosystem.
Bellemare, M., Srinivasan, S., Ostrovski, G., Schaul, T., Saxton, D., and Munos, R. Unifying count-based ex- ploration and intrinsic motivation. In Proceedings of the 202 Advances in Neural Information Processing Systems (NeurIPS), 2016
Bellemare, M. G., Naddaf, Y., Veness, J., and Bowling, M. The Arcade Learning Environment: An evaluation plat- form for general agents. Journal of Artificial Intelligence Research, 47:253 2013.
Mikel Malagón acknowledges a predoctoral grant from the Spanish MICIU/AEI with code PREP2022- 000309, associated with the research project PID2022-1337422B--00 funded by the Spanish MI- CIU/AEN/10. 100011033 and FEDER, EU. Josu Ceberio has been partially supported by the Spanish 1tt./////....000//122222223200
Burda, Y., Edwards, H., Storkey, A., and Klimov, O. Ex- ploration by random network distillation. In Proceedings of the 2019 International Conference on Learning Repre- sentations (ICLR), 2019.
This work is also funded through the BCAM Severo Ochoa accreditation CEX2021-001 142- AMICII/AAE//O01103991/ 10001 1033; and the Research Groups 2022-2025 (IT1504-22), the BERC 2022-2025 program, and Elkartek (KK-2024/00030) from the Basque Government.
Chen, B., Xu, Z., Kirmani, S., Ichter, B., Sadigh, D.,, Guibas, L., and Xia, F. SpatialVL.M: Endowing vision-language models with spatial reasoning capabilities. In Proceed- ings of the 2024 IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Chevalier-Boisvert,, M., Dai, B., Towers, M., de Lazcano,
Impact Statement
R., Willems, L., Lahlou, S., Pal, S., Castro, P. S., and Terry, J. Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks. arXiv preprint arXiv:2306. 230.133333, 2023.
This paper presents work whose goal to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Cobbe, K., Hesse, C., Hilton, J., and Schulman, J. Lever- aging procedural generation to benchmark reinforcement learning. arXiv preprint
References
2020.
Abel, D., Barreto, A., Van Roy, B.,, Precup, D.,, van Hasselt, H. P., and Singh, S. A definition of continual reinforce- ment learning. In Proceedings of the 2023 Advances in Neural Information Processing Systems (NeurIPS), 2023.
Crandall, J. W. and Goodrich, M. A. Learning to com- pete, compromise, and cooperate in repeated general-sum games. In Proceedings of the 2005 International Confer- ence on Machine Learning (ICML), pp. 161-168, 2005.

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
Deitke, M., VanderBilt, E., Herrasti, A., Weihs, L.,, Ehsani, K., Salvador, J., Han; W., Kolve, E., Kembhavi, A., and Mottaghi, R. ProcTHOR: Large-scale embodied AI using procedural generation. Proceedings of the 2022 Advances in Neural Information Processing Systems (NeurlPS), 2022.
scalable continual learning. In Proceedings of the 2023 International Conference on Learning Representations (ICLR), 2023.
Grbic, D., Palm, R. B., Najarro, E.,, Glanois, C.,, and Risi, S. Evocraft: A new challenge for open-endedness. In Proceedings of the 2021 Applications of Evolutionary Computation: 24th International Conference, EvoAppli- cations 2021, pp. 325-340) Springer, 2021.
Dennis, M., Jaques, N., Vinitsky, E., Bayen, A., Russell, S., Critch, A., and Levine, S. Emergent complexity and zero- shot transfer via unsupervised environment design. In Proceedings of the 2020 Advances in Neural Information Processing Systems (NeurIPS), 2020.
Gu, J., Xiang, F., Li, X., Ling, Z., Liu, X., Mu, T., Tang, Y., Tao, S., Wei, X., Yao, Y., Yuan, X., Xie, P., Huang. 7 Chen, R., and Su, H. ManiSkill2: A unified benchmark for generalizable manipulation skills. In Proceedings of the 2023 International Conference on Learning Repre- sentations (ICLR), 2023.
Earle, S., Kokkinos, F., Nie, Y., Togelius, J., and Raileanu, R. Dreamcraft: Text-guided generation of functional 3d environments minecraft. In Proceedings of the 2024 International Conference on the Foundations of Digital Games (FDG), 2024.
Guss, W. H., Houghton, B., Topin, N., Wang. P., Codel, C., Veloso, M., and Salakhutdinov, R. MineRL: large- scale dataset of minecraft demonstrations, arXiv preprint arXiv: 13440, 2019.
Faldor, M., Zhang, J., Cully, A., and Clune, J. OMNI- EPIC: Open-endedness via models of human notions of interestingness with environments programmed in code. In Proceedings of the 2025 International Conference on Learning Representations (ICLR), 2025.
Hafner, D. Benchmarking the spectrum of agent capabilities. In Proceedings of the 2022 International Conference on Learning Representations (ICLR), 2022.
Fan, L., Wang, G., Jiang, Y., Mandlekar, A., Yang, Y., Zhu, H., Tang, A., Huang, D.-A., Zhu, Y., and Anand- kumar, A. MineDojo: Building open-ended embodied agents with internet-scale knowledge. Proceedings of the 2022 Advances in Neural Information Processing Systems (NeurIPS), 2022.
Huang, S., Dossa, R. F. J., Raffin, A., Kan- ervisto, A., and Wang, W. The 37 implemen- tation details of proximal policy optimization. In ICLR Blog Track, 2022a URL https: f/iclr-blog-t 03/25/ppor o0//2//pooimpleeenntaaeee
Fleckenstein, L., Wuzzy, davedevils, and contribu- tors. VoxeLibre, a voxel-based sandbox game for luanti. https: miiitesttttt...... / VoxeLibre/VoxeLiin 2025. Accessed: 2025-05- 21.
Huang, S., Dossa, R. F. J., Ye, C., Braga, J., Chakraborty, D., Mehta, K., and Araújo, J. G. CleanRL: Highh-quality single-file implementations of deep reinforcement learn- ing algorithms. Journal of Machine Learning Research, 23(274):1-18, 2022b.
Gan, C., Schwartz, J., Alter, S., Mrowea, D., Schrimpf, M.,
Hughes, E., Dennis, M. D., Parker-Holder J., Behbahani, F., Mavalankar, A., Shi, Y., Schaul, T., and Rocktäschel, T. Position: Open-endedness is essential for artificial superhuman intelligence. In Proceedings of the 2024 International Conference on Machine Learning (ICML),
Traer, J., Freitas, D., Kubilius, J., Bhandwaldar, A.,
Haber, N., Sano, M., Kim, K., Wang, E., Lingelbach, M., Curtis, A., Feigelis, K. T., Bear, D., Gutfreund, D.,
Cox, D. D., Torralba, A., DiCarlo, J. J., Tenenbaum, J. B., Medermott, J., and Yamins, D. L. ThreeDWorld: A plat- form for interactive multi-modal physical simulation. In Proceedings of the 2021 Advances Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2021.
2024.
Ierusalimschy, R. Programming in Lua. Roberto Ierusalim- schy, 2006.
Jiang, Y., Liu, Q.. Ma, X., Li, C., Yang, Y., Yang, J., Liang, B., and Zhao, Q. Learning diverse risk preferences in population-based self-play. In Proceedings of the 2024 AAAI Conference Artificial Intelligence, 2024.
Garcin, S., Doran, J., Guo, S., Lucas, C. G., and Albrecht, S. V. DRED: Zero-shot transfer in reinforcement learning via regulaagglr environment design. In Proceedings of the 2024 International Conference on Machine Learn- ing (ICML), 2024.
Johnson, M., Hofmann, K., Hutton, T., and Bignell, D.
The Malmo platform for artificial intelligence experimen- tation. In Proceedings of the 2016 International Joint Conference on Artificial Intelligence (IJCAI), volume 16, pp. 4247, 2016.
Gaya, J.-B., Doan, T., Caccia, L., Soulier, L., Denoyer, L., and Raileanu, R. Building a subspace of policies for
10

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
Jordan, S. M., White, A., Silva, B. C. D., White, M., and Thomas, P. S. Position: Benchmarking is limited in rein- forcement learning research. In Proceedings of the 2024 International Conference on Machine Learning (ICML), 2024.
Matthews, M., Beukman, M., Ellis, B., Samvelyan, M., Jackson, M. T., Coward, S., and Foerster, J. N. Craftax: A lightning- fast benchmark for open-ended reinforcement learning, In Proceedings of the 2024 International Con- ference on Machine Learning (ICML), 2024.
Kolve, E., Mottaghi, R., Han, W. VanderBilt, E., Weihs, L., Herrasti, A., Deitke, M., Ehsani, K., Gordon, D., Zhu, Y., et al. AI2-THOR: An interactive 3D environment for visual AL, arXiv preprint arXiv: 1712.0547 2017.
Mnih. V., Kavukcuoglu. K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. Playing atari with deep reinforcement learning. In NIPS Deep Learning Workshop 2013, 2013.
Küttler, H., Nardelli, N., Miller, A., Raileanu, R., Selvatici, M.. Grefenstette, E., and Rocktäschel, T. The NetHack Learning Environment. Proceedings of the 2020 Ad- vances in Neural Information Processing Systems, 2020.
Moritz, P., Nishihara, R., Wang, S., Tumanov, A., Liaw, R., Liang, E., Elibol, M., Yang. Z., Paul, W., Jordan, M. I., al. Ray: A distributed framework for emerging AI applications. In USENIX Symposium Operating Systems Design and Implementation (OSDI) of 2018, pp.
Leibo, Z.., Duzmaaz-Guzman, E. A., Vezhnevets, A., Aga- piou, J. P.. Sunchag, P.. Koster, R., Matyas, J., Beattie, C., Mordatch, I., and Graepel, T. Scalable evaluation of multi-agent reinforcement learning with Melting Pot. In Proceedings of the 2021 International Conference on Machine Learning (ICML), 2021.
561-577, 2018.
Ostrovski, G., Bellemare, M. G., Oord, A., and Munos, R. Count-based exploration with neural density models. In Proceedings of the 2017 International Conference on Machine Learning (ICML), pp. 2721-2730, PMLR, 2017.
Paolo, G., Sonzalez-Billladdon J., and Kégl, B. Position: A call for embodied AI. In Proceedings of the 2024 International Conference on Machine Learning (ICML), 2024.
Liu, H., Li, C., Li, Y., and Lee, J. Improved baselines with visual instruction tuning. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024a. 2024a.
Prasanna, S., Farid, K., Rajan, R., and Biedenkapp. A. Dreaming of many worlds: Learning contextual world models aids zero-shot generalization. arXiv preprint arXiv:2403. 10967, 2024.
Liu, H., Li, C., Li, Y., Li, B., Zhang. Y., Shen, S., and Lee, Y. J. Llava-next: Improved reason- ing, ocr, and world knowledge, January 2024b. URL https://llavaaa 2024-01 3024-01-30-1lava-mees
watthub.io/bllc.
Puig, X., Undersander, E., Szot, A., Cote, M. D., Yang, T.-Y., Partsey, R., Desai, R., Clegg, A., Hlavac, M., Min, S. Y., Vondruš, V., Gervet, T., Berges, V.-P., Turner, J. M., Maksymets, O., Kira, Z., Kalakrishnan, M., Malik, J., Chaplot, D. S., Jain, U., Batra, D., Rai, A., and Mot- taghi, R. Habitat 3.0: A co-habitat for humans, avatars, and robots. In Proceedings of the 2024 International Conference on Learning Representations (ICLR), 2024.
Luanti Team. Luanti's modding API reference. https:
//api. peppi.luanti.orpe 2025a. Accessed: 2025-05-
21.
Luanti Team. Luanti's main page. /ttps://www. luanti. 2025b. Accessed: 2025-05-21.
Luanti Wiki.
Luanti's wiki FAQ page.
Raad, M. A., Ahuja, A., Barros, C., Besse, F., Bolt, A., Bolton, A., Brownfield, B., Buttimore, G., Cant, M., Chakera, S., et al. Scaling instructable agents across many simulated worlds. arXiv preprint 2XXi:2404.101799 2024.
https:://eww.aaatii.c www.aaantirogggg//////////
ris-luanti-a-cnn caattti---cccneeeeee ooone-of----e-oo-- of-minerraft, 2025.
Accessed: 2025-05-21.
Machado, M. C., Bellemare, M. G., Talvitie, E., Veness, J., Hausknecht, M., and Bowling, M. Revisiting the Arcade Learning Environment: Evaluation protocols and open problems for general agents. Journal of Artificial Intelligence Research, 61 31:5225662, 2018.
Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., and Dormann, N. Stable-Baselines. Reliable rein- forcement learning implementations. Journal of Machine Learning Research, 22(268): 2021.
Malagon, M., Ceberio, J., and Lozano, J. A. Self-composing policies for scalable continual reinforcement learning. In Proceedings of the 2024 International Conference on Machine Learning (ICML), 2024.
Rigter, M., Jiang. M., and Posner, I. Reward curricula for training robust world models. In Proceedings of the 2024 International Conference on Learning Representa- tions (ICLR), 2024.

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
Rimon, Z., Jurgenson, T., Krupnik, O., Adler, G., and Tamar, A. MAMBA: effective world model approach for meta-reinforreement learning. In Proceedings of the 2024 International Conference on Learning Representations (ICLR), 2024.
Towers, M., Kwiatkowski, A., Terry, Balis, J. U., De Cola, G., Deleu, T., Goulão, M., Kallinteris, A., Krimmel, M., KG, A., et al. Gymnasium: A standard interface for reinforcement learning environments. arXiv preprint arXiv:2407. 20ii:24077.10332, 2024.
Rutherford, A., Ellis, B., Gallici, M., Cook, J., Lupu, A., Ingvarsson, G., Willi, T., Khan, A., de Witt, C. S., Souly, A., Bandyopadhyay, S., Samvelyan, M., Jiang, M., Lange, R. T., Whiteson, S., Lacerda, B., Hawes, N., Rocktaschel, T., Lu, C., and Foerster, N. JaxMARL: Multi-agent Il environments in JAX. In Proceedings of the 2024 NeurlPS Datasets and Benchmarks Track, 2024.
Wang. G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C.,, Zhu, Y., Fan, L., and Anandkumar, A. Voyager: An open- ended embodied agent with large language models. arXiv preprint arXiv: Arxiv-2305. 16291,2211222 2023.
Wang, Y., Xian, Z., Chen, F., Wang. T.-H., Wang, Y., Fragki- adaki, K., Erickson, Z., Held, D., and Gan, C. RoboGen: Towards unleashing infinite data for automated robot learning via generative simulation. In Proceedings of the 2024 International Conference on Machine Learning (ICML), 2024.
Samvelyan, M., Kirk, R., Kurin, V., Parker-Holder, J., Jiang, M., Hambro, E., Petroni, F., Küttler, H., Grefenstette, E., and Rocktäschel, T. MiniHack the planet: A sandbox for open-ended reinforcement learning research. Proceedings of the 2021 NeurlPS Datasets and Benchmarks Track, 2021.
Ward, A. ContentDB: a content database for Luanti mods, games, and more. https: wwtt/ottttttttccccccccoccteccttt /oontettt..tntnii..... luanti. org/, 2025a. Accessed: 2025-05-21.
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.07777. 2017.
Ward, A. Luanti modding book. https:
/ /rrbbewwardy book / en hon//ndex.hiie html, 2025b. Accessed: 2025-05- 21.
Serrano-Muñoz, A., Chrysostomou, D., Bøgh, S., and Arana-Arexolaleiba, N. skrl: Modular and flexible li- brary for reinforcement learning. Journal of Machine Learning Research, 24(254):1-9,005)
Wotezyk, M., Zajac, M., Pascanu, R., Kuciñski, E.., and Miloš, P. Continual world: A robotic benchmark for continual reinforcement learning. Proceedings of the 202/ Advances in Neural Information Processing Systems (NeurIPS), 34:28496 14:28496-28510,, 2021.
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. Mastering the game of Go without human knowledge. Nature, 550006665507777
Wolczyk, M., Cupial, B., Ostaszewski, M., Bortkiewicz, M., Zajkac, M., Pascanu, R., Kuciñski, E., and Mitoš, P. Fine-tuning reinforcement learning models is secretly a forgetting mitigation problem. In Proceedings the 2024 International Conference on Machine Learning (ICML), 2024.
2017.
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L.,, Kumaran, D., Grae- pel, T., Lillicrap. T., Simonyan, K., and Hassabis, D. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362 (6419):1140-114) 2018
Wydmuch, M., Kempka, M., and Jaskowski, W. ViZDoom Competitions: Playing Doom from Pixels. IEEE Trans- actions on Games, 1 11(3):288-259, 2019.
Sutton, R. S. and Barto, A. G. Reinforcement Learning: An Introduction. The MIT Press, 2018
Ying, D., Zhang, Y., Ding, Y., Koppel, A., and Lavaei, J. Scalable primal-dual actor-critic method for safe multi- agent RI. with general utilities. In Proceedings of the 2023 Advances in Neural Information Processing Systems (NeurIPS), 2023.
Terry, J., Black, B., Grammel, N., Jayakumar, M., Hari, A., Sullivan, R.. Santos. S., Dieffendahl, C., Horsch, C. Perez-Vicente, R., et al. PettingZoo: Gym for multi-agent reinforcement learning. Proceedings of the 2021 Advances in Neural Information Processing Systems (NeurIPS), 2021.
Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., and Levine, S. Meta- World: A benchmark and evalu- ation for multi-task and meta reinforcement learning. In Proceedings of the 2020 Conference on Robot Learning (CoRL), 2020.
Tomilin, T., Fang, M., Zhang, Y., and Pechenizkiy, M. Coom: a game benchmark for continual reinforcement learning. Proceedings of the 2023 Advances in Neural Information Processing Systems (NeurIPS), 2023.
12

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
A. Modifications to Luanti
Although Luanti is an extremely flexible game engine with extensibility built into its core, we had to modify its source code for this work. As Luanti is a large C++ project with thousands of files, modifications have been thoughtfully introduced to minimize possible conflicts with future updates of the engine. Most of the introduced code is limited cra ft i um. h file and some modifications the cl ieeeet... cpp and game . cpp files. These are the main modifications that have allowed running autonomous agents in Luanti:
dedicated
Implementation of a client that connects to the Python process with the agent's implementation. This is the communi- cation channel from which Luanti sends RGB frames and other timestep data to Python, and Python sends the next actions to be executed.
Executing the agent's actions keyboard and mouse commands in Luanti. All actions are translated as virtual keyboard keypresses or mouse movements (for moving the camera and controlling the inventory).
Extensions to the Luanti API to incorporate vital functionalitics for RL environments. Extensions include new Lua functions that implement functions such as setting the episode termination flag or sending reward values.
Luanti has client/server architecture, where the server runs the world's logic and the client interfaces with the player (e.g., game control and rendering). However, the asynchronous nature of this architecture introduces issues when using slow agents (e.g., large multi-modal models). For example, the server could update the world many times while the client waits for the agent to return an action. This causes many reproducibility issues and behaviors, such as monsters attacking the player while the client waits for the agent's response. For this purpose, Craftium introduces optional synchronous client/server updates. This ensures that (when needed) the server waits for the client be updated before continuing with the next update.
Related to the previous modification, even fast agents (e.g., smaller NNs) can introduce small delays (i.e., lags) to Luanti, for example, when training the model in batches while running the environment. Consequently, we have modified Luanti to avoid being affected by these delays and ensure the environment's and physics's coherence.
Resetting episodes in complex environments can be time-consuming, sometimes requiring closing and restarting the internal engine of the environment. In consequence, the training time in environments with frequent episode resets (c.g., hard survival games like SpidersAttack) increases substantially. To avoid these hard resets, we implement several functionalities in Craftium to soft reset the environment, not requiring reinitializing the engine. Unlike hard resets, soft resets delegate the reset to the environment itself, to the Lua mod in the case of Craftium (see Appendix C for more details).
B. Action Space Details
B.1. Default Action Space
The default action space of Craftium environments is composed of combinations of 21 keyboard actions and mouse movements on the horizontal and vertical axes. Keyboard actions are binary values, where translates to a key press and O if not used. Available keyboard commands are listed and described in Table 2. Note that these actions are a subset of the default keyboard controls that Luanti offers and its selection inspired by the action space of MineRL. (Guss et al., 2019). Mouse movements are defined by a tuple (horizontal and vertical movements) of real values in the [-1, 1] interval (see Section 3.1).
B.2. Action Wrappers
By default, Craftium environments have a large action space with discrete (binary) and continuous values (see Section 3). However, many tasks do not require the complete default action space and can be greatly simplified by considering only the relevant actions solve the specific task that the environment defines. Consequently, Craftium provides tools for
Some controls like pausing the game opening the chat have been excluded. For additional information, visit: https: / //dev. paanti. www.//rr/ooooo con/trlstrr....
13

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
Table List of available keyboard actions in Craftium environments, their corresponding key the default Luanti controls, and their description.
ACTION
KEY
DESCRIPTION
Forward
W
Move the player forward.
Backward
S
Move the player backward.
Left
A
Move the player left.
Right
D
Move the player right.
Jump
Space
Jump and move up.
Aux
E
Run faster.
Sneak
Shift
Sneak, move downwards. Zoom in at the center of the camera.
Zoom
Z
Dig
Left mouse button
Punch if using weapon or mine if using tool.
Use the pointed object if usable, otherwise attempt to build at the pointed block. Drop the wielded item. Show/hide inventory.
Place
Right mouse button
Drop
Q
Inventory
Slot [1-9]
0-9
Select the item in the [0-9] position of the hotbar.
customizing the action space of environments by using Gymnasium Wrappers.? Specifically, Craftium implements two rand wet.rettecctiiiraaapppppe
BinaryActionWaapet allows selecting the subset of keyboard actions (see Table 2 for the complete list) to use in the new action space. This wrapper also simplifies the continuous mouse movement actions by discretizing them into four binary actions: move the mouse left, right, up, and down. The magnitude of these movements can be chosen by the developer. For example, this wrapper allows simplifying the default {0, 1]21 U [0, 1J2 action space into {0, 1]3 space where binary values correspond to: move forward, move mouse right, and move mouse left.
DiscreteAct onnrraperr allows selecting the subset of keyboard actions and discretizes the mouse movement similarly to the previous wrapper. However, in this case, actions are not binary vectors but a single discrete value. Thus, actions can not be combined as in the case the previous wrapper. Following the previous example, instead of simplifying the default action space into (0, 1}3 this wrapper defines the new space as 1,2]. where 0 corresponds to move forward, I moves the mouse to the right, and moves to the left.
C. Extensions the Luanti Modding API
Luanti counts with an extensive and powerful API (Luanti Team, 2025a) that can be used to modify the behavior of the game engine and create mods or entire games (Ward, 2025a). However, Luanti lacks the functionality to define RL environments by itself. Therefore, Craftium distributes modified version of the game engine (see Appendix A) that includes additional functionalities in the API to make possible implement RL environments from Luanti mods. Table lists and describes the new functions added the API. Note that besides basic RL environment funetionalities, these additions the API also include functions for soft resetting the environments (see Appendix A for additional information).
D. Using Craftium through the PettingZoo (Multi-Agent) Interface
Figure 12 shows an example use case of the PettingZoo API in Craftium for multi- environments. Note that PettingZoo greatly inspired by Gymnasium and shares many similarities and design choices." 9
Like the Gymnasium example from Figure 6, the first lines (1-5) instantiate Craftium environment by name. In this case, Chattuum/MMliiAggnnCCmbbee is loaded, corresponding to the multi-agent environment example showcased in
"Refer Gymnasium's documentation for more information: https: /typ:::///ymmmsiuum farama wttpaaaaaa//p///////a/aaaaeaee/e /wwappers/
act ionn wmm_wwrapers///
"More information at: https:: //ttt:://pettiie http://ttttttto.....o/ farama org//pi/aac.//////
"In fact, both projects are developed under the same Farama foundation, see https: htappp:://faaaaaaa htf.aaaaaaarrr.////
14

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
Table List of the functions added the Luanti API. The "_" character is used indicate that function takes no arguments.
NAME
PARAMETERS
DESCRIPTION
Sets the reward value to the given value until another call to function that modifies the reward is made. Returns the reward value of the current timestep, and ni if not set.
reward
float
get get_reward
Sets the reward to the first parameter only for the current timestep. resetting it to the second parameter afterwards. Sets the termination flag true for the current timestep. Returns a if the termination flag is set to true, otherwise. Resets the episode termination falg.
set ret_reward_ence
float, float
terminatiionn
get_termination
reset_termination
get ret_sofftreest
Returns whether the environment should soft reset.
Section 3.5.2. Then, line resets the environment the initial state, initializing Luanti for the first time internally, Next, lines 9-16 define the main agent-environment interaction loop. As defined in line 9, the loop cycles through the agents (two agents for this specific environment). Line 10 obtains the observation, reward, fermination/rrunation flags, and the information dictionary (similarly to the Gymnasium example). Next, lines 12-13 check if the episode should terminate. If the episode continues, line 15 selects the action for the current agent, and line 16 executes it, rumning single environment step for the current agent. Finally, 18 closes the environment, shutting down Luanti and removing any temporary files.
from craftium import
3 env pettingzoo_ env.
4
s
7 env.reset. ()
9 for agent_ic id in env agent_iter()
observation, reward, termination, truncation, info = env. last ()
if termination or truncation: break
13
14
action agents [agent_id] (observation) env. step (action)
18
env. clo.close ()
Figure Python code illustrating an example multi-agent scenario using the PettingZoo interface in Craftium.
E. Performance Benchmarks
Due to the page limit constraint the paper, this section extends Section 3.4 in the main text, including additional details and analyses for different setups.
Single-nnviromentt.
To complement the results illustrated in Figure 7, Table provides the exact average and standard deviation values. The measurements aggregate the results of different runs of 1K steps in environments per framework. Note that all environments considered for this experiment were single-agent, as MineDojo does not support multi-agent scenarios 10 and VizDoom does not provide multi-agent environments (although technically supports this setting)." " The environments were: Speleo, Room, and Spiders Attack for Craftium (see Appendix G.1); VicdoomHHalttGGahherig v0, VizdoomCorridor-1,, and VizdoomDeefnndCenttrrv// for VizDoom; and harvest milk, creative:255, and Harvest for
10 Revelaat discussion at (accessed May 2025):
Il For details the multi-agent capabilities of VizDoom (accessed May 2025): https: / thub.com/ Fa Darama-Founde i 2Doom/ issues/546.
15

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
MineDojo. In all cases, observations were RGB images, without frameskip, and actions were selected uniformly random. In the case of MineDojo and Craftium environments observation size was set to 64 64 pixels, and to 320 240, as the latter resolution is not available for VizDoom environments.
As can be observed in Table 4, Craftium achieves substantially higher steps per second than the Minecraft alternative, MineDojo. The reasons for such a significant performance gap are many, as both frameworks are complicated systems with many interacting components. One of the most significant differences is the choice of implementation language: MineDojo based on Minecraft, which is implemented in Java 8, while Craftium relies on Luanti, implemented in C++ and known to perform significantly higher than Java. 12 Another relevant aspect that Minecraft is a complete game, which has grown in complexity over the years, directly affecting the environments implemented on it. As it is closed-source game, developers are not allowed to modify its source code to remove irrelevant parts of the game for the environment at hand to improve computational efficiency. Contrarily, Luanti is open source and exposes highly flexible API to modify its behavior. This allows building environments with only the relevant components for the task at hand. Along the same line, the open-source nature of Luanti allowed its modification to tightly integrate it with the proposed framework. For example, to incorporate a system to execute the actions sent from the Python interface as keyboard and mouse commands. Conversely, Minecraft does include many layers of complexity to
not allow modifications to its source code, which requires MineRL and MineDojo adapt the Minecraft game to the RL setting. Most notably, Minecraft is game and is not intended to on server without a monitor. Therefore, MineRI. and MineDojo use external tool, Xyfb. 14 to emulate a monitor without showing any screen output, which causes significant performance drawbacks. This also implies that the X11 windowing system 15 is installed, which is not often the case in HPC clusters.
Table 4. Average and standard deviation values ohtained in the environment framework performance comparison conducted in Section 3.4.
FRAMEWORK
STEP/S
CRAFTIUM
2746.691230.41
VIZDOOM
2091.91
MINEDOJO
71.87+11.82
Parallel environments. Beyond single environment setups, Figure 13a compares Craftium and VizDoom in vectorized (asynchronous) environments, popularly employed in many on-policy RL methods (e.g., A2C or PPO). As can be observed, Craftium significantly benefits from parallelization, achieving comparable performance to the simpler VizDoom, see Appendix F. In the best setting for the employed hardware, Craftium surpasses the 12K steps per second using the same number of environments as CPU cores (16 in this case), an unprecedented efficiency for such rich and complex 3D environments. In contrast, MineDojo lacks parallel environment support, and despite efforts, we could not include this framework in Figure 13a. 16 This issue makes MineDojo impractical for many research scenarios, where learning from parallel environments can significantly enhance performance and reduce costs.
Observation size. Although many RL tasks might require relatively small observation resolution, as the 64 x 64 pixel resolution employed in Section 3.5, some applications might require larger observation sizes, such as large multimodal mosel-based agents (see Section 3.5.3). Figure 13b shows Craftium's 's loss in performance for various observation sizes relative to the steps per second achieved with the x 64 pixel resolution. As can be seen, Craftium's step per second loss for 128 x 128 pixel observations is minimal: less than 5% compared to the baseline performance with 64 x 64 pixels. For larger resolutions, 512 x 512 pixels in this case, the performance drops considerably, around 33%. However, in such cases, the performance bottleneck is likely in the model processing the images (c.g., VLM) rather than in Craftium itself.
Environment's complexity. Another important aspect that impacts an environment's performance is its complexity richness. Note that to ensure a fair analysis and comparison, performance benchmarks in this paper consider environments of
Foo example, see the performance comparison wawpshaarkssgmm/eeaaseee parieeessttptttteeee java. html.
https: https///eeeencnnagnneeeeee team. pages. debian. an.ttt/
mages.debiane
Noot that MineDojo is based on MineRL. Refer to the work by Fan et al. (2022) for details.
'"See https:://eet wikipedia. org./ii...rr//.... wiki//vvžb
'SSee httpp:://en.wwiiipdiiiii org/wi /wtp//wiiii/ wiikii//_M
'SSee pettps:/giihhi wttp:/h...////e/////////////// neDo ssues/9 96 (accessed May 2025).
16

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
10000
Step/s
8000
-15
6000
-20
Craftium (Frameskip - 4)
-25
4000
Craftium (Frameskip 8)
-30
VizDoom
2000
-35
64x64
128x128 Observation Size
512x512
10
20 Environments
25
30
(a) Parallel asynchronous environments.
(b) Resolution's impact on performance.
7000
6000
5000
Stepis
Scepls
-2
-3
1000
-5
0
world
minetest_wen
World Type
Number of Agents
(c) Comparison of environments of different richness.
(d) Number of agents and relative performance.
Figure Additional performance henchmarks and comparisons. figures above aggregate results from repetitions per setup, running IK time steps each.
different nature and requirements, see the beginning of Appendix E for details. To analyze how richness affects Craftium' performance, Figure 13c benchmarks environments using worlds (see Section 3.2) of different complexity. The figure shows the results obtained using world types: flat_world, simple flat world without procedural generation biomes," !7, and mi dungeons, plants, etc. 18 Results for flat_world were collected using the Room and Spiders Attack environments (see Appendix G.I), while Chop Tree and Speleo were used for minetest_ worrdwoordd Finally. four parallel environments were employed for both cases. Observing Figure 13c, we see that the world's complexity affects the Craftium's 's performance (around 30% in this case). However, Craftium's versatility allows the developer to choose within this richness-performance tradeoff, selecting the relevant parts for their specific needs, while discarding unnecessary complexities: enabling procedural generation or not, including animals or NPCs, additional biomes, etc.
substantially more complex world with procedural generation, biomes, underground
Number of agents. Regarding Craftium's multi-agent capabilities, in Figure 13d we analyze how the number of agents operating in the same environment impacts performance. The figure analyzes the loss in steps per second as the number of
See https: //oo/////./....////// Tuanti www.rr//a..g./seri...ooo / More information https: //conentt luanti, httpa//rrr//acccas///aaaaaa /enittimmiiieeee /minetest__gammmm
17

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
agents increases; for each agent, not in total, 19 and relative the steps per second achieved with two agents. As can be seen, although the number of agents might decrease the relative steps per second performance, the maximum average loss is lower than 4%. Moreover, the figure shows no noticeable relationship between the two axes: adding more agents has negligible impact on the relative step per second reached. Finally, at the time of this writing, Craftium supports maximum number of agents equal to the number of CPU cores of the machine (16 in the case of Figure 13d). This issue limits Craftium's 's usage on massively multi-agent environments, which we aim to address this issue in future updates.
Memory usage. Besides steps per second analyzed in the previous benchmarks, another important efficiency measure the memory usage of an environment. For instance, memory requirements directly limit the number of parallel environments that can be employed (as studied in the paragraph above on parallel environments). To fairly compare Craftium to VizDoom and MineDojo, we analyzed the memory usage of these frameworks across different tasks, the same ones as in the single- environment paragraph at the beginning of this appendix. Results show that Craftium (660MB) is notably lighter than MineDojo (I.7GB), the only framework with comparable environment richness. This result highlights Craftium's 's capabilities to create lightweight environments that avoid extra complexities in tasks that do not require them. Finally, VizDoom (84MB) the lightest due to its minimalist design. However, reduced memory usage comes at the cost of simplicity, which limits its diversity (e.g., no 3D) and its application to a broad range of research fields (refer Appendix F for a detailed discussion on the topic).
F. Limitations of 2.5D Environments
VizDoom is based on ZDoom, a modern and open-source implementation of the original Doom game. The Doom game, released in 1993, employed innovative rendering techniques that made it appear 3D while not having fully three-dimensional scenarios. These rendering techniques, referred as 2.5D perspective, make VizDoom environments computationally efficient while having some visual features of 3D scenarios. However, 2.5DD graphics limits VizDoom environments from an autonomous agent research standpoint compared to fully 3D frameworks such Craftium. Some of the most notable limitations are the following:
The agent's viewpoint is restricted to a horizontal plane, preventing it from truly looking up or down.
Level height (floor and ceiling) is stored in a 2D matrix, making it impossible to create overlapping structures like bridges, floors, or buildings.
Enemies and objects are 2D sprites that change in size and angle based on the agent's 's position.
These limitations make VizDoom environments significantly different from more realistic and diverse 3D scenarios those in Craftium, failing to cover fundamental challenges for autonomous agents that are of interest for current research, e.g.. spatial 3D reasoning (Chen et al., 2024) and complex agent-cnvironment interactions (Wang et al., 2023).
Furthermore, 2.5D environments greatly limit the diversity of tasks and scenarios, which is particularly relevant for areas like continual reinforcement learning, unsupervised environment design, and meta-learning, all of which are of growing interest to the research community (Hughes et al., 2024). Craftium is especially relevant to these fields, as it enables diverse, 3D, vast open-world environments that are computationally efficient and support multi-agent scenarios, opening the door exciting future research directions.
G. Details on the Illustrative Examples
Due to the size limitations of the main paper, this section includes additional information on the illustrative examples shown in Section 3.5.
G.I. Environments for Single-Agent RL
All tasks share the same observation space of 64 x 64 pixel RGB images. In all cases, the action space has been simplified into a discrete space E 10, 1,2....) as described Section 3.1 (see Appendix B.1 for details). The simplified action space
The relative performance is computed as total steps per second (running the agents in serial, not in parallel) divided by the numher of agents.
2 See https:///en wi kipida.. ieteedia.or//aae worg/witi httg/wiki/// /2.
18

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
(a) Chop tree.
(b) Small room.
(c) Room.
(d) Speleo.
(e) Spiders attack.
Figure 14. Visualizations of the example environments for single-agent RL Section 3.5.1.
also introduces nop action (do nothing) to all tasks. The following lines describe the five tasks introduced in this section.
Chop tree. The agent placed dense forest, equipped with a steel axe (see Figure 14a). Every time the agent chops a tree, a positive reward of is given; 0 otherwise. Therefore, the task is chop as many trees as possible until episode termination. Available actions are nop, move forward, jump, dig (used to chop), and move the mouse left, right, up, and down. Episodes terminate when 2K timesteps are reached.
Room and small room. These tasks present the same objective in different scenarios. In both cases, the agent is placed in one half of closed room with red block in the other half of the room. The objective is to reach this block as fast as possible. The difference between both tasks is the size of the room (see Figures 14c and 14b). The reward is constant; all timesteps have reward value of -1, and the episode terminates when the agent reaches the block. To avoid solving the task by memorization, the initial position of the agent and the red block randomized in every actions are: move forward, move mouse left, and move mouse right. The timestep budget is 1K in SmallRoom, and 2K for the variant with the larger room. Four actions are available: nop, move forward, and move the mouse right and left.
episode. Available
Speleo. The agent is located in closed cave illuminated with torches (see Figure 14d). The task is reach the bottom of the cave as fast possible. For this purpose, the reward each timestep the negative altitude (Y-axis position) of the agent. Therefore, the reward increases the agent goes deeper into the cave. Actions are nop, move forward, jump, and move the mouse left, right, up, and down. Episodes terminate if the agent dies (falling from a great height) or if 3K timesteps are reached.
Spiders attack. The agent placed in large cage together with hostile spiders (see Figure 14e), it is equipped with steel sword, and the objective survive. In the beginning, there is single spider in the cage, but every time all spiders defeated, new round starts with one more spider than in the previous one (until spiders). The reward of defeating siper is +1. Actions are: nop, move forward, move left, move right, jump, attack, and move mouse left, right, up, and down. Finally, episodes terminate if the agent dies or the 4K timestep limit is reached.
Complementing the examples from Section 3. Figure 15 provides the episodic return curves of PPO in all of the presented tasks. Results aggregate runs per task, where PPO was trained for 1M timesteps each. These experiments are mere examples complement Section 3. and thus, hyperparameter tuning was performed to improve the obtained results. Moreover, the performance in some of the tasks might be substantially improved if more training timesteps considered.
Regarding the PPO algorithm, we employed the high-quality implementations from CleanRL. (Huang et al., 2022b). Specifi- cally, the PPO implementation for Atari environments was adapted to Craftium environments, as both observation spaces consist of RGB images and action spaces are discrete (in the case of the environments presented Section 3.5.1). Moreover, this implementation already considers many details shown to benefit PPO (Huang et al., 2022a). The hyperparameters and CNN network architecture were set according their default values in the original PPO implementation from CleanRI.""
19

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
Chop tree
Small room
Room
Speico
Spiders attack
Epitodic ratur
150
100
-300
183
-353
-1083
-250
-123
60
63
0.4
64
04
66
64
15
06
32
10
22
10
Steps
Figure 15. Episodic return curves obtained by PPO in all of the tasks from Section 3.5. Lines aggregate the average values of 5 different seeds per task, while the contour denotes the standard error of the results.
Figure 16. Screenshot the illustrative multi agent environment from Section 3.5.2.
G.2. Multi-Agent Combat
This section describes the multi-agent environment example from Section 3.5.2 in detail. As can be seen in Figure 16, the scenario consists of a completely flat world, where two agents are placed in a closed jail. Both agents have no items or tools available, and cannot escape the jail. Similarly to the classic single-agent RI task (see Section 3.5.1 and Appendix G.I), observations are 64 x 64 RGB images, and the action space consists of a simplified discrete space using the DiscreteAct onWrapper from Appendix B.2. Specifically, the discrete action space consists the following actions: nop, forward, left, right, jump, attack, and move the mouse right left. An agent gets positive reward (+1) when punching other agents and (-0.1) on damage (i.e., losing one health point). Finally, episodes terminate if the number of health points (initialized to 20) of any of the agents is zero, or the maximum number of timesteps (2K by default) is reached.
Regarding the self-play method employed in Section 3.5.2, we employ the same CNN architecture and PPO algorithm implementation as in the single-agent environment examples from Appendix G.1 (refer to the last part of this appendix for details). In this case, as we employ self-play (Silver et al., 2017). both agents share the same internal NN- based policy, which updated every 128 steps. Finally, the agents were trained for 1M timesteps using grayscale versions of the observations and frame stacking of 4 frames, resulting a 64 64 pixel observation space.
G.3. Open World
In Section 3.5.3 we introduce an open-world environment. In this environment, the agent has to survive and gather resources in an open world based on the open-source VoxeLibre (Fleckenstein et al., 2025) game for Luanti. The environment is designed to have three different tracks: tools, hunt, and defend.
The first, the Tools track, consists of 4 different milestones: collect two wood blocks, three stone blocks, three iron blocks, and finally, diamond block. When the agent unlocks one of the stages (i.c., tasks), it receives a reward and new set of tools to employ to solve the next task. The reward for completing each of the stages is 128, 256, 1024, and 2048, respectively. Morcover, when the agent unlocks a new stage, it receives a sword and pickaxe made of the material of the completed stage. For example, if the agent unlocks the wood stage (collect two wood blocks), a wood sword and pickaxe are automatically added its inventory. To simplify solving the first stage of this track, the initial inventory of the agent composed of
21 Source code the original PPO implementation: https: / wiigthhh......... wwm www.///www///////////oo/oooooo// clannrl eanr1 (commit 8cbca61).
20

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
stone axe and 256 torches. The stone axe allows the agent to more easily chop trees collect wood, while it also serves to defend from enemies (i.e., monsters) and hunt animals.
Conversely, the Hunt and Defend tracks are non-sequential. The agent is expected to develop skills to handle increasingly complex scenarios rather than progressing linearly (although this could also be the case). In these tracks, a reward is provided to the agent every time it punches an enemy or an animal. In the case of enemies, the reward value is equal to the damage caused by the tool, while in the case of the animals, this value is reduced to half. The motivation behind this particular reward function is the following. If the agent defeats an enemy or hunts an animal, the episodic return obtained by the agent linear the life of the enemy or animal. Moreover, the agent is also encouraged use the correct tool for these tasks. For example, using sword to fight monster will provide more reward than using torch pickaxe for the same task.
In Luanti, the time of day of the game is linked to the real clock time, where the day/night cycle lasts for 20 minutes by default. 22 In consequence, in this environment, the time of day is set according to the global timestep to maintain consistency and avoid relying on real clock time while training agents. If the latter is not considered, the time of day experienced by the agents could vary depending on the time required by the agent to select an action, which greatly varies depending on its implementation and architecture.
The following lines provide details on the methods used in the experiment from Section 3.5.3. Note that in both cases, the action space of the agents was composed of 18 discrete actions, defined using Di discreteAct onWrapper from Appendix B.2. The actions are: nop, move forward, backward, left, and right, jump, sneak, dig, place, slot 1, slot 2, slot 3, slot 4, slot 5, move the mouse right, left, up, and down. Slot [] corresponds to the actions of selecting the tool or object in that position of the inventory (i.e., often referred to as the hotbar).
PPO+LSTM. This method is based on the popular PPO algorithm while employing a convolutional neural network to encode observations and an LSTM module providing memory capabilities to the agent. As the experiments in Appendix G.I, this agent is based on CleanRL's PPO implementations, in this case in PPO+LSTM for Atari games. 23 Similarly, hyperpa- rameters were kept fixed (not optimized), as the purpose of this experiment is to serve as an example. Finally, the observation space for this agent was set to 84x84 of greyscale images using 4 observations for frame stacking.
LLa Va-Agent. This agent is based the open-source large multimodal model (LMM) LLaVaa by (Liu et al., 2024a), specifically version 1.6 (Liu et al., 2024b). This agent is not intended new proposal for LMM for embodied AI, but just as an example of how LMMs can be employed within Craftium environments to solve general tasks by leveraging their world knowledge. For this purpose, LL has been directly employed with no fine-tuning for the open-world environment. Specifically, at each timestep. L.La Va is provided with the current observation (512 x512 pixel RGB image) and a short prompt describing the current task. The prompt also includes a list of all available actions, where LLaVa asked to choose one. Actions are taken by parsing the response from the model, where random action chosen if a parsing error occurs, although we observed that this barely happens. The employed prompt has been selected from a set of prompts of different nature listed in Table 6 based the results from Figure 17, which led to the use of the prompt (ID 1 in Table 6).
Note the <object i ve> placeholder, this is replaced with the text corresponding to the current objective: "is to chop a tree", "is to collect stone", "is to collect iron", or "is to find diamond blocks". This text is automatically placed every time the agent unlocks a stage of the Tools branch of the skills tree.
Details of Figure 9. The figure aggregates results from 10 different random seeds for each method, PPO+LSTM, and ALgeet In the case of VLaVen,gent, each run was constrained by a l-hour limit (<s 7000 prompting iterations per run) and limited to IM steps in the case of PPO+LSTM. Consequently, the X-axis has been set to the training time percentage to accommodate both cases and for the sake of visualization. Finally, the Y-axis shows the best and average cumulative reward obtained for each method. The latter is made to properly visualize when a method unlocks one of the milestones from the skills tree.
G.4. Procedural Environment Generation
The procedural environment generation example employs random dungeon generator implemented for this work. Although the generator can randomly create a vast number of different environments, their reward function is the same. In these
22 Additional information at https: //vp::///K/ ki. wtw.aa.ttngoiaiiia..aa orgg/Time_oo_dayy The original implementation can he found at: https: //tp://gtttt/// wtt.ub..ccm/vvw..e wwtp///w/////////cc/cctcoccctttt el eeannn/////// (commit 8chca61).
21

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
250
Final Episodic Return
200
100
50
1
2
3
4
Prompt ID
Figure 17. Performance comparison of Aggentt employing different prompt cadidates from Table The plot aggregates the results from five I-hour runs for each prompt.
ID
Prompt
You are a reinforcement learning agent the Minecraft game. You will be presented with the current observation, and you have to select the next action with the ultimate objective of fulfilling your goal. this case, the goal <objectivee secondary objective and survival. Available actions <acti ons> From now on, your responses must only contain the name of the action you will take, nothing else.
You should fight monsters and hunt animals just
You are a reinforcement learning agent in the Minecraft game. Your primary objective is: <object ive>. You must decide the best action based on the current observation. Fighting monsters and hunting animals are secondary tasks and should only be performed when necessary for survival or when they directly contribute to your goal. The available actions are: <actions>. Your response must be only the name of the action you will take, with no extra text.
2
You are an Al reinforcement learning agent in the Minecraft game. Your goal is: <objective>> Rach step, you receive observation and must select action from the following list: <list- aaatiaciiinsss the most effective action based on the current observation. You must respond with only the name of the action, nothing else.
3
Your task is to prioritize the main objective while ensuring survival. Choose
You autonomous reinforcement learning agent in the Minecraft game. Your mission is complete the following objective: cobjective. Each step, follow structured decision decision-making process: (1) Analyze the current observation.
4
(2) Determine whether to focus on the main objective or take necessary survival actions. (3) Choose the best action from: <act ionss, Your response must he strictly one action name, with no explanations.
Table Prompt candidates for the I.I L. Va-based agent in Section 3.5.3: (1) direct and imperative tone, (2) emphasis the primary task, (3) listing possible actions, and (4) structured decision decision-making making.
22

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
environments, the agent is randomly placed (equipped with a sword) in room and has navigate labyrinthic dungeon full of hostile enemies (monsters) reach the diamond. This process is divided into two steps: I randomly generate the dungeon's s map, represented in ASCII (defined in Appendix G.4.1), and @ build the 3D environment from the map.
① This first step is accomplished by the RandomMapGer: Python class, which implements the dungeon generation algorithm. Given some input parameters, BandomMapGen returns an ASCII representation of the generated map. Internally, RandomMapGen first creates the rooms, places the enemies, and locates the objective and the agent's initial position (the agent and the objective are never located in the same room). Then, an iterative algorithm based on repelling forces used to place the rooms so that none intersect. Secondly, computes the minimum number of corridors needed to create map where all rooms reachable. Finally, it rasterizes the map into its ASCII representation using Bresenham's 's line algorithm. 24
The complete list of parameters that RandomMapGen accepts is the following:
Number of rooms of the dungeon.
Minimum and maximum sizes of the rooms. The final size is randomly selected from this range.
A dispersion parameter in the [0, 1] range that controls the distance between the rooms.
Minimum and maximum number of monsters per room. If the minimum is set equal to the maximum, the number of monsters per room is fixed.
The probability of each monster type being located in one room. RandomMapGen considers up to 4 types of different monsters. Monster types are denoted as: a, b, c, or d. The specific monster that will be considered for each type is defined by the user in step 2.
A boolean flag indicating whether monsters can appear in the room selected for the agent's 's initial position.
A boolean flag indicating whether to add ceiling to the map. This option is used when using monsters that can climb over or fly out of the map.
Once the ASCII map is created, a mod is used to generate the final 3D dungeon inside Luanti. This mod iterates over the characters that compose the map and places the blocks and enemies (referred to as mobs in Luanti and gaming terminology, not to be confused with mods) accordingly. The configuration parameters of the mod are the following:
The ASCII map generated in step ! (or via another process).
Names of the monsters for types a, b, d. Available monsters are described in the documentation of the mobs monsters project.25
The material used for the construction of the dungeons, 26
The name of the object use as the objective (a diamond by default).
The reward of reaching the objective (100 by default).
The reward of defeating single monster by default).
G. THE ASCII MAP FORMAT
The ASCII map format has been intentionally designed be human-readable and to facilitate the implementation of custom procedures create them (or even be specified by hand). The format consists of 9 possible characters, listed and described in Table 7. As can be seen in Figure 18a, maps are divided into layers, divided by the (dash) character. The first layer is commonly employed to define the floor of the dungeons, while the second defines the walls and the positions of all characters and the objective; the rest of the layers are used for determining the height of the walls.
http://wwkkk//eeeeaaa222ess 1ine wtwpaaaeeee__ieeeaaaaeaaaaaaaaa. algerithn. 25 Accessibe at: http:///oooaabbeeerreee//ee/aaaa
24 See https::///eee
http///ooooo.ooose.sseemm monster.
List of some availahle materials: https: wtt.tiiiiiiiiiiiii..aaaaa wrgg//aee////oooooooooo /intames/Miietett...mm
27 List of some available items: https: 1uanti www.o///g//////eeeeeeeeeeeeeeeee Gametttemmmmm Ttems.
23

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
Table 7. List of characters that comprise the ASCII map format and their meaning.
Character
Meaning
(whitespace)
Air block.
Construction block. Used for the floor and walls.
Glass block. can be used for the ceiling.
The initial position of the agent.
Position of the objective. Location of a monster of type a, b, C, or d New layer.
a, b, C, d
L
(a) ASCII map representation.
(b) Resulting 3D dungcon environment.
Figure Example ASCII map format of dungeon environment and resulting 3D scenario in Craftium environment. Note 3D characterizations of the spider (denoted with in the ASCII map) and the diamond (O in the ASCII map). The ceiling has been removed for the sake of visualization.
G.S. Environment Sequence for Continual RL
In Section 3.5.4, the procedural environment generator is applied to CRL defining a sequence of related and increasingly difficult scenarios. Similarly to the examples from Section 3, the FS (baseline) and FT-L2 methods are based the PPO implementations from CleanRL. The difference between the FS and FT-L2 is that the latter fine-tunes the model learned in the previous task and uses L2 regularization during training, while the FS always learns a model from scratch. FT-L2 was selected for this example as has shown significant forward knowledge transfer capabilities in other works (Gaya et al., 2023; Wolezyk et al., 2024; Malagon et al., 2024).
Regarding the observation and action spaces, they have been kept constant across the sequence. The observation space is set to 64 pixel greyscale images, with 4 frames for frame stacking, and the same quantity for frame skipping (Huang et al., 2022a). The action space consists of set of 10 discrete actions: nop, move forward, left, right, jump, attack, move the mouse right, left, and down. Finally, episodes terminate if the health of the agent is exhausted or 5K timesteps are reached.
For the sake of visualization, figure 20 provides simplified 2D visualization the environments. Observing the figure, we see that the first two environments employ the same map (with the initial position the agent and the objective switched), This is intended, as the training time in each environment is low (1M timesteps), thus, the first two environments offer
24

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
0.6
0.8
0.45
0.5
Episodic Retur
0.6
0.5
0.35
2
0.4
0.2
0.30
0.4
0.3
0.2
0.1
0.2
0.2
0.2
0.0
0.20
Step 2M
3M
4M
5M
6M
8M
9M
10M
15
FT-L2
Figure Episodic return curves of the baseline (FS) and FT-L2 over tasks from the CRL sequence Section 3.5.4, Note that the first environment is omitted as the FT-L2 is not applied in this case (there is previous model to fine-tune). See Appendix G.5 for details Figure 20 for simplified visualizations of all the environments the sequence.
CRL methods way to learn to reach their objective before more difficult tasks arrive. From the 2nd task onwards, the environments contain two or more monsters, whereas tasks 3 and have single monster between the agent and the objective (the diamond), and from the 5th task onwards have two or more.
As be seen in Figure 19, FT-L2 substantially improves the results of the baseline in the 2nd and 4th, showing considerable forward knowledge transfer between some of the generated environments. Although the final episodic return is lower, environments the 5th and 6th also show some forward knowledge transfer in the first parts of the training.
3
SB
5
10
la CN
lo D
Figure 20. Overview of the maps generated for the CRL environment sequence Section 3.5.4. Note that these 2D representations of the environments (for proper visualization) and that the actual environments are 3D, as can be seen in Figure 1a and Figure 18b. The robot indicates the initial position the agent, while the yellow characters indicate sand monsters, and black characters denote spiders. Maps have been cnumerated with their corresponding position in the CRL sequence.
25

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
H. Environment Creation Flexibility of Craftium and Hineeraft-Based Frameworks
Note that the visual similarity between Craftium (and Luanti) and the popular Minecraft game arises from their shared use of voxel-based graphics and sandbox-style" environments. However, it is important to clarify that Luanti, as game engine, not an implementation or clone of Minecraft, and serves fundamentally different goals compared Minecraft, which standalone game, see Luanti Wiki (2025).
Besides significant performance improvements, multi-agent support, and fully open-source nature compared to Minecraft- based alternatives, Craftium also provides an extremely flexible interface for creating new environments via the Luanti Modding API. The flexibility and versatility this API are demonstrated by the rich and complex environments that can be created with it, see Figure 9, thanks to the wide range of mods created by the community (see Ward (2025a) for examples). This section focuses on showcasing some code examples that directly compare the flexibility of Craftium's API with the MineDojo API to create new environments. Note that we only compare Craftium MineDojo is, currently, the only Minecraft- based framework that allows the creation of custom environments.
One major limitation of MineDojo's API is that although it allows for spawning different Minecraft entities (mobs and items) in a given location, the behavior, aspect, and other properties of the entities are those of Minecraft (the default ones) and cannot be changed. Figure 21 shows how MineDojo allows spawning entities. On the other hand, Craftium leverages the Luanti API, which allows access to the internal state of the game engine, allowing to change any aspect of real time. This illustrated with an example code in Figure 23 and Figure 24 that show how many properties and behaviors of entities can be modified in Craftium.
Another crucial difference between Craftium's and MijeDojo's APIs is the map generation capabilities. MineDojo limits map generation to some predefined scenarios (only 5) and biomes. Figure 22 shows the map customization capabilities of MoneDojo. On the other hand, Craftium's API allows the user to define any type of custom biome and combine them in any way. 29 In Figure 25 we showcase a simple example of defining a custom desert biome in Craftium's API. Note that Craftium users can employ any of the vast number of biomes already implemented by the community (some of them illustrated in Figure 9). 30
env. sspan_mmoallleee ("spider", [5, 0, 5])
Figure 21. MineDojo. Although MineDojo allows for spawning entitics in some positions, lacks the capability to modify the behavior of entities any way.
1 env minedo jo make ("open-ended"
Figure 22. MineDojo. MineDojo only allows defining worlds from a set of predefined biomes and scenarios.
local mob_def core. rwgi mob_def.ee on__uunh function (self, hitter) hitter:set hp (hitter:get. hp () 5)
immobs_monsterrzzmmb
4 end
Figure 23. Craftium. Example code demonstrating how the behavior of entities can modified in Craftium. In this case, the definition of zombies is changed to increase the health of the agent by 5 when successfully attacking zombie.
Sandbox games allow players extensive creative freedom explore, build, and manipulate the game environment with few constraints or predetermined goals. More information and tutorials at Ward (2025b).
30 Examples https: /onttett. luanti
26

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
mobs:register_mmb ("erattiummm.sssiee""" {
2
docile_by_day false,
3
group_attack = true,
type = "monster", passive false,
attack_type "dogfight", reach 2, damage 3, hp min 25, hp 25,
7
armor 200,
12
walk_vod malk_velocity 3,
run_velocity = 6,
jump = false, on_die function (self, pos)
Set reward to 1.0 for a single timestep, then reset to 0.0
17
set -eward__ocee (1.0, 0.0)
Spawn more spiders sum_spiders num spiders 1 for i=m_nun_spiders do
spawn_monster (f 3.7 i, y 4.5, 2 0.0 1
end end }
26 local monster mobs: add__mb (pos, name = salfftiummm__sp__ee ignore_count true,
}
Figure 24. Craftium, Example of completely custom spider type. Note that only show few options of those available: group attack capabilities, health, reach, attack type, armor, velocity, etc. Moreaver, a custom behavior is defined to set the reward and spawn more spiders when the spider dies.
Register custom biome (e.g., desert) core. regirsseeeeteee
name "custoo__eseert"
4 node_top "default:sand",
5 depth_top 1, node filler = "default:stone",
})
9 Generate a random landscape with different biomes core. woggltter_on_generateeeeee (function (minp, maxp, blockseed) if random 0.5 then
core biome (minp, maxp, "eustom_desert" end end)
Figure 25. Craftium. Example showing how custom biomes can be created and used in Craftium.
27

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
I. Creating Custom Environments
In the following, we outline the steps to create a custom open-world environment from scratch, where the task is to find the deepest possible cave within limited number of steps (i.e., episode). Note that the following instructions assume that Craftium is already installed.
! Creating the world. The first step is to create the environment's 's world. Run the Luanti binary (which should already be built and available in the Craftium installation directory) and follow the instructions in Figure 26. Note that the first five steps are required only once; for future environments, follow only steps (5) and c from Figure 26. Close Luanti once the world is created (happens almost instantly).
Luaati
Minetest Game
MOME
SRIME
aiime lingg
MONE
GAMEE
Figure 26. Creating the world using Luanti's graphical menu (left to right, top to bottom). @ Click the Content tab in the main menu. @ Click Browxe online content. ④ Select Minetest Game. ④ Click the green Install button and wait a few seconds. Return the main menu and click the Minetest logo at the bottom. c Click New to create a world. Enter a name (world in this tutorial) and click Create.
② Creating the mod. Before coding the Craftium environment, set up a directory to store all environment-related data. This directory should contain the game, world, and mod, which we will name cra iim_. create the environment's directory, run the following CL.I commands from Craftium's 's main directory:
mkdir -z www_een//ddssso http//eevveeaaareeeeeeeeeeeeeeee
2 cp -r worlds games my env
3 echo beaad_mo__en_cree_ = true" >> ww..nn//oorld///oe1////o111
Figure 27. CLI commands to create and set up the environment's directory.
After running the commands above, we can create the mod. conf and init. lua files, as described in Section 3.2. In this example, the task to find the deepest cave possible within an episode. To set up the mod, first, create file named mod. fonf inside the my_env htoe//mmd////ee looos/eev_cee eeve_eerccaaaaaeeeeeeeee um directory with the contents from Figure 28. Then, create
28

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
iinit. lua in the directory using the contents from Figure 29, Although there are many possibilities for the init. lua file, in this case, the agent is rewarded based on its negative Y-axis position (depth).
1 name env_craftium
2 description = Craftium ennment depends default
Figure 28. Configuration file for the Craftium environment, specifying the mod name and its dependencies (which, in this case, only includes the default mod).
rori.regitter _al obal step (function ()
-- Get the player's 's object local player
oeeeeseet_cmmeecceepppaaeees [1]
5
Check if the player connected
6
if player = nil then
return
end
9
Get the player's position
10
local = playeriget_po () [2]
Set the reward value for the current step
set (-w_
end)
15
This function is run every time the player dies core. registerronnneeeeeeeeer oegetter_n_dieplayer (function (obj, rn)
16
Set the termination flag to true set set_termination)
19
end)
Figure 29. Example Lua mod. The code defines callback functions: the first (line runs every timestep, setting the reward the player's negative Y-axis position (i.e., depth). The second (line 16) triggers when the player dies (e.g., after a f'atal fall) and sets the environment's termination flag to true (see Section 3.3).
Running the environment With the world and mod set up, the final step is to run the environment. Figure 31 shows Python script that, when executed in the same directory the enviromment (Craftium's 's main directory in this tutorial), loads the environment and performs random actions while plotting the current observation at each timestep (see Figure 30). This example simply showcases the custom environment using a random agent. Note that more complex agents and learning algorithms can be easily integrated by replacing with a call the desired method.
Figure 30. Example output from the script in Figure 31, showing observation from the environment created in this section.
29

---

Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments
import matplozlib.pypi pppplot plt import craftium
from craftium import CraftiunEav
erv (
env_dir="my_snn
7
obs_width=512,
8
obe_hcight=512,
observation, info reset
ep_ret = 0 Episodic return
for step in range (100): Display the current observation pit.cla())
17
pit. imshow (observation)
pit. pause (0.03) 01)
Sample random action
action env. seatiio__spaeeeamae...
22
obscrvation, reward, terminated, truncated, info = env step (cn)
ep_ret reward
print (step, reward, terminated, truncated, ep_ret)
if terminated truncated:
observation, info resereset ret
env. close ()
Figure 31. Example Python script that loads and runs the custom environment with randomly acting agent.
30


Reviews:
[{"review_id": "8DA3G5ZGsu", "reviewer": "Reviewer_eFW5", "summary": "The paper introduces Craftium, a new platform for creating 3D environments aimed at reinforcement learning and multi-agent research.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's thoughtful feedback and the time spent evaluating our paper. Below, we address the concerns raised.\n\n**[Incremental contribution]**\n\nWhile Craftium's individual components have existed separately, no prior framework combines ease of use, high efficiency, multi-agent support, extensive flexibility, and open worlds. This unique integration is more than an incremental improvement: it makes previously impractical research directions feasible. For instance, works in unsupervised environment design, open-ended learning, continual reinforcement learning, and meta-learning can now leverage large-scale experiments in rich 3D open-world and (single- or) multi-agent environments without prohibitive computational costs. We believe these capabilities make Craftium a valuable tool for future research and of interest to the ML community.\n\n**[Performance]** \n\nWe have significantly expanded the performance benchmarking in line with the reviewer's suggestions (we have followed the experimental setup from Section 3.4).\n\n  - **Observation resolution (see [plot](https://bit.ly/4iIisJA)):** Craftium's step/s loss for 128×128 observations is minimal (less than 5%). For larger observations (512×512), the loss is more significant (around 33%). However, in such cases, the performance bottleneck is likely in the model processing the images rather than in Craftium itself.\n\n  - **Number of agents (see [plot](https://bit.ly/43mW6bT)):** As shown, Craftium's multi-agent implementation scales efficiently, with a 5% performance loss in the worst case. This loss is minimal, and the fluctuations in the plot are due to the randomness of performance measurements due to the OS, other processes, etc.\n\n  - **World's complexity (see [plot](https://bit.ly/4lmKCvJ)):** The plot compares Craftium's performance in two worlds of varying complexity: `flat_world`, a minimal environment containing only the task scenario, and `minetest_world`, the base open-world environment. Although complexity impacts performance, it remains very high in both cases.\n   \n  - **Memory:** We have measured memory consumption across different tasks. Craftium (660MB) is notably lighter than MineDojo (1.7GB), the only framework with comparable environment richness. VizDoom (84MB) is the lightest due to its minimalist design at the cost of simplicity (e.g., no 3D).\n    \n  - **Number of environments (see [plot](https://bit.ly/43YfHPO)):** Craftium benefits significantly from parallelization, achieving impressive performance comparable to the much simpler VizDoom. In contrast, MineDojo lacks parallel environment support, making it impractical for many research scenarios, and despite extensive efforts, we could not obtain its performance metrics. \n  \nThese results will be included in the final version of the paper together with further details and extended analysis.\n\n**[Environment creation process]**\n\nWe fully agree with the reviewer and have added a new appendix to the paper detailing the complete environment creation process. As we cannot include a direct link to the text, we provide new [figures](https://bit.ly/4jcJWXL) that illustrate this simple process.\n\n**[RL outcome comparison]**\n\nWe believe that directly comparing RL outcomes from identical tasks in Craftium and MineDojo may not accurately reflect Craftium's contributions and could shift attention from the main focus of the paper, potentially leading to misleading conclusions. Craftium is a new framework designed to create diverse environments for AI research, and the experiments showcase its capabilities and compare them to other environment creation tools. A comparison between RL outcomes in Craftium and MineDojo could be misinterpreted, as differences are likely to arise from various confounding factors (e.g., different mechanics or texture packs), rather than any inherent superiority of one of the environments. \n\n**[Other comments]**\n\n- **Claims:** We will revise the claims based on the reviewer’s suggestion in the final version of the paper.\n\n- **Typo:** We sincerely appreciate the reviewer pointing out the typo. It will be corrected in the final version.\n \n- **Reference:** We are grateful to the reviewer for suggesting an additional reference, it will be included in the final version of the paper.\n\n**[Questions]**\n\n**Q1:** Due to fundamental differences between the frameworks, replicating the exact environment setup is not feasible. We have used multiple scenarios for each framework to ensure a fair comparison (see Appendix E).\n\n**Q2:** Please refer to the *\"Performance\"* section above.\n\n**Q3:** Yes, by default. However, more complex physics can be implemented using the Modding API (some examples [here](https://bit.ly/4hUIU1k)). \n\n**Q4:** Currently, Craftium supports continuous control only for the main camera. We see continuous control as a promising direction for future work and plan to implement it using Lua mods within Craftium's current framework.", "claims_and_evidence": "Claim 1: High Flexibility and Rich Features. The authors assert that Craftium allows “nearly limitless” possibilities for creating custom environments, in contrast to many existing platforms that only offer fixed games or limited parametric tweaks.\n\nClaim 2: First to Support Vast 3D Open-World + Multi-Agent. The paper highlights that Craftium is “the first framework” enabling huge 3D open-worlds with multi-agent support\n\nClaim 3: Major Efficiency Improvement (Speed). A core claim is that Craftium is significantly more computationally efficient than comparable rich 3D environments.\n\nClaim 4: Ease of Integration and Use. The authors claim Craftium is user-friendly: it supports Gymnasium/PettingZoo APIs out-of-the-box and is opensourced\n\n\nThe most problematic claims are those around “unlimited flexibility” and being uniquely first in certain capabilities – these are somewhat overstated without direct comparative experiments..", "methods_and_evaluation_criteria": "The paper is largely an engineering contribution – the “methods” consist of describing the Craftium framework’s design and implementation. The approach taken (using a modified game engine with scripting for environment definition) is sensible for the stated problem.\n\nThe performance benchmarking focuses only on throughput (steps/sec). While this is crucial, the evaluation could be more comprehensive. For example, the paper does not report memory usage, scalability with number of agents, or how performance is affected by increasing observation resolution or world size.\n\nThe choice of comparison frameworks in the benchmark is limited. The authors compare to MineDojo and VizDoom. The RL task demonstrations are not used to rigorously evaluate learning or agent performance.\n\nNo user study or complexity analysis is provided for the environment authoring process. Since a major point is that Craftium makes it easy to create custom environments, it would strengthen the paper to have a more formal evaluation of that.", "theoretical_claims": "This paper does not make any significant theoretical claims.", "experimental_designs_or_analyses": "The illustrative RL experiments are less about rigorous analysis and more about showcasing capabilities. The analysis of the performance results is quite minimal – essentially, “Craftium is faster, presumably because of C++ vs Java and fewer layers”​. This explanation is valid and actually quite obvious. The authors didn’t attempt any deeper profiling.\n\nAs noted, the experiments miss direct comparisons for RL outcomes across frameworks. It would strengthen the paper to see, for example, the same task implemented in both MineDojo and Craftium and an RL agent’s learning curve in each.", "relation_to_broader_scientific_literature": "The paper situates Craftium in the context of existing environment platforms and benchmarks. The authors demonstrate a good awareness of prior work, citing a broad range of relevant literature.", "essential_references_not_discussed": "Unity: A General Platform for Intelligent Agents\nScalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot (ICML 2021)", "other_strengths_and_weaknesses": "The paper’s contribution is primarily engineering-oriented, which can make it seem less novel in a research sense. Craftium is not introducing a new algorithm or new scientific finding; it is packaging existing ideas (voxel world, Lua scripting, Gym interface) into a platform. In terms of novelty, many components of Craftium existed separately: Minecraft proved the value of voxel worlds for open-ended tasks, Lua modding is a known technique in games, and others have integrated environments with Gym. The authors’ work in combining these elements in a coherent, open-source system is commendable, but it feels more like an incremental", "other_comments_or_suggestions": "There is a minor typo in the abstract: “Micraft-based frameworks”​ should be “Minecraft-based frameworks.” This appears to be a small spelling mistake. Make sure to fix “Micraft” to “Minecraft” in the final version.", "questions_for_authors": "1. What tasks or environments were used for the performance comparison with MineDojo and VizDoom? Can you clarify if all frameworks were tested under similar conditions (e.g., same observation resolution, minimal entities)?\n\n2. How does Craftium perform with multiple agents simultaneously? Have you benchmarked scenarios with, say, 5 or 10 agents in one environment all acting at once?\n\n3. How is the physics simulation in Craftium? Only axis-aligned gravity and simple collisions like Minecraft?\n\n4. Can Craftium support continuous control of an agent’s joints or is it strictly the discrete game controls?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ksWv1IMLSh", "reviewer": "Reviewer_ts6y", "summary": "The paper presents Craftium, a highly customizable and efficient multi-agent 3D environment tailored to RL research. Craftium builds on the open-source game engine environment Luanti. It can be used to easily build rich 3D environments with numerous possibilities. Benchmarks against other environments show the superiority of Craftium in terms of average steps per second, which allows significant scaling of training and testing RL agents. The paper includes several examples of environments that were built with Craftium. Code examples are included to show its simplicity, and results of RL agents trained on the environments to demonstrate its usability.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are pleased that the reviewer recognizes Craftium as an important contribution and finds the paper well-written with well-supported claims.\n\nWe also appreciate the reviewer for suggesting a valuable reference that strengthens our coverage of realistic simulators. We will include ThreeDWorld in the final version of the paper.\n\nRegarding the reviewer's question, the 2.5D nature of VizDoom (due that is based on the Doom game) limits its environments in multiple ways compared to fully 3D frameworks such as Craftium. Some of the most critical limitations are the following:\n\n- The agent's viewpoint is restricted to a horizontal plane, preventing it from truly looking up or down.\n\n- Level height (floor and ceiling) is stored in a 2D matrix, making it impossible to create overlapping structures like bridges, floors, or buildings.\n\n- Enemies and objects are 2D sprites that change in size and angle based on the agent's position.\n\nThese limitations make VizDoom environments significantly different from more realistic and diverse 3D scenarios as those in Craftium, failing to cover fundamental challenges for autonomous agents that are of interest for current research e.g., spatial 3D reasoning [1] and complex agent-environment interactions [2].\n\nFurthermore, 2.5D environments greatly limit the diversity of tasks and scenarios, which is particularly relevant for areas like continual reinforcement learning [3], unsupervised environment design [4], and meta-learning [5], all of which are of growing interest to the research community [6].\n\nCraftium is especially relevant to these fields, as it enables diverse, 3D, vast open-world environments that are computationally efficient and support multi-agent scenarios, opening the door to exciting future research directions.\n\nFinally, we agree with the reviewer on the importance of this topic and will include a more in-depth discussion in the final version of the paper.\n\n[1] Chen et al. \"SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities.\" CVPR 2024.\n\n[2] Wang et al. \"Voyager: An Open-Ended Embodied Agent with Large Language Models.\" TMLR 2024. \n\n[3] Abel et al. \"A Definition of Continual Reinforcement Learning.\" NeurIPS 2023.\n\n[4] Beukman et al. \"Refining Minimax Regret for Unsupervised Environment Design.\" ICML 2024. \n\n[5] Bauer et al. \"Human-Timescale Adaptation in an Open-Ended Task Space.\" ICML 2023.\n\n[6] Hughes et al. \"Open-endedness is Essential for Artificial Superhuman Intelligence.\" ICML 2024.", "claims_and_evidence": "The paper is very well written. The claims are supported with examples of environments and code snippets.", "methods_and_evaluation_criteria": "Methods and evaluation makes sense.", "theoretical_claims": "There are no theoretical claims.", "experimental_designs_or_analyses": "I don't see any issue with the experimental design.", "relation_to_broader_scientific_literature": "The space of video games / game engine environments for RL research is pretty crowded. Still, I think that this work makes an important contribution by providing a general-purpose easy to use and highly efficient environment for RL researchers.", "essential_references_not_discussed": "The related work section covers most relevant work. In the high-end photo/physics-realistic space I would mention ThreeDWorld (https://www.threedworld.org) in addition to Habitat and AI2-Thor, as it supports both outdoor and indoor environments.", "other_strengths_and_weaknesses": "Nothing to add.", "other_comments_or_suggestions": "Nothing to add.", "questions_for_authors": "A general question which could be worthwhile touching on in the paper is how much gain do we get for RL research using 3D environments such as Craftium over 2.5D such as VizDoom?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7AS2bPJuJu", "reviewer": "Reviewer_xfHx", "summary": "The paper introduces Craftium, a flexible, efficient, and user-friendly framework for creating rich 3D environments for single- and multi-agent research in autonomous systems, such as reinforcement learning (RL), embodied AI, and multi-agent reinforcement learning (MARL). It addresses the trade-off between computational efficiency and environment richness, which is a common limitation of current platforms.\nCraftium is built upon the open-source Luanti game engine, allowing users to easily design customizable 3D voxel environments through an accessible Lua Modding API, rather than relying on complex Domain-Specific Languages (DSLs) or restrictive game-based platforms. It supports essential research features like procedural environment generation, customizable reward structures, adjustable observation/action spaces, and single- and multi-agent setups. Craftium is compatible with widely-used standards in the RL community, including Gymnasium and PettingZoo interfaces.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are pleased that the reviewer acknowledges Craftium's role in bridging the gap between computationally efficient yet simple environments and those that are rich and flexible but computationally costly, as well as its logical, pragmatic, and well-justified design. \n\nWe agree that an ablation study or hyperparameter sensitivity analysis would further strengthen our experimental results and demonstrate Craftium's suitability for rigorous algorithm development. To this end, we identified the LLaVa-Agent prompt as the most arbitrary hyperparameter in our experiments (detailed in Appendix F.3), as the remaining hyperparameters follow standard values known to work correctly (please refer to this [ICLR 2022 blog](https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/)).  \n\nAccordingly, we have created four distinct prompts: **(1)** direct and imperative tone, **(2)** emphasis on the primary task, **(3)** listing possible actions, and **(4)** structured decision-making. Due to response length limitations, we provide a [table](https://drive.google.com/file/d/1LS-by0HVsH17aBO7cMDOzQik3AfSXcrh/view?usp=sharing) detailing the prompts used.  \n\nTo evaluate their impact on agent performance, we conducted five 1-hour runs for each prompt. The results, shown in the [plot](https://drive.google.com/file/d/1KMep8_1is82-Jfd7ebkLzf0eo0A1z-Ov/view?usp=sharing), present the average final episodic return for each option. As observed, the first prompt yields the best performance, followed by the third, suggesting that prompts with a more direct and imperative tone are most effective in guiding the LLaVa-based agent toward the environment's goal.\n\nWe appreciate the reviewer's suggestion, as we believe that this experiment further highlights Craftium's suitability for controlled and insightful analyses. We are open to any additional questions or requests for further clarification.", "claims_and_evidence": "The authors provide substantial evidence through examples, benchmarks, and detailed descriptions. \n- Craftium significantly reduces computational cost compared to existing similarly rich alternatives (e.g., MineDojo and VizDoom). A clear benchmark comparison is provided (Fig. 7), showing quantitative results across multiple environment setups. The authors measure and report steps per second, clearly demonstrating the computational superiority of Craftium over MineDojo and competitive performance with VizDoom.\n- Craftium enables easy creation of diverse, customizable environments. Multiple illustrative examples provided in the paper demonstrate diverse scenarios created with minimal Lua scripting. Code snippets and mod examples explicitly illustrate ease of use (e.g., Figs. 4, 5, and examples in the appendix).\n- Craftium’s architecture offers flexibility via the Luanti Modding API and is enhanced by an active community. The authors present clear examples demonstrating mod customization (e.g., Figs. 21, 22 in the appendix), and explicitly mention community-contributed assets and games (e.g., VoxeLibre).", "methods_and_evaluation_criteria": "The paper introduces Craftium as a flexible, efficient framework designed specifically to address the limitations found in existing environments for reinforcement learning (RL), multi-agent RL (MARL), and embodied AI research.\n\nThe choice of using Luanti, an open-source voxel-based game engine with a robust Lua Modding API, aligns effectively with the stated need for customizable and computationally efficient environments. This decision directly addresses the identified limitations of closed-source and computationally expensive platforms like Minecraft or overly simplistic 2D grids.\n\nLua Modding API and Python Interfaces (Gymnasium/PettingZoo):\nThe combination of the Lua-based modding approach with the widely adopted Gymnasium and PettingZoo interfaces is logical and pragmatic. It ensures broad usability and integration with existing reinforcement learning tools and workflows.\n\nSingle- and Multi-Agent Setup:\nThe decision to explicitly support both single-agent and multi-agent settings is clearly justified by the growing research interest in MARL and complex agent interactions, which are currently underserved by existing frameworks.\n\nProcedural Generation and Customizability:\nIncluding procedural generation capabilities and extensive modding options addresses the critical needs of research in fields such as continual reinforcement learning (CRL), unsupervised environment design (UED), and embodied agents. The proposed API is coherent, easy to understand, and relevant for the intended use-cases.", "theoretical_claims": "The paper does not contain any theoretical claims or formal mathematical proofs.\nIts contributions and claims are practical, conceptual, and empirical in nature, such as introducing a computationally efficient framework, providing a versatile API, and demonstrating applicability through a variety of illustrative experiments and benchmarks.", "experimental_designs_or_analyses": "I carefully checked the experimental designs and analyses provided in the paper. Overall, the experiments and analyses are sound, valid, and well-designed to demonstrate the capabilities and utility of the proposed Craftium framework. The paper explicitly states that hyperparameters were not tuned (e.g., Appendix F.1). While this is understandable given the illustrative nature, a minimal ablation or hyperparameter sensitivity analysis would further strengthen experimental conclusions, particularly in demonstrating Craftium’s suitability for rigorous algorithm development.", "relation_to_broader_scientific_literature": "Craftium bridges these two extremes by providing efficient, highly customizable environments using the open-source Luanti engine, combining the computational efficiency often found in simpler platforms (such as MiniGrid by Chevalier-Boisvert et al., 2023, or Craftax by Matthews et al., 2024) with the flexibility and richness typically exclusive to fully-featured environments (like MineDojo).", "essential_references_not_discussed": "n/a", "other_strengths_and_weaknesses": "see above", "other_comments_or_suggestions": "see above", "questions_for_authors": "see above", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kV6MY3PG8P", "reviewer": "Reviewer_2FiT", "summary": "The paper introduces Craftium, a new platform for creating rich 3D environments that balance flexibility and computational efficiency. This design allows researchers to develop complex environments with minimal code. Notably, Craftium supports both single-agent and multi-agent scenarios in the same rich 3D worlds, which addresses a gap in existing platforms that often lack multi-agent support or open-world scale. Compared to 3D game-based platforms like VizDoom or MineDojo, Craftium is fully customizable and open-source – researchers can create new worlds, objects, and game mechanics using Lua. In contrast, many established environments allow only limited parameter tweaks or rely on closed-source games​. The platform is released as open source with extensive documentation and community asset support, distinguishing it as a practical and extensible tool for researchers​.\n\n**Strengths**:\n- Novelty and significance of the Craftium platform. It addresses a clear need in the RL/AI research community for environments that are both rich and scalable. \n- Empirical performance advantage: the paper provides benchmarking evidence that Craftium can simulate on the order of thousands more steps per second than alternatives​, a crucial improvement for running long or multi-agent experiments.\n- Craftium’s design emphasizes ease of use and integration – by adopting the Gymnasium/PettingZoo interfaces and providing documentation, it can plug into existing workflows with minimal friction.\n\n**Weaknesses**:\n- Craftium’s environments are voxel-based, which means the world is composed of blocky units\n- The paper does not explicitly address how well Craftium could handle detailed physics or continuous control dynamics beyond grid-aligned interactions.\n- Craftium is highly flexible; it does not come with a standardized benchmark suite of tasks; it is a toolkit.\n   - However, these weaknesses are not fundamental flaws but rather inherent trade-offs or scope limitations.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's comments on Craftium's novelty and its role in addressing a clear need in the RL/AI research community. We also thank the reviewer for the valuable feedback and references. Below, we address the concern regarding the parallel environment benchmark and respond to the questions.\n\n**[Environment parallelization]**\n\nWe completely agree with the reviewer on the importance of evaluating Craftium's performance with parallel environments. Accordingly, we conducted a new performance benchmark following the reviewer's suggestion. As shown in the [plot](https://bit.ly/43YfHPO) (using the setup from Section 3.4), Craftium significantly benefits from environment parallelization, achieving competitive results compared to VizDoom (despite VizDoom being considerably simpler and not 3D) and reaching over 12K steps per second at the high end. Finally, MineDojo lacks support for parallel environments, preventing us from obtaining its performance metrics despite considerable effort. This limitation makes MineDojo impractical for many research scenarios (e.g., large-scale experimentation) and for algorithms that greatly benefit from parallel environments (e.g., PPO).\n\n**[References]**\n\nWe appreciate the reviewer for suggesting three relevant and up-to-date works that enrich the literature on simulation frameworks for robotics. We will incorporate and discuss these references alongside other photorealistic simulators in the final version of the paper.\n\n**[Questions]**\n\n**Q1:** Please refer to the *\"Environment parallelization\"* section above.\n\n**Q2:** Currently, we focus on other aspects of Craftium's development, but we recognize the potential benefits of ray tracing (e.g., enhanced realism). However, Craftium directly benefits from Luanti updates, which may introduce ray tracing in the future. Luanti has an active community that continuously enhances the engine. For instance, a recent [update](https://blog.luanti.org/2024/11/10/5.10.0-released/) implemented rendering improvements. In the meantime, a more realistic appearance can be achieved using high-resolution texture packs like [Realistic 1024px](https://content.luanti.org/packages/Clemstriangular/phototextures_1024px/).\n\n**Q3:** Yes, integrating Craftium with LLMs for automatic environment generation is an exciting research direction we plan to explore in the near future. Craftium's simple and expressive Lua interface reduces boilerplate code and helps improve the reliability of LLM-generated code. Moreover, Craftium and its engine are extensively documented, which could further facilitate integration by providing LLMs with structured, high-quality reference documentation. Additionally, recent approaches like OMNI-EPIC [1] could greatly benefit from Craftium's richness, flexibility, and efficiency for generating more complex and diverse scenarios.\n\n[1] Faldor et al. \"OMNI-EPIC: Open-endedness via Models of Human Notions of Interestingness with Environments Programmed in Code.\" ICLR 2025.", "claims_and_evidence": "- They measured the environment simulation speed (steps per second) for Craftium versus two popular platforms: MineDojo (a Minecraft-based environment suite) and VizDoom (based on the Doom engine). The test used three different environments per platform and averaged results over 5 runs each, on a machine with a single NVIDIA A5000 GPU and an Intel Xeon Silver CPU. The outcome (presented as Figure 7 in the paper) showed Craftium achieving on the order of 2000–2500 steps per second, which is competitive with VizDoom (noting that VizDoom isn’t fully 3D) and vastly higher than MineDojo.\n- The results, summarized in Figure 8, show different learning outcomes depending on task difficulty. For instance, in a “ChopTree” task (rewarding the agent for chopping trees), PPO was able to learn a successful policy, reaching an average of over 6 trees chopped per episode at convergence.\n- While the paper does not delve into detailed multi-agent metrics.", "methods_and_evaluation_criteria": "The methods are well-designed for demonstrating the platform’s strengths, and the evaluation criteria align with the key claims of the paper. However, the absence of standardized benchmark environments and limited large-scale multi-agent experiments could be areas for improvement in future research.", "theoretical_claims": "There are no Theoretical Claims in the paper that need to be checked.", "experimental_designs_or_analyses": "They measured the environment simulation speed (steps per second) for Craftium versus two popular platforms: MineDojo (a Minecraft-based environment suite) and VizDoom (based on the Doom engine). The test used three different environments per platform and averaged results over 5 runs each, on a machine with a single NVIDIA A5000 GPU and an Intel Xeon Silver CPU. I think there should be an experiment that shows the parallelized training environment. e.g., Several environments start at the same time.", "relation_to_broader_scientific_literature": "Craftium extends the work of these environments by combining customizability, computational efficiency, and open-ended 3D world design, making it suitable for both traditional RL training and more exploratory AI research.\nCraftium builds on MARL research by providing a fast, open-ended alternative to MineDojo and PettingZoo-compatible 3D environments, making it an attractive platform for large-scale MARL experiments.\nCraftium expands open-ended RL research by providing scalable, procedurally generated 3D worlds where lifelong learning and AI-driven exploration can be tested in ways that go beyond static datasets or limited pre-defined tasks.\nCraftium positions itself as an alternative to existing game-based RL research tools, benefiting from its lighter computational footprint and easier modding support.", "essential_references_not_discussed": "- ProcTHOR: Large-Scale Embodied AI Using Procedural Generation\n- RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation\n- ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations\n\nAlthough the environments presented by the authors differ from conventional robotic simulators, I believe it may still be necessary to mention and discuss similar RL environments in the paper.", "other_strengths_and_weaknesses": "No other strengths and weaknesses.", "other_comments_or_suggestions": "No other comments and suggestions.", "questions_for_authors": "1. In a parallel environment setting, does Craftium still maintain its high-speed performance? For example, when training RL models, can we efficiently run training across 512 parallel environments simultaneously?\n2. Is there any plan to enhance Craftium’s rendering capabilities by integrating a more realistic renderer, such as the Minecraft Raytracing Renderer(Sonic Ether’s Unbelievable Shaders)?\n3. Are the authors interested in leveraging large models for the automatic generation of simulation-based environments? Would it be possible to introduce a framework for automated environment generation in future updates?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This submission introduces Craftium, an easy-to-use and customizable platform that allows for building rich 3D environments, both in single-agent and multi-agent settings, based on the open-source game engine Luanti. The authors show the superiority of Craftium in terms of step per second against other environments, which can allow scaling the training of RL agents more efficiently. The manuscript includes various example environments built with Craftium. 

The reviewers have noted the following strengths of the paper (among others):
* Craftium provides a general-purpose, easy-to-use and highly efficient environment for RL researchers.
* Empirical performance analysis by the authors shows that Craftium can simulate on the order of thousands more steps per second than alternative platforms, which is crucial for running long experiments.
* Craftium expands open-ended RL research by providing scalable, procedurally generated 3D worlds where lifelong learning and exploration can be tested in ways that go beyond static datasets or limited pre-defined tasks. 
* Experiments and analyses are sound, valid, and well-designed to demonstrate the capabilities and utility of the framework.

Some of the weaknesses mentioned by the reviewers include:
* Continuous control of the agent’s joints is not currently implemented.
* The environments are constrained to be voxel-based.

Overall Assessment: The reviewers agreed that Craftium makes an important technical contribution and provides clear practical benefits to the RL and MARL communities.
Decision: Accept (poster)