# Voronoi-GRU-Based Multi-Robot Collaborative Exploration in Unknown Environments
###### Abstract
Model-based methods often leverage explicit representations of the environment and robot dynamics to achieve optimal or near-optimal exploration strategies. Deep Reinforcement Learning (DRL) [17] and Multi-Agent Reinforcement Learning (MARL) approaches [18, 19] have also been widely applied to multi-robot collaborative exploration due to their ability to learn complex policies from raw sensory inputs [20]. The corresponding algorithms enable robots to adapt to dynamic environments and unforeseen challenges without explicit programming, thereby enhancing their decision-making capabilities [21]. Therefore, the multi-robot collaborative exploration problem can be effectively modeled as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP) [22], where each robot acts as an agent. Despite these advancements, existing methods face several challenges. Voronoi partitioning lacks adaptability in highly dynamic or unpredictable environments, as static partitioning may not reflect current environmental states or robot positions [1]. Conversely, MARL approaches, while powerful in learning collaborative behaviors, often suffer from slow convergence rates, scalability issues, and high communication overhead, especially when applied to large-scale or complex real-world scenarios [23, 24]. Furthermore, many existing MARL methods are validated in simplified two-dimensional grid environments, limiting their applicability to practical exploration tasks that involve continuous and high-dimensional spaces [25]. To address these challenges, we propose a hierarchical exploration framework based on Voronoi partitioning and an enhanced MATD3, termed V-R-MATD3. This framework first utilizes Voronoi diagrams to partition the environment, autonomously assigning exploration areas to each robot to minimize redundant exploration and facilitate scalability [26]. Our choice of a learning-based approach, specifically V-R-MATD3, is motivated by its inherent adaptability [27, 28]. Research [17] has demonstrated that model-free reinforcement learning provides greater flexibility in robotics applications and is particularly suitable for addressing complex problems. Unlike model-based methods, V-R-MATD3, which is based on the TD3 algorithm [29], learns directly from experience, allowing it to handle complex, high-dimensional state spaces, unmodeled dynamics, and sensor noise more effectively [30, 31]. The integration of a Gated Recurrent Unit (GRU) layer further amplifies this adaptability by allowing robots to capture temporal dependencies within their observational data streams, thereby enhancing the coordination and informed decision-making capabilities of exploration strategies [32]. Additionally, we optimize the navigation and mapping performance by utilizing advanced SLAM techniques, providing high-quality experimental data and accelerating the training process. The main contributions of this paper are as follows:  
1. An Innovative Decision-Making Framework: We propose V-R-MATD3, a hierarchical framework that, for the first time, combines Voronoi-driven spatial decomposition with GRU-augmented MATD3 for multi-robot exploration. The Voronoi partitioning allocates non-overlapping regions to eliminate redundant coverage, while the enhanced MATD3 algorithm leverages Voronoi-constrained action spaces and dual critic networks to optimize collaborative policies. The GRU layers model sequential state transitions, ensuring temporally coherent decision-making. 2. Development of a Multi-Robot RL Toolkit: We develop an open-source toolkit compatible with the ROS and Gazebo platforms for multi-robot reinforcement learning. By adjusting the Cartographer framework, we achieve high-precision joint map construction, ensuring data quality during accelerated training. 3. Design of a Prior-Guided Adaptive Hybrid Reward (PGAHR) Function: We propose a hybrid reward function, termed PGAHR, which combines dynamically adjusted intrinsic rewards with fixed extrinsic rewards.