# Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning
## 4 Experiments
### Single-Robot Experiments  
In this section, we discuss the single robot mapless navigation task. The training procedure of our robot is implemented in the indoor scene simulated by Gazebo. We train our robot in several indoor scenes with different obstacle placement. We randomly set the robot at the start position with different initial directions. To guarantee the variety of the navigation trajectories and overcoming the overfitting, the targets are randomly chosen outside the restricted zone within the indoor scenes. It can reduce the gap between the training and testing of the policy module.  
In the simulation environment, the training procedure can be sped up 10 times. With the simulation acceleration, the control frequency of the mobile robot can reach 100 Hz. The safety threshold of the mobile robot is defined as 0.3 m. In addition, the arrival threshold is also defined as 0.3 m. In other words, if the distance between the mobile robot and the target is less than 0.3 m, this episode will be terminated. Other parameters of the single robot experiments are illustrated in Table 1.  
To compare the performance between the improved DDPG and the classical DDPG, we trained both of them with the same hyperparameters. As shown in Figure 3, the negative Q value of the improved DDPG illustrated by the orange line has a steady decline, while the negative Q value of the classical DDPG illustrated by the blue line increases slightly. Since the Q value evaluates the performance of the policy module, this means the improvement of DDPG becomes better during training, and the classical DDPG falls into local optimum. By analyzing the experiment results, we can infer that the modification in the improved DDPG can address the single robot mapless navigation tasks properly.  
For the navigation task that has a high-dimensional decision space, the improved DDPG with the switchable controller and the prioritized experience replay [17] technique can quickly constrain the searching field of policy module. Then, the policy module will be improved gradually. In the simulation environment, the robot with classical DDPG has the possibility to go around in circles. In addition, the robot with improved DDPG can arrive to the target position in most cases. We tested the improved DDPG policy module in the training environments 80 times, and its arrival rate can even reach 95%. When we move it to the unseen environment, the arrival rate declines to 87.5%. Figure 4  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Parameter** & **Batch Size** & **Max Step** & **LR (Actor)** & **LR (Critic)** & **Discount** & **Buffer Size** & **Explore Decay** \\ \hline Value & 128 & 800 & 0.001 & 0.0001 & 0.99 & 100,000 & 0.998 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Hyperparameters of the networks on a single-robot navigation task.  
Figure 3: The negative Q value comparison between the improved deep deterministic policy gradient (DDPG) and the classical deep deterministic policy gradient(DDPG) and the classical DDPG of the single robot navigation experiments.  
shows the single robot navigation experiments with the improved DDPG. The left part shows the simulation environment, and the right part illustrates the navigation trajectories of the mobile robot. The trajectories are separately visualized by the Rviz toolkit. The robot starts to navigate from the star and ordinally traverses through target 1 to target 6. When the robot arrives at target 6, it will set target 1 as the next destination and repeat this cycle.