# Occupancy Reward-Driven Exploration with Deep Reinforcement Learning for Mobile Robot System
## 6 Simulation Results and Comparison
### Comparison  
In this subsection, the deep-RL and the nature-inspired algorithms are compared. In the authors' previous studies [21], the GWO algorithm for mobile-robot exploration showed the best result compared with the other nature-inspired optimization techniques. As a consequence, the GWO exploration algorithm is selected for the comparison analysis.  
Table 5 presents the comparison between the proposed occupancy reward-driven exploration and the nature-inspired exploration. The GWO exploration algorithm works with waypoints. To enable the robot to move somewhere, it needs to provide the robot a point to go to and check that the robot reaches the point in each time step. The continuous action is a more nature-driven action for the robot.  
Considering the development criteria, the GWO exploration algorithm requires more work in implementation than the DDPG one. A waypoint should be calculated based on some known parameters (frontier points, robot position) and algorithm logic, which should be considered in the programming. In this case, the DDPG agent is the more intelligent and straightforward developing tool. It should describe the RL environment and denote a reward function. The training process and a neural network find appropriate decisions for the RL environment.  
Figure 11: The episode reward graph during the training. The light blue line is the reward function. The dark blue line is the average of the reward function. The orange line is the trained critic data. It can be seen that the overfitting of the actor-critic occurred after 500 episodes.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Robot Motion** & **Development** & \begin{tabular}{c} **Processing** \\ **Result** \\ \end{tabular} & \begin{tabular}{c} **Map Coverage** \\ \end{tabular} \\ \hline \multirow{3}{*}{\begin{tabular}{c} _GWO exploration_ \\ \end{tabular} } & \multirow{3}{*}{Waypoints} & \
\begin{tabular}{c} Waypoint \\ computation, \\ algorithm logic \\ \end{tabular} & Waiting for the &
\begin{tabular}{c} 91.21\%, average \\ result of \\ 10 simulation runs \\ 99\% for simple \\ environment, 86\% \\ for complex \\ environment \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table}
Table 5: Comparison analysis of the GWO exploration and the DDPG agent exploration. The advantages are highlighted in bold.  
The processing result criteria is about obtaining the best performance of the algorithm. The GWO exploration is a stochastic algorithm that returns different results every simulation run. The best result is unpredictable. It can appear in the first 10 simulation runs or 100 runs. It needs to test and wait for the best result using the GWO exploration algorithm. For the DDPG exploration, the training takes a long time, for instance, 57 h (around 3 days) for the one-experiment result presented in Figure 10.  
Considering the map coverage criteria, the DDPG exploration has the greatest percentage result. However, it is only for the free-obstacle environment. Thus, the two approaches are not universal algorithms for the mapping problem. This is a disadvantage.