# A Deep Reinforcement Learning Approach for Active SLAM
## 4 Discussion of Results
### Deep Reinforcement Learning
In the first two environments the evolution is similar with both agents, however, there is a big difference on how they achieve such behavior: while D3 QN\({}^{\ddagger}\) agent looks for the uncertainty decrease, D3 QN achieves it accidentally due to the reduced topological complexity of the environments. In the third scenario, which is topologically more complex and where D3 QN left several areas unvisited (and therefore loop closures), the use of D3 QN\({}^{\ddagger}\) results on a significant reduction of _D-opt_ (\(\approx\) 34%). Also, note how the increasing monotonicity property is hold and how discontinuities occur due to loop closures. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Agent** & **SR(\%)** & **Steps** & **R** \\ \hline \multirow{4}{*}{Env. 1} & DQN & 100 & 500 (6) & 352.14 (0.1) \\ \cline{2-5}  & DDQN & 100 & 500 (6) & 337.56 (0.2) \\ \cline{2-5}  & D3 QN & 100 & 500 (6) & **355.31 (0.3)** \\ \cline{2-5}  & D3 QN \({}^{\ddagger}\) & 100 & 500 (6) & 300.1 (0.3) \\ \hline \multirow{4}{*}{Env. 2} & DQN & 72.8 & 312(13) & 95.55 (14.4) \\ \cline{2-5}  & DDQN & 100 & 500 (6) & 192.52 (0.0) \\ \cline{2-5}  & D3 QN & 89.3 & 419(9.3) & 196.5 (0.7) \\ \cline{2-5}  & DQN \({}^{\ddagger}\) & 100 & 500 (6) & **269.98 (0.4)** \\ \cline{2-5}  & D3 QN \({}^{\ddagger}\) & 100 & 500 (6) & **241.1 (29.2)** \\ \hline \multirow{4}{*}{Env. 3} & DQN & 0 & 170(15) & \(-\)24.89(9.3) \\ \cline{2-5}  & DDQN & 0 & 253(16) & \(-\)42.15(4.8) \\ \cline{1-1} \cline{2-5}  & D3 QN & 0 & 432(18) & 241.56(0.5) \\ \cline{1-1} \cline{2-5}  & D3 QN \({}^{\ddagger}\) & 26 & 459(90) & **261.62**(0.5) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Mean evaluation results in all environments for DQN, DDQN and D3 QN agents trained with \(\mathcal{R}_{\mathcal{F}}\). Superscript \({}^{\dagger}\) refers to an agent that was allowed to train briefly before its evaluation on that scenario, while \({}^{\ddagger}\) does to an agent retrained with \(\mathcal{R}_{aug}\). Figure 7 contains the maps generated online by the SLAM algorithm in the three environments after about 150 testing episodes. Images also contain starting positions (red circles), the detection of loop closures (black stars), the robot's trajectory (red arrows) and its uncertainty in X and Y directions (yellow ellipses). Maps are depicted for D3 QN (top) and D3 QN\({}^{\ddagger}\) (bottom) agents. Firstly, it is notable the lower uncertainty achieved by the proposed agent in all cases, and the effectiveness of resampling in comparison to the traditional agent. Moreover, trajectories followed are more optimal and angular velocity is applied more sparsely (source of uncertainty increase). In the third environment, D3 QN\({}^{\ddagger}\) agent is strikingly capable of selecting different paths to explore a greater area and resample the particles (i.e., close the loop) twice. Note that although these maps have been hand-selected to represent a common behavior of the agent's policy, do not always reproduce equally due to the environment and simulation's randomness. Finally, a discussion on the complexity of this approach with respect to the traditional active SLAM algorithm is presented in Appendix B.  
Figure 6: Evolution of the D-optimality criterion during evaluation for the agents trained with \(\mathcal{R}_{\mathcal{F}}\) (D3 QN, red) and \(\mathcal{R}_{aug}\) (D3 QN\({}^{\ddagger}\), blue).