# NRNH-AR: A Small Robotic Agent Using Tri-Fold Learning for Navigation and Obstacle Avoidance
## 4 Results
### Static Environment
It can be clearly observed that the NRNH-AR agent reaches high and stable reward values from the very beginning of the training process. This rapid convergence can be attributed to the previous self-supervised and unsupervised learning stages that provided the agent with structured, informative representations before reinforcement learning began. In contrast, the baseline algorithms (DQN, TD3, DDPG) exhibit highly unstable learning dynamics in early episodes, with frequent oscillations and negative rewards. DDPG struggles significantly during the first 100 episodes, suggesting poor initial policy exploration. Although all three baseline algorithms eventually stabilize and improve their performance, they require substantially more episodes to reach comparable performance levels to NRNH-AR. Figure 12 presents the performance of three reinforcement learning algorithms, TD3-m, DDPG-m, and NRNH-AR, which was trained in the environment shown in Figure 7a, after integrating SSL and USL as pre-training stages before DRL. This pre-training strategy was designed to improve sample efficiency and reduce the overall training time, enabling the agent to extract structural knowledge from the environment before reward-driven interaction began. It is worth mentioning that the DQN is not included in this evaluation, because it uses only two networks, whose structures are not compatible with the SSL and USL phases. The impact of the SSL and USL phases is evident when comparing the behaviors of TD3-m and DDPG-m with their original versions, as shown in Figure 11. TD3-m and DDPG-m exhibit significantly smoother learning trajectories from the beginning of training, avoiding the prolonged periods of instability seen earlier. Overall, the results reinforce the importance of tri-fold learning, e.g., SSL and USL before applying DRL. The good  
Figure 11: Episode-wise reward comparison of DQN, TD3, DDPG, and NRNH-AR (DRL stage). The maximum number of steps per episode is 30.  
performance of TD3-m indicates that the replacement of our DRL with TD3-m or other more efficient DRLs compatible with previous SSL and USL phases can be useful. Table 6 shows that the proposed NRNH-AR method achieves the highest accuracy (97.62%) among all tested DRL algorithms in the physical robot environment. Compared to the standard versions of DQN (81.90%), DDPG (56.19%), and TD3 (68.57%), NRNH-AR outperforms them significantly, indicating better policy learning and generalization under real-world conditions. Moreover, the modified versions DDPG-m and TD3-m, which were adapted for low-resource hardware and tuned to the test environment, achieve improved results (86.19% and 96.19%, respectively) over their original counterparts. However, even the best-performing baseline (TD3-m) remains slightly below the proposed method. In dynamic environments, the proposed NRNH-AR algorithm achieved an accuracy of 86%. Baseline algorithms were not evaluated under dynamic conditions, as their low performance in static environments did not justify further testing. Additionally, comparing against modified versions (DDPG-m and TD3-m) would not be appropriate, since they incorporate the same pre-training methodology as NRNH-AR and do not represent the original algorithms. Based on our observations, the performance of NRNH-AR exhibits diminishing returns beyond a certain amount of training data. For example, CNN 2 achieves adequate generalization when trained with approximately 623 samples, as indicated by a loss value under 1.0. Similarly, learning oscillation in the SSL and USL phases stabilizes after around 200-250 episodes. These empirical thresholds allow us to keep training time and memory usage low, in line with our focus on constrained hardware. \begin{table}
\begin{tabular}{c c c c c c c} \hline  & **DQN** & **DDPG** & **TD3** & **OUR** & **DDPG-m** & **TD3-m** \\ \hline Accuracy & 81.90\% & 56.19\% & 68.57\% & 97.62\% & 86.19\% & 96.19 \\ \hline \end{tabular}
\end{table}