## 4 Experimental Results and Analysis  
### Performance Evaluation  
The experimental results for total rewards per episode are visualized through two graphs. Figure 2 shows the total rewards for 100 episodes, while Figure 3 presents the total rewards for 1000 episodes. In the analysis of 100 episodes in Figure 2, the observed reward values of 0 in the 25 th and 80 th episodes represent failures where the agent failed to reach the goal point. The fact that only two failures were recorded out of a total of 100 episodes shows a high success rate of 98%, which indicates that the LLM-based reinforcement learning approach has a stable navigation ability similar to that of traditional reinforcement learning algorithms. From a long-term perspective over 1000 episodes, the reward distribution remains within a similar range, indicating that the agent's performance does not change significantly over time. This reflects the consistent performance of the LLM. Although there is some variability in rewards, this suggests adaptability to different initial conditions. Figure 3: Total reward results for 1000 episodes of LLM reinforcement learning. Figure 2: Total reward results for 100 episodes of LLM reinforcement learning. High rewards can be interpreted as a high goal achievement rate and efficient path selection. Since high rewards were recorded for most episodes, this indicates that the goal achievement rate is maintained at a relatively high level. Figure 4 illustrates the total rewards over 100 episodes when using a randomized algorithm, providing a comparative perspective with LLM-based reinforcement learning. The graph reveals a wide distribution of rewards, ranging from 0 to 1, highlighting the instability and unpredictability of the randomized approach. This high variability in rewards across episodes indicates that the agent fails to consistently achieve the goal, resulting in a low success rate and significantly reduced consistency in performance. These results underscore the inefficiency of randomized algorithms for goal-oriented tasks, while simultaneously supporting the superior stability and performance demonstrated by LLM-based reinforcement learning in this study. Figure 5 illustrates that even over a long-term analysis of 1000 episodes, random actions exhibit significant variability, with rewards distributed randomly without any discernible pattern. This indicates that the agent fails to adapt to changes in the environment. The goal achievement rate is low, and the average number of steps required is high, demonstrating the inefficiency of random actions. These results emphasize that the random algorithm does not function effectively in complex environments and struggles to reach the goal. LLM-based reinforcement learning demonstrates significant advantages over PPO, DQN algorithms, and random approaches, as shown in Table 4. While PPO and DQN  
Figure 4: Total reward results for 100 episodes of random algorithm reinforcement learning. Figure 5: Total reward results for 1000 episodes of random algorithm reinforcement learning. algorithms achieve moderate success rates (461 and 494, respectively), they exhibit higher variability in rewards, with standard deviations of 0.48 and 0.66. In contrast, the LLM-based approach achieves the highest success rate (953) and the lowest standard deviation (0.24), indicating not only superior goal achievement but also greater stability during learning. The ability of LLMs to interpret environmental states in natural language plays a crucial role in determining effective behaviors. By leveraging their pre-trained contextual knowledge, LLMs excel in assigning precise and consistent rewards, significantly reducing the dependency on random exploration. This is evident in the stable reward trends observed in Figures 2 and 3, where LLMs outperform traditional methods by maintaining high rewards with minimal fluctuation across episodes.