## 4 Experimental Results and Analysis
### Analysis of LLM's Action Selection Patterns  
A comparison between learning with the LLM and the random algorithm shows that the LLM-based agent demonstrates superior understanding of and adaptability to environmental states. By interpreting the given environmental state and selecting appropriate actions, the LLM effectively reduces the number of steps needed to reach the goal.  
As shown in the graph in Figure 6, the LLM-based agent consistently achieves the goal in fewer steps across multiple episodes. This suggests that the LLM appropriately responds to various environmental states. Notably, in some episodes, the agent reaches the goal with very few steps, demonstrating the LLM's strong reasoning ability and understanding of the environment.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Number of** & **Average** & **Standard** \\  & **Successes** & **Reward** & **Deviation** & **Notes** \\ \hline PPO & 461 & 0.44 & 0.48 \\ DQN & 494 & 0.47 & 0.48 \\ LLM & 953 & 0.66 & 0.24 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Algorithms’ results after 1000 episodes in MiniGrid-5x5 environment.  
Figure 6: Number of steps to success with LLM application.  
On the other hand, as shown in a second graph, in Figure 7, the random algorithm exhibits significant variability in the number of steps, indicating a lack of consistency in achieving the goal. Since random actions are selected without any specific pattern, it often takes many steps to reach the goal.  
The LLM interprets environmental states in natural language to determine the optimal action for each situation. As shown in Figure 8, this ability allows the agent to quickly adapt to complex state changes, resulting in a high goal achievement rate. The LLM-based agent considers the positions of walls and obstacles to select efficient paths, reducing the average number of steps required.  
Compared to the random algorithm, the LLM's responses are more systematic, which is evident in the reward graph. While LLM-based learning maintains high and stable rewards, the random algorithm shows significant variability and remains at a lower level.  
In conclusion, the LLM demonstrates superior performance compared to the random algorithm in reinforcement learning by deeply understanding environmental states and selecting appropriate actions. These results highlight the LLM's strong problem-solving and generalization abilities, suggesting the potential for even greater performance in future research. Furthermore, the strong performance of the LLM is not limited to simple environments like MiniGrid-5x5. To further substantiate the motivation for this research, additional experiments were conducted in a more complex grid environment (e.g., 16x16). The results showed high success rates even in the early stages, as the LLM utilized its pre-trained knowledge to quickly understand the structure of the environment and efficiently  
Figure 8: Reward results for environmental changes in MiniGrid-16x16.  
Figure 7: Number of steps to success with random algorithm application.  
navigate optimal paths to the goal. This characteristic contributes to enhancing initial success rates and reinforcing learning stability, even in moderately complex environments. Figure 8 illustrates the total rewards per episode in the 16x16 grid environment, showing consistently high reward values from the early stages.