## 4 Experiment and Results
### Experiment Results  
Two algorithms were used in the experiment, a DQN-based IQL algorithm and a DDQN-based IQL algorithm. The first experiment was conducted with the algorithm itself, and the second experiment was performed with algorithms and the proposed reward model together. We did the experiment five times for each test case and visualized the experimental results for each agent. The experimental results of DQN and DDQN can be seen in Figures 12 and 13. In the DQN experiment, rewards started to occur irregularly at various points between 100,000 and 700,000 steps, and learning began. Once the reward was created, the reward value continued to increase and learning progressed. In the DDQN experiment, rewards began to occur at various points ranging from 100,000 to 500,000 steps. Comparing the test results of DQN, the difference is that even after the reward is generated, the increase in the reward value sometimes decreases slightly and variably. By means of the experiments of DQN and DDQN, we confirmed that there are rare cases in which rewards do not occur occasionally until the end of learning. The experimental results of DQN and DDQN using the proposed Dual Segmented Reward Model are shown in Figures 14 and 15. The results of DQN and DDQN using the proposed reward model are very similar. The reward occurred between 100,000 and 200,000 steps, and the learning progressed stably with little deviation. In addition, we confirmed that a small reward value was generated from the beginning. This change helped ameliorate the sparse reward problem. In addition, the proposed reward model was effective in reducing the learning deviation between each experiment and stably maintaining the learning. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Learning Rate (\(\alpha\)) & 0.00008 \\ Discount Rate (\(\gamma\)) & 0.99 \\ Reward Decay Rate (\(\lambda\)) & 0.99 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Parameter values for test. Figure 12: Test Result of DQN: (**a**) Agent 1; (**b**) Agent 2. Figure 14: Test Result of DQN with Dual Segmented Reward Model: (**a**) Agent 1; (**b**) Agent 2. Figure 13: Test Result of DDQN: (**a**) Agent 1; (**b**) Agent 2. Figure 15: Test Result of DDQN with Dual Segmented Reward Model: (**a**) Agent 1; (**b**) Agent 2. In order to check more statistically, the sum of the reward values of Agent 1 and Agent 2 was calculated for the results of five experiments, and the minimum, maximum, and average reward values for each time step were visualized in Figure 16. The gray area represents the range from the minimum reward value to the maximum reward value for each step, and the orange line represents the average of five experiments. Comparing the visualization diagrams of each test case, the difference between the minimum and maximum rewards is very large and the gray area is wide in the cases of DQN and DDQN. However, when the Dual Segmented Reward Model method is applied with DQN and DDQN, the difference between the minimum and maximum rewards is significantly reduced and gray area is also narrowed. The reward values of the experimental results for all test cases are compared in Figure 17. The reward values of the five experiments were averaged without any agent classification. In the experiments conducted with the DDQN algorithm, the rewards started to increase slightly faster than in the experiments conducted with the DQN, and the rewards were also slightly higher. When the proposed reward model was applied, we confirmed that the experimental results were consistent and similar regardless of the algorithm DQN or DDQN. Hence, the proposed reward model works better and more stably than did the experiment using only the DQN or DDQN algorithm in terms of timing and pattern of reward increase. Figure 16: Reward Summation Analysis of Test Results: (**a**) DQN; (**b**) DDQN; (**c**) DQN with Dual Segmented Reward Model; (**d**) DDQN with Dual Segmented Reward Model.