# Reinforcement Learning Decision-Making for Autonomous Vehicles Based on Semantic Segmentation
## 3 Experimental Procedure and Results
### Simulation Results and Discussion
#### 3.5.2 Agent Training Results
In Table 3, the variance values represented the degree of fluctuation in the reward values and average deviation from the center, which indicated the stability of the agent's decision-making process during training. Smaller variance values meant that the agent's decision-control was more stable and consistent, with less fluctuation during the training process; conversely, larger variance values indicated that the agent's decisions had greater uncertainty or volatility. As shown in Table 3, in the range of 0-40,000 episodes, DDQN-SS and PPO-SS reached reward values between 100 and 1800, while DDQN and PPO reached reward values between 100 and 1200. This indicated that the integration of semantic segmentation improved the agent's ability to achieve maximum reward earlier in training. As the training progressed (40,000-80,000 and 80,000-160,000 episodes), the gap between the methods became more pronounced, with DDQN-SS and PPO-SS continuing to show significantly higher reward ranges, reinforcing the idea that semantic segmentation enhances the agent's  
Figure 9: (**a**) Average Deviation from Center of DDQN and DDQN-SS. (**b**) Average Deviation from Center of PPO and PPO-SS. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **Episodes (102)** & **DDQN** & **DDQN-SS** & **PPO** & **PPO-SS** \\ \hline \multirow{4}{*}{Reward Value Range} & 0–400 & 100–1200 & 100–1800 & 200–1800 & 100–1700 \\ \cline{2-6}  & 400–800 & 500–1500 & 1200–3500 & 500–2100 & 1000–3000 \\ \cline{2-6}  & 800–1600 & 1300–4500 & 1000–3700 & 700–4700 & 1000–4200 \\ \cline{2-6}  & 1600–2000 & 1800–5000 & 3000–4000 & 1000–4500 & 2800–4500 \\ \hline \multirow{4}{*}{Average Deviation from Center Range} & 0–400 & 0.2–0.66 & 0.25–0.48 & 0.3–0.7 & 0.22–0.63 \\ \cline{2-6}  & 400–800 & 0.2–0.6 & 0.25–0.43 & 0.35–0.45 & 0.25–0.3 \\ \cline{2-6}  & 800–1600 & 0.28–0.48 & 0.25–0.43 & 0.3–0.4 & 0.18–0.28 \\ \cline{2-6}  & 1600–2000 & 0.25–0.38 & 0.2–0.35 & 0.3–0.43 & 0.15–0.25 \\ \hline \multirow{4}{*}{Reward Value Variance} & 0–600 & 199,093.14 & 803,704.74 & 225,405.99 & 440,959.80 \\ \cline{2-6}  & 600–1400 & 985,624.10 & 347,929.36 & 967,823.92 & 318,713.77 \\ \cline{1-1} \cline{2-6}  & 1400–2000 & 587,908.79 & 392,691.37 & 839,591.20 & 203,899.19 \\ \hline \multirow{4}{*}{Average Deviation from Center Variance} & 0–600 & 0.163 & 0.005 & 0.013 & 0.012 \\ \cline{2-6}  & 600–1400 & 0.006 & 0.003 & 0.002 & 0.001 \\ \cline{1-1} \cline{2-6}  & 1400–2000 & 0.001 & 0.001 & 0.003 & 0.001 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Data Comparison of DDQN-SS, PPO-SS with DDQN, PPO. decision-making process by providing richer environmental information. In the range of 0-40,000 episodes, the deviation of DDQN-SS was between 0.25 and 0.48, while DDQN's deviation was between 0.2 and 0.66, reflecting better lane-keeping performance due to semantic segmentation. As training progressed, the deviation values of DDQN-SS and PPO-SS further decreased, indicating that, compared to DDQN and PPO, they performed better in lane centering, and their driving behavior was more stable, suggesting less fluctuation in their deviation from the center. This trend indicated improvements in decision-control, especially in maintaining stable trajectories in complex driving scenarios. In the parts with smaller reward value fluctuations, DDQN-SS and PPO-SS stabilized at 2500, which was an increase of approximately 25% compared to the traditional algorithm's reward value of 2000. Regarding the average deviation-from-center values, DDQN-SS stabilized at 0.35 (a 14.2% improvement over DDQN), and PPO-SS stabilized at 0.25 (a 28.5% improvement over PPO), indicating that the proposed method demonstrated greater advantages in driving stability. A detailed data comparison between the relevant algorithms is shown in Table 3.  
To assess the potential impact of segmentation accuracy on the overall performance of the proposed framework, an additional test was conducted to simulate the effect of different levels of semantic segmentation accuracy on the RL agent's decision-making.