## 4 Experimental Results
### Ablation Study
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Evaluation**} & \multicolumn{4}{c}{**Methods**} \\ \cline{2-5}  & **MPC** & **TD3 + CER[M]** & **TD3 + CER** & **TD3** \\ \hline Euclidean Distance & \multirow{2}{*}{0.00 m} & \multirow{2}{*}{417.20 m} & \multirow{2}{*}{1171.41 m} & \multirow{2}{*}{1530.63 m} \\ to MPC Trajectory & & & & \\ \hline Euclidean Distance & \multirow{2}{*}{1.26 m} & \multirow{2}{*}{8.58 m} & \multirow{2}{*}{5.82 m} & \multirow{2}{*}{47.89 m} \\ to Target Line & & & & \\ \hline Average & \multirow{2}{*}{27.53 m/s} & \multirow{2}{*}{25.61 m/s} & \multirow{2}{*}{22.63 m/s} & \multirow{2}{*}{20.51 m/s} \\ Velocity & & & & \\ \hline Number of & \multirow{2}{*}{11} & \multirow{2}{*}{9} & \multirow{2}{*}{3} & \multirow{2}{*}{0} \\ Lane Changes & & & & \\ \hline Computational Time & \multirow{2}{*}{6.99 s} & \multirow{2}{*}{0.76 s} & \multirow{2}{*}{0.78 s} & \multirow{2}{*}{0.72 s} \\ for 1000 Steps & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Quantitative analysis in Scenario 4. Considering the sum of Euclidean distance to the target lane as an indicator of how well the agent adheres to sub-goals, TD3 + CER outperforms the baseline method. Furthermore, TD3 + CER[M] generates trajectories most similar to MPC, as evidenced by the criteria of Euclidean distance to the MPC trajectory. Notably, the computation time of the RL agent is significantly less than that of MPC, suggesting the potential of substituting MPC with an RL agent trained using our proposed method. We assessed the trained policy of each method across two-, three-, and four-lane highway environments, conducting 100 repetitions for each test. Success for each episode was defined as the agent completing 1000 time steps without encountering forced termination conditions (such as collision or deviation from the track). The success rate, representing the number of successful cases out of the total cases, is presented in Table 4. Additionally, we analyzed the average quality of maneuvers based on deviation and velocity, as outlined in Table 5. Our method consistently exhibited the smallest deviations from the lane center across all environments, while also maintaining velocity within the desired bounds. Collision with other vehicles was the sole cause of failure. In the two-lane environment, the absence of additional lanes made it challenging to overtake the slow-moving vehicles ahead, necessitating more delicate acceleration control. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Domain**} & \multicolumn{3}{c}{**Methods**} \\ \cline{2-4}  & **TD3** & **TD3 + CER[M]** & **TD3 + CER** \\ \hline
2-lane & 53/100 & 95/100 & 94/100 \\ \hline
3-lane & 78/100 & 98/100 & 98/100 \\ \hline
4-lane & 72/100 & 96/100 & 92/100 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Quantitative analysis: Success rate measured as the number of successful cases out of the total cases. Out of 100 random road environment conditions, each with a corresponding number of lanes, TD3 + CER[M] exhibits the highest success rate. Figure 16: Maneuver and velocity analysis with the addition of the MPC expert demonstration method. When the agent employs MPC demonstration, it trains to imitate the MPC’s policy more straightforwardly.