## 4 Experimental Results
### Ablation Study  
In our MDP formulation, we integrated trajectories from the MPC controller as odometry references in the calculation of the running reward. These trajectories were repurposed as expert demonstrations during the training of our motion planner, with the expectation of accelerating convergence in the training process. This variant, referred to as CER[M], employed approximately 10% of each buffer for MPC demonstrations to mitigate the risk of policy failures due to extrapolation errors [41]. Consequently, TD3 + CER[M] was trained with two behavior policies: the agent itself and the MPC policy. Figure 15 illustrates the reward graph of the proposed methods (TD3 + CER and TD3 + CER[M]) alongside the baseline method (TD3). Notably, TD3 + CER[M] achieved convergence much faster compared with TD3 + CER, indicating that CER can significantly enhance the performance of the trained agent. Moreover, the judicious use of expert demonstrations can expedite convergence speed during training. In Figure 16, the maneuver and velocity changes of each algorithm's best-performing policy are visualized. Each algorithm generates distinct trajectories under the same scenario. Notably, TD3 + CER[M], trained with MPC demonstrations, exhibits similar maneuver and velocity changes to MPC during the evaluation period. Table 3 provides a detailed comparison of each trajectory, focusing on trajectory similarity to MPC and deviation from the target lane. The accumulated Euclidean distances \begin{table}
\begin{tabular}{c c c c} \hline
**DM Parameters** & **Original** & **Aggressive** & **Tolerant** \\ \hline \(TTC_{\text{front}}\) & 7 s & 10 s & 3 s \\ \(TTC_{\text{rear}}\) & 4 s & 8 s & 2 s \\ \hline \end{tabular}
\end{table} 
Table 1: DM parameters for different driving styles. \begin{table}
\begin{tabular}{c c c c} \hline
**Analysis** & **Original** & **Aggressive** & **Tolerant** \\ \hline \(N_{\text{Lc}}\). & 9 & 6 & 8 \\ \(v_{\text{avg}}\) & 25.2 m/s & 25.1 m/s & 22.4 m/s \\ \hline \end{tabular}
\end{table} 
Table 2: Analysis results for different driving styles. Figure 14: Action diversity evaluation with the TD3 + CER agent. Table 1 displays varying hyperparameters of the decision-making modules: “original”, “aggressive”, and “tolerant”. (**a**) demonstrates different maneuvers made according to the varying decision-making modules. The distance and velocity plots in (**b**,**c**) reveal more evident characteristics of each decision-making module, along with the corresponding trajectories generated from each module. This result indicates that the trained MP module agent does not overfit to specific DM modules (“original”) but adheres to sub-goals. between the MPC trajectory and the target line throughout the scenario are presented, reflecting the trajectory similarity criterion. Additionally, the deviation from the target lane, indicative of decision-making module responses at each time step, is analyzed. Among the RL-based approaches, TD3 + CER[M] demonstrates the closest trajectory similarity to MPC and the smallest deviation from the target lane. While exhibiting trajectory similarities with the MPC motion planner, our proposed method achieves a significant reduction in computational time during the scenario, approximately by a factor of one-tenth. Figure 15: Reward curves of baseline method (TD3) and proposed method with (TD3 + CER) and without MPC expert demonstration (TD3 + CER[M]).