# Reinforcement-Learning-Based Trajectory Learning in Frenet Frame for Autonomous Driving
## 5 Results
### Driving Results Comparisons  
To demonstrate that RLTF generates future-aware trajectories in complex scenarios, we compared it to a conventional trajectory generation (Traj. Gen.) method using the Frenet frame [19], an information-theoretic model predictive controller with neural networks (IT-MPC-NN) [29], and two other methods: supervised learning with behavior cloning (supervised BC) and reinforcement learning with behavior cloning using TD3 (TD3+BC). Supervised BC and TD3+BC have the same size of layers as RLTF, and these methods are trained to learn steering angle and throttle/brake values. All methods share the same reward function. We first conducted driving evaluations on the road used for training. All vehicle initializations were the same as in the training phase, with the number of obstacle vehicles varying from episode to episode, consisting of one in the rear and 0 to 2 in the front. Each method was evaluated for 900 episodes. The results of the comparative methods are shown in Table 3, where RLTF achieves the highest success rate, the fastest average speed, and the shortest computation time among all methods. While the other strategies did not achieve a 100% success rate, RLTF achieved 100% success. Figure 9 presents the driving trajectories during simulation for TD3+BC and RLTF. TD3+BC and RLTF managed to pass the first obstacle, but TD3+BC collided with the second obstacle. Additionally, it is worth noting that RLTF made earlier movements to avoid the first obstacle compared to TD3+BC. This result demonstrates that learning trajectories allow for better future planning, as RLTF exhibits a more proactive response in avoiding obstacles.  
We also conducted evaluations with 900 tests for each method on 12 unseen roads, as described in Table 4. The IT-MPC-NN algorithm demonstrated competitive performance on unseen roads, with a slightly lower average reward and success rate compared to RLTF but similar precision in path following. The 5% failure rate of IT-MPC-NN is mainly caused by the erroneous prediction of the NN dynamic model. Unlike the previous driving results, the performance of supervised BC improved on unseen roads. However, RLTF outperformed all other methods, with a 99.6% success rate. Furthermore, RLTF exhibited a lower average deviation from the road center than the other methods. A higher average deviation indicates that the vehicle deviates farther from the road center during driving.  
Figure 8: Comparison of trajectory variations based on curvature \(\kappa\). The red color becoming darker indicates higher \(\kappa\) and the blue color becoming darker indicates lower \(\kappa\).  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Algorithms** & **RLTF** & **IT-MPC-NN [29]** & **Traj. Gen. [19]** & **Supervised BC** & **TD3+BC** \\ \hline Average reward & 424.58 & 414.04 & 401.01 & 234.55 & 325.45 \\ Average success rate & 100 & 100 & 98 & 80.77 & 90.11 \\ Average speed & 17.74 & 16.81 & 14.98 & 14.44 & 14.29 \\ Computation time & 0.458 & 2.29 & 1098.96 & 1.176 & 1.18 \\ Average deviation & 0.05 & 0.07 & 0.76 & 0.6 & 0.05 \\ Average throttle value & 0.15 & 0.14 & 0.48 & 0.13 & 0.11 \\ Average steering angle & 1.78 & 1.71 & 1.63 & \(-\)1.47 & \(-\)1.42 \\ Average acceleration & 0.69 & 0.65 & 0.62 & 0.58 & 0.41 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Driving results on the training roads.