# Reinforcement-Learning-Based Trajectory Learning in Frenet Frame for Autonomous Driving
## 1 Introduction  
Autonomous driving (AD) is a long-standing challenging problem that requires intelligent decision making to achieve long-term objectives based on the perception of surrounding environments. Recently, AD has received considerable attention, primarily due to its potential advantages in convenience and safety [1; 2].  
A conventional approach to address AD problems involves the integration of path planning and control. Several methods have proposed a strategy that first plans an optimal path and executes the planned path by using feedback control or model predictive control [3; 4; 5]. However, those approaches have a drawback in that path planning can be time consuming, particularly for long-horizon plans. In this regard, recently, learning-based approaches have been proposed to train reactive controllers for driving [6; 7; 8; 9]. Hence, once the controller is trained, path planning becomes unnecessary, and autonomous driving can be executed directly using the trained controller.  
Learning-based methodologies have successfully trained controllers to perform various autonomous driving tasks, but they still face two primary challenges. Firstly, most of these methods train policies to output abstract low-level commands like steering angles, acceleration/deceleration, or target regions [10; 11; 12; 13]. Consequently, they lack explainability for passengers. For example, in low-level control, the immediate control signals are provided by the policy, making it difficult for passengers to understand where the autonomous vehicle is heading in the future. Similarly, in the case of high-level policies, they express areas like merging zones in merging scenarios or areas to change lanes in lane-changing scenarios, making it equally challenging to convey how the vehicle will maneuver precisely. The second challenge lies in encoding path variations along curves. Most experiments areconducted on highways, where features primarily include surrounding vehicles' speeds, angles, and accelerations. As a result, these features often fall short in effectively representing the curvature of the road. In environments with pronounced curves, as depicted in Figure 1, the learned policy tends to struggle with generalization.  
To address explainability and the issues associated with curve-based learning, policies need to model the vehicle's short-term path. If the policy can provide short-term path predictions, it can offer passengers insights into the future positions of the vehicle, enhancing explainability. Additionally, by verifying whether the policy correctly produces appropriate curves based on curvature, we can assess whether the policy understands curvature correctly. Furthermore, this approach allows for collision checking along the output path in advance, enhancing safety guarantee.  
We propose reinforcement-learning-based trajectory learning in the Frenet frame for learning drivable trajectories in complex urban scenarios that require obstacle avoidance on narrow roads with curves, as depicted in Figure 1. To learn trajectories based on curvature, we utilize the Frenet frame. The Frenet frame offers advantages due to its intuitive and convenient nature as it unfolds the coordinate system along a reference line of a road. This allows for the generation of trajectories accommodating various curvatures when a reference line is available. Since the actions of autonomous vehicles must remain consistent within a given state, whether expressed in terms of on-road position or egocentric coordinates, using the Frenet frame is essential to generate paths compatible with the road [14]. However, it is worth noting that the Frenet frame has some limitations in modeling real-world kinematics and potential distortions in object shapes, making it less suitable for curvy roads [15], which can lead to challenges in obstacle avoidance. We addresses these limitations through reinforcement learning, successfully overcoming them and enabling the effective learning of trajectories for obstacle avoidance.