## 2 Related Works  
Many prior studies have employed RL for driving tasks, adopting diverse approaches such as end-to-end architectures [29; 30] and hierarchical methods [10; 11; 12]. End-to-end approaches often oversimplify the complexity of driving tasks, encompassing multiple decision, planning, and control layers in dynamic environments. For instance, ref. [29] utilized Deep Deterministic Policy Gradients (DDPGs) [31] to generate control inputs like steering and throttle from image inputs on a two-lane track. However, their assumption of an environment with only the ego vehicle limited realism, as other vehicles were neglected. A similar constraint applied to the use of Dueling Deep Q Networks (DQNs) in [30], where the training and testing tracks were identical, hindering generalization. Unfortunately, typical end-to-end approaches intertwine actions with decisions, making maneuvers challenging to interpret and necessitating relearning from scratch when environmental conditions change. In contrast, hierarchical approaches, as demonstrated by [10], segregate the decision-making process from the trajectory generation process, mitigating the drawbacks of the end-to-end paradigm. Such approaches employ multiple agents to address various behaviors in driving situations.  
In recent years, transformer [32] has emerged as a promising tool for training sequential actions, with several studies adopting them for self-driving tasks [33; 34]. However, their large network size and computational demands pose significant challenges, particularly in real-time control applications such as autonomous vehicles.  
Unlike end-to-end architectures, hierarchical approaches face fewer constraints on task complexity. Here, the driving task is divided into decision-making and trajectory generation, with RL used as the decision-making module to generate high-level commands, such as lane keeping or lane changing. The motion planner is designed to capture vehicle dynamics using a two-dimensional grid map planning method [10]. Alternatively, in [35], RL is used for both decision making and motion planning with separate RL agents, initialized using the inverse RL method [36]. Although this approach avoids rule-based methods, designing and shaping rewards for each agent demands considerable time and effort.  
In another study, ref. [12] employed a conditional variational autoencoder for high-level decision making and an RL agent for control input generation based on the decision. While RL excels at handling unseen scenarios, it falls short in reflecting changes in traffic regulations and ensuring decision diversity. Moreover, limitations arise if speed limits are not included in the observation during training, requiring the agent to be trained anew. To address these concerns, some approaches leverage RL for decision-making and rule-based models for trajectory generation, as exemplified by [11]. This method utilizes a rule-based decision making module and generates control inputs from an RL agent, specifying decisions more comprehensively in both lateral and longitudinal directions.  
However, the efficiency of designing and training multiple specialized agents with this approach is questionable.  
To tackle these issues, our research incorporates a straightforward rule-based decision-making module to adapt to changes in traffic conditions and individual preferences. The control input is generated by the RL motion planner, ensuring the adaptive management of situations by utilizing concatenated state inputs from sub-goals and observations.