# Vehicle Lane Change Models--A Historical Review
## 2 Vehicle Lane Change Models
### Empirical Models of Lane Change
#### 2.1.2 Reinforcement Learning Models
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Paper** & **Input Variables** & **Algorithm** & **Key Issues to Be Resolved** \\ \hline
[25] & Crash data & Random logit with heterogeneity in means models & Relationship between young, inexperienced and older drivers and crash factors during lane changes \\ \hline
[26] & US-101 NGSIM database & Binary logistic + CTM & Improve the accuracy of macroscopic traffic state estimation \\ \hline
[29] & Distance-dependent factors and velocity-dependent factors & A random parameters logit approach with heterogeneity in means and variances & The dynamic trade-off between safety and efficiency in the decision-making process of autonomous lane change and its influencing factors \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Cont. traffic environment, it is challenging to model the decision-making about whether to make a vehicle lane change. In a rule-based constrained environment [45], [PERSON] et al. [46] used a deep Q-network to make lane change decisions. A safer and more efficient lane change strategy can be obtained by combining high-level lateral decision-making with rule-based trajectory monitoring. The experimental results showed that the rule-based reinforcement learning algorithm performed better than the reinforcement learning algorithm. [PERSON] [47] proposed an automatic driving reinforcement learning system based on previous studies [48; 49], which can improve the safety of multi-mode future trajectory prediction. The safety system consists of two safety components: rule-based and multimodal learning. The rule-based safety system is mainly founded on general driving rules, while the multimodal learning safety system learns safety patterns from historical driving data. Mixed-density recurrent neural networks (MD-RNNs) [50] are used to predict multimodal future trajectories and to accelerate the learning process by simulating the potential behavior of autonomous agents [51]. Experimental simulation results show that the proposed automatic driving reinforcement learning system has excellent performance in terms of average reward and collision frequency. Efficient lane changing is an important task in intelligent transportation systems, and it is challenging to realize efficient lane changing for multiple vehicles at the same time in a crowded traffic environment. [PERSON] et al. [52] proposed a lane change model that discretizes traffic lanes into units and combines forced lane change and free lane change maneuvers [53], and they proposed a request-response mechanism algorithm based on a deep Q-network. The lane change system is divided into an independent group and symmetric group, based on Q-learning and the concept of a \"central agent\", to simplify the training process. In the proposed method, [PERSON] et al. [52] outlined a request group and response group. During training, the request group only considers the status of the group to train the agent. During training, the response group not only considers the status of the group but also has to consider the superimposed actions (i.e., request messages) from the request group, as shown in Figure 2 below. The implementation process, like the training process, is handled in a decentralized manner. The proposed algorithm is compared with a rule-based method, game-theory-based decomposition algorithm, and simple deep Q-learning. The experimental results show that the performance of this algorithm is not much different from that of the game-theory-based decomposition algorithm, and the efficiency of lane change is better than the other two algorithms. The calculation time during the execution of this algorithm is less than that of the decomposition algorithm based on game theory [54]. Table 2 gives a summary of the models based on reinforcement learning.