# A Novel Intelligent Approach to Lane-Change Behavior Prediction for Intelligent and Connected Vehicles

Authors: Luyao Du; Wei Chen; Jing Ji; Zhonghui Pei; Bingming Tong; Hongjiang Zheng
Journal: Computational Intelligence and Neuroscience
 (2022)

---

## Abstract

The prediction of lane-change behavior is a challenging issue in intelligent and connected vehicles (ICVs), which can help vehicles predict in advance and change lanes safely. In this paper, a novel intelligent approach, which considering both the driving style-based lane-change environment and the driving trajectory-related parameters of the ICV and surrounding vehicles, is proposed to predict the lane-change behaviors for ICVs. By analyzing the characteristics of the lane-change behavior of the vehicle, a modified dataset for the prediction of lane-change behavior was established based on the Next-Generation Simulation (NGSIM) dataset, which is made up of real vehicle trajectories collected by camera. In the proposed approach, the hidden Markov model (HMM)-based model is designed to judge whether the current environment is suitable for lane change according to the driving environment parameters around the vehicle; then according to the driving state of the vehicle, a learning-based prediction-then-judgment model is proposed and designed to realize the prediction of the ICV’s lane-change behavior. Experiments are implemented by using the modified dataset. From the experimental results, the lane-change probability value on the target lane in the truth of the lane-change behavior calculated by the designed HMM-based model is basically above 0.5, indicating that the model can make a more accurate judgment on the surrounding lane-change environment. The proposed learning-based prediction-then-judgment model has an accuracy of 99.32% for the prediction of lane-change behavior, and the accuracy of the lane-change detection algorithm in the model is 99.56%. The experimental results show that the proposed approach has a good performance in the prediction of lane-change behavior, which could effectively assist ICVs to change lanes safely.


---

## Body

## 1. Introduction

The intelligent and connected vehicle (ICV) [1] integrates modern communication and network technology and has environment perception, intelligent decision-making, and collaborative control functions. It can achieve safe, efficient, comfortable, and energy-saving driving and realize a new generation of vehicle that replaces humans [2].Lane-change behavior detection and prediction plays an important role in the ICV technology. During the driving of the vehicle, the current driving environment may be misjudged due to the occlusion of surrounding vehicles or the driver’s inattention, resulting in greater safety hazards. Therefore, the sensor and communication technology can assist the ICV to perceive and judge the surrounding environment and the state of the vehicle and combined with artificial intelligence technology can predict the lane-changing behavior, thereby improving the driving safety.Many methods have been proposed for lane-change detection and prediction, in which the main technical means and data sources used can be summarized as trajectories, steering wheel, surrounding environment, driving style, computer vision and roadside LiDAR, etc., as shown in Table1. Lane-change behavior detection methods based on trajectory data are proposed in Refs. [3–8], such as fuzzy logic [3], support vector machine (SVM) [4], long short-term memory network and convolutional neural network (LSTM-CNN) [5], maneuver classification [6], and hidden Markov model [7, 8]. Panichpapiboon and Leakkaw explore an approach to detect lane-change behavior using steering wheel angles extracted from the smart phone [9]. Zheng and Hansen propose an approach to detect lane-change behavior using the steering angle signal from CAN-bus [10]. Ali et al. propose a wavelet transform (WT)-based method to detect failed lane-changing attempts and used the random parameter binary logic model to study how the connected environment affects related parameters [11]. Woo et al. present a method to determine the driving styles and use the result to detect the lane-change behavior [12]. Nguyen et al. introduce a vision-based lane and vehicle detection approach for the lane-change assistant system [13]. Wang et al. present a method to detect lane-change behavior based on candidate lane markings [14]. Wei et al. develop a computer vision system to detect the lane-change behavior [15]. Cui et al. develop the methods to detect and predict lane-change behavior using vehicle trajectories from roadside LiDAR data [17]. Xu et al. present a V2X-based lane-change prediction model using vehicle trajectories [18]. Zhang and Fu present a lane-change intention detection method using motion parameters of the vehicle and surrounding vehicles [19]. Gao et al. introduce a lane-change behavior detection approach using multiple differing modality data [20]. Jin et al. present an optimal lane-change timing prediction model based on the driver’s habits [21]. Huang et al. present a trajectory planning and control approach based on user preferences [22]. Xing et al. propose a driving pattern analysis and motion prediction system that determines the trajectory according to user’s preference [23]. Xing et al. develop a driver intention inference system for highway lane-change maneuvers [16]. Xing et al. present a leading vehicle trajectory prediction approach that considers different driving styles [24].Table 1
Comparison of lane-change detection or prediction approaches.
TrajectoriesSteering wheelSurrounding environmentDriving styleComputer visionRoadside LiDAR[3–8]√[9, 10]√[11]√[12]√[13–16]√[17]√[18, 19]√√[20]√√[21]√√[22–24]√√Proposed approach√√√In the above studies, different methods and models using different technical means and considering the influence of different characteristic parameters have been designed and proposed, fully verified, and achieved great results. However, few studies have simultaneously considered the effects of the vehicle, environment and driver, and the relationship between them when the ICV changes lanes. In this paper, a novel intelligent approach combines the driving state of the vehicle, the surrounding driving environment, and the driving style is proposed to predict the lane-change behaviors for ICVs. First, based on the learning of the driving habits of manual drivers, the current lane-change environment is judged according to the driving state of surrounding vehicles. If the current environment is suitable for lane change, then the vehicle driving state parameters are predicted, and the lane-change behavior detection method is proposed to judge the predicted value, so as to predict the lane-change behavior. The main contributions can be summarized as follows.(i)
According to the relevant characteristic parameters of the vehicle lane change, the NGSIM dataset is processed and analyzed, so that a modified dataset for the lane-change behavior prediction is established(ii)
Based on the driving habits of manual drivers, a HMM-based model is designed to judge whether the current surrounding environment of the vehicle is suitable for the lane change(iii)
Based on the analysis of lane-change behavior characteristics, a prediction model based on LSTM and lane-change feature judgment method is proposed to predict the state parameters of the vehicle and determine whether it will change lanes(iv)
A novel approach based on intelligent and connected technology, which in combination with the driving style-based lane-change environment and the driving trajectory-related parameters of the vehicle and surrounding vehicles, is proposed and performed on the established dataset to predict the lane-change behavior of vehiclesThe rest of the paper is organized as follows. In Section2, the establishment process of the dataset is described. In Section 3, on the basis of fully analyzing the characteristics of lane-change behavior, the proposed approach to lane-change behavior prediction is introduced in detail. Section 4 gives the experimental results and analysis of the proposed approach performed on the modified dataset. Section 5 concludes the research work and presents the future work.


### 1.1. Dataset Establishment

In this paper, the NGSIM dataset is processed to obtain the vehicle’s trajectory and surrounding driving environment data, so as to combine the driver’s driving style to build and verify the vehicle’s lane-change prediction model.


### 1.2. Data Description

The NGSIM is a dataset of different sections initiated by the United States Department of Transportation (US DOT) Federal Highway Administration (FHWA) [25]. In the NGSIM, I-80 and US-101 are the datasets collected in highway, which are studied in this paper. As shown in Figure 1, both I-80 and US-101 consist of five main lanes, one distribution lane, one on-ramp, and one off-ramp (the off-ramp of I-80 is not located within the study area). In I-80, the 1650-foot-long study area is divided into seven sub-areas by seven cameras to record the relevant data, while in US-101, the 2100-foot-long study area is divided into eight sub-areas by eight cameras. The dataset contains the trajectory data of all vehicles in the study area during the recorded time period.Figure 1
Study area description of the dataset.
(a)(b)


### 1.3. Data Preprocessing

In order to analyze the characteristics of the lane-change behavior, characteristic parameters such as the coordinates and velocity of the vehicles are extracted from the NGSIM dataset. The coordinates of the ramp and the most adjacent lane have a large overlap, which will cause great interference to the study. Therefore, the data related to the ramp and the most adjacent lane are eliminated in the study. To further analyze the influence of the surrounding lane-change environment on lane-change behavior and the relationship between them, the distance between the vehicle and the front and rear vehicles on the current lane and adjacent lanes is calculated. The lateral speed of the vehicle is also calculated in order to predict the lane-change behavior. The complete data of 92 lane-changing vehicles, a total of 92932 frames, are finally screened out and processed; then a modified dataset is established.In the processed data, the surrounding lane-change environment at the time of a certain vehicle lane-change frame is shown in Figure2. The range of lane coordinates calculated according to the data in the processed dataset is shown in Table 2.Figure 2
Schematic diagram of vehicle lane-changing state frame.Table 2
The range of lane coordinates.
Coordinate (feet)Lane IDDataset1234I-800.21–12.2911.90–24.3823.74–36.1035.63–48.42US-1011.46–13.3311.51–24.4523.41–35.6833.58–46.29Coordinate (feet)Lane IDDataset5678I-8047.83–60.4259.52–82.4872.65–96.75US-10144.99–61.3057.19–69.6057.38–72.6058.04–72.92


## 1.1. Dataset Establishment

In this paper, the NGSIM dataset is processed to obtain the vehicle’s trajectory and surrounding driving environment data, so as to combine the driver’s driving style to build and verify the vehicle’s lane-change prediction model.


## 1.2. Data Description

The NGSIM is a dataset of different sections initiated by the United States Department of Transportation (US DOT) Federal Highway Administration (FHWA) [25]. In the NGSIM, I-80 and US-101 are the datasets collected in highway, which are studied in this paper. As shown in Figure 1, both I-80 and US-101 consist of five main lanes, one distribution lane, one on-ramp, and one off-ramp (the off-ramp of I-80 is not located within the study area). In I-80, the 1650-foot-long study area is divided into seven sub-areas by seven cameras to record the relevant data, while in US-101, the 2100-foot-long study area is divided into eight sub-areas by eight cameras. The dataset contains the trajectory data of all vehicles in the study area during the recorded time period.Figure 1
Study area description of the dataset.
(a)(b)


## 1.3. Data Preprocessing

In order to analyze the characteristics of the lane-change behavior, characteristic parameters such as the coordinates and velocity of the vehicles are extracted from the NGSIM dataset. The coordinates of the ramp and the most adjacent lane have a large overlap, which will cause great interference to the study. Therefore, the data related to the ramp and the most adjacent lane are eliminated in the study. To further analyze the influence of the surrounding lane-change environment on lane-change behavior and the relationship between them, the distance between the vehicle and the front and rear vehicles on the current lane and adjacent lanes is calculated. The lateral speed of the vehicle is also calculated in order to predict the lane-change behavior. The complete data of 92 lane-changing vehicles, a total of 92932 frames, are finally screened out and processed; then a modified dataset is established.In the processed data, the surrounding lane-change environment at the time of a certain vehicle lane-change frame is shown in Figure2. The range of lane coordinates calculated according to the data in the processed dataset is shown in Table 2.Figure 2
Schematic diagram of vehicle lane-changing state frame.Table 2
The range of lane coordinates.
Coordinate (feet)Lane IDDataset1234I-800.21–12.2911.90–24.3823.74–36.1035.63–48.42US-1011.46–13.3311.51–24.4523.41–35.6833.58–46.29Coordinate (feet)Lane IDDataset5678I-8047.83–60.4259.52–82.4872.65–96.75US-10144.99–61.3057.19–69.6057.38–72.6058.04–72.92


## 2. Methodology

### 2.1. Analysis of Lane-Change Characteristics

According to the study of Balal et al. [26], the main characteristics that affect drivers’ lane change are Dft, Dpft, Dpt, Dpc, and Vc. Lane-change behaviors can be divided into left-lane change and right-lane change, so Dft, Dpft, and Dpt can be divided into Dfl, Dfr, Dpfl, Dpfr, Dpl, and Dpr, which were defined in Table 3. The typical lane-change scenario taking the right-lane change as an example can be described in Figure 3.Table 3
Definition of selected parameters.
ParametersDefinitionDflThe distance between the current vehicle and the following vehicle in the left/right laneDfrDpflThe distance between the preceding vehicle and the following vehicle in the left/right laneDpfrDplThe distance between the current vehicle and the preceding vehicle in the left/right laneDprDpcThe distance between the current vehicle and the preceding vehicle in the current laneThe velocity of current vehicleVcFigure 3
The typical lane-change scenario.


### 2.2. Intelligent Prediction Approach

Based on the analysis of lane-change characteristics in real scene dataset, an intelligent prediction approach is proposed and established, in which the HMM-based model is used to judge the lane-change conditions, LSTM-based model is used to predict the current vehicle motion data that are suitable for change lanes, and then the designed lane-change detection algorithm is performed to complete the lane-change behavior prediction.


#### 2.2.1. HMM-Based Lane-Change Environment Judgment

HMM can be used to predict the probability of whether a vehicle changes lanes [27–29]. The vehicle and surrounding vehicles’ driving state determines to a large extent whether the vehicle has the conditions for changing lanes. In this paper, based on the analysis of vehicle lane-change characteristics, eight parameters, Dfl, Dfr, Dpfl, Dpfr, Dpl, Dpr, Dpc, and Vc, are selected as observations to judge the surrounding lane-change environment.As shown in Figure3, in the HMM model, the eight continuous observation values, Dfl, Dfr, Dpfl, Dpfr, Dpl, Dpr, Dpc, and Vc, are used as observation vectors. In order to simplify the model and facilitate implementation in practical applications, the continuous values of the observation vector are divided into eight segments according to the importance of each observation vector’s influence on lane-change behavior. The observation vector can be defined as follows:(1)VT=OM=DftT,DpftT,DptT,DpcT,VcT,where V = [V1, V2, …, VT] is the observation sequence, T is the sequence length, O = [O1, O2, …, OM] is the observation state, M is the distribution of the observable state, which is divided into 8 states according to the value of the observation vector and its importance [26], and M∈{1, 2, 3, 4, 5, 6, 7, 8}. The hidden states are the lane-change behaviors, including nonlane change, left-lane change, and right-lane change, which are represented as H1, H2, and H3, respectively.As shown in Figure4, the parameters of designed model can be defined as follows:(2)λ=A,B,π,where A means the state transition probability matrix, in which aij is the probability of transition to state hj at time T + 1 under the condition that time T is in state hi,(3)A=aij3×3,aij=PiT+1=hj|iT=hi,i=1,2,3;j=1,2,3.Figure 4
Schematic diagram of designed HMM.B represents the observation probability matrix, in which bj(M) is the probability of generating the observation OM under the condition that time T is in state hj:(4)B=bjM3×8,bjM=PVT=OM|iT=hj,M=1,2,...,8;j=1,2,3.π indicates the initial state probability distribution, in which πi is the probability of being in state hi at time t = 1:(5)π=πi,πi=Pi1=hi,i=1,2,3.In this paper, the dataset contains the observation sequence and the corresponding state sequence. Therefore, the supervised learning method can be used to estimate parameters of HMM. The maximum likelihood estimation method is used, and the specific method is as follows:(1)
Estimate the transition probability. Assume that the frequency of the sample at timet in state i and transition to state j at time t + 1 is Aij, then the estimation of state transition probability aij is as follows:(6)aijˆ=Aij∑j=13Aij,i=1,2,3;j=1,2,3.(2)
Estimate the probability of observation. Assume that the frequency of the sample state isj and the observation is M is BjM, then the estimation of the probability bj(M) that the state is j and the observation is M is as follows:(7)bjMˆ=BjM∑M=18BjM,j=1,2,3;M=1,2,...,8.(3)
Estimate the initial state probability. The estimateπiˆ of the initial state probability πi is the frequency at which the initial state is hi in the sample.After the model parameters are determined, using the forward probability and the backward probability, given the modelλ and the observation O, the probability of being in the state hi at time t can be obtained:(8)γt=Pit=hi|O,λ=Pit=hi,O|λPO|λ.From the definition of forward probabilityαti and backward probability βti,(9)αtiβti=Pit=hi,O|λ.Then(10)γti=αtiβtiPO|λ=αtiβti∑j=13αtjβtj.


#### 2.2.2. LSTM-Based Vehicle Trajectory Prediction

After judging the surrounding lane-change environment, the data suitable for lane change would be screened out to predict the lane-change behavior. An LSTM [30] model is designed to predict the current vehicle motion data. The structure of an LSTM block [31] is shown as Figure 5, in which fi is the input activation function, fo is the output activation function, and fg is the gate activation function. At time t, xt is the input, ht is the hidden layer state, it is the output state of input gate, ft is the output state of forget gate, and ot is the output state of output gate, which can be expressed as follows:(11)it=fgwxixt+whiht−1+bi,(12)ft=fgwxfxt+whfht−1+bf,(13)ot=fgwxoxt+whoht−1+bo,where wxi, wxf, and wxo are the input weight matrices; whi, whf, and who who are the feedback weight matrices; and bi, bf, and bo are the bias vectors.Figure 5
The structure of an LSTM block.The intermediate states at timet are as follows: the output state Cint corresponding to the input function, the output state Ct corresponding to the output function, and the output state ht corresponding to the hidden layer.(14)Cint=fiwxcxt+whcht−1+bCin,where wxc, whc, and bCin are the input weight matrix and the corresponding bias vector, respectively. Cint, as the input function, the output state at time t will participate in the overall update of the input state at time t together with the output state it of the input gate at time t. As the output state of the input function at time t, Cint participates in the overall update of the input state at time t together with the output state it of the input gate at time t.At timet, through the new input and state feedback at previous time, the entire LSTM unit is updated, including the update of Ct and ht:(15)Ct=ftCt−1+itCint,(16)ht=otfoCt.In the update process of each gate function and the output state of the entire unit, the key information in the input feature is retained and transferred through the forget gate function and the transfer of the state.


#### 2.2.3. The Lane-Change Behavior Prediction Approach

The predicted data with conditions for lane change are used to determine whether the current vehicle will change lanes through the lane-change detection algorithm, which can be described as shown in Algorithm1. An optimal sampling interval length (δ sampling period) is obtained according to the data training, and then the velocity data on x-axis are used as input according to the obtained sampling interval length. In the process of lane-change detection, first, calculate all zero-crossing points in the input data and sort them by time; then, calculate the lateral velocity integral between all adjacent zero-crossing points; finally, train the calculation results based on the KNN [32, 33] method, and output lane-change detection result (the lane-change behavior category) and the start and end time of lane-change process.Algorithm 1: Lane-change detection algorithm.
Input: The velocity data on x-axis in δ sampling periods, Vx;Output: The predicted lane-change behavior, behavior; The beginning and ending time of the predicted lane-change behavior, t1, t2;(1)
fori = 1 to length do(2)
ifVxi≥0&&Vxi+1<0 or Vxi≤0&&Vxi+1>0then(3)
cj=i;j=j+1;(4)
end if(5)
end for(6)
for all zero crossing value m = 1 to length do(7)
Vm=∫cmcm+1Vxi;(8)
end for(9)
for all training object Vm,behavior∈I and test object z=Vm,∧behavior∧do(10)
Compute distanced=∑i=1nVmi∧−Vmi2 between z and every object Vm, behavior∈I;(11)
SelectIz⊆ I, the set of k closest training objects to z; returnbehavior∧=argvmax∑Vmi,behaviori∈IzFv=behaviori; t1 = c(m), t2 = c(m + 1);(12)
end forThe structure of proposed multimodel fusion lane-change behavior prediction approach can be described as shown in Figure6. First, the surrounding environment parameters related to the lane-change condition are used to judge the current lane-change condition through the HMM-based model. Then, the vehicle trajectory data which meet lane-change conditions are predicted by the LSTM-based model. Finally, the lane-change behavior is predicted by the proposed lane-change detection algorithm; the predicted lane-change behavior and the start and end time of lane-change process are output.Figure 6
The structure of lane-change behavior prediction approach.


## 2.1. Analysis of Lane-Change Characteristics

According to the study of Balal et al. [26], the main characteristics that affect drivers’ lane change are Dft, Dpft, Dpt, Dpc, and Vc. Lane-change behaviors can be divided into left-lane change and right-lane change, so Dft, Dpft, and Dpt can be divided into Dfl, Dfr, Dpfl, Dpfr, Dpl, and Dpr, which were defined in Table 3. The typical lane-change scenario taking the right-lane change as an example can be described in Figure 3.Table 3
Definition of selected parameters.
ParametersDefinitionDflThe distance between the current vehicle and the following vehicle in the left/right laneDfrDpflThe distance between the preceding vehicle and the following vehicle in the left/right laneDpfrDplThe distance between the current vehicle and the preceding vehicle in the left/right laneDprDpcThe distance between the current vehicle and the preceding vehicle in the current laneThe velocity of current vehicleVcFigure 3
The typical lane-change scenario.


## 2.2. Intelligent Prediction Approach

Based on the analysis of lane-change characteristics in real scene dataset, an intelligent prediction approach is proposed and established, in which the HMM-based model is used to judge the lane-change conditions, LSTM-based model is used to predict the current vehicle motion data that are suitable for change lanes, and then the designed lane-change detection algorithm is performed to complete the lane-change behavior prediction.


### 2.2.1. HMM-Based Lane-Change Environment Judgment

HMM can be used to predict the probability of whether a vehicle changes lanes [27–29]. The vehicle and surrounding vehicles’ driving state determines to a large extent whether the vehicle has the conditions for changing lanes. In this paper, based on the analysis of vehicle lane-change characteristics, eight parameters, Dfl, Dfr, Dpfl, Dpfr, Dpl, Dpr, Dpc, and Vc, are selected as observations to judge the surrounding lane-change environment.As shown in Figure3, in the HMM model, the eight continuous observation values, Dfl, Dfr, Dpfl, Dpfr, Dpl, Dpr, Dpc, and Vc, are used as observation vectors. In order to simplify the model and facilitate implementation in practical applications, the continuous values of the observation vector are divided into eight segments according to the importance of each observation vector’s influence on lane-change behavior. The observation vector can be defined as follows:(1)VT=OM=DftT,DpftT,DptT,DpcT,VcT,where V = [V1, V2, …, VT] is the observation sequence, T is the sequence length, O = [O1, O2, …, OM] is the observation state, M is the distribution of the observable state, which is divided into 8 states according to the value of the observation vector and its importance [26], and M∈{1, 2, 3, 4, 5, 6, 7, 8}. The hidden states are the lane-change behaviors, including nonlane change, left-lane change, and right-lane change, which are represented as H1, H2, and H3, respectively.As shown in Figure4, the parameters of designed model can be defined as follows:(2)λ=A,B,π,where A means the state transition probability matrix, in which aij is the probability of transition to state hj at time T + 1 under the condition that time T is in state hi,(3)A=aij3×3,aij=PiT+1=hj|iT=hi,i=1,2,3;j=1,2,3.Figure 4
Schematic diagram of designed HMM.B represents the observation probability matrix, in which bj(M) is the probability of generating the observation OM under the condition that time T is in state hj:(4)B=bjM3×8,bjM=PVT=OM|iT=hj,M=1,2,...,8;j=1,2,3.π indicates the initial state probability distribution, in which πi is the probability of being in state hi at time t = 1:(5)π=πi,πi=Pi1=hi,i=1,2,3.In this paper, the dataset contains the observation sequence and the corresponding state sequence. Therefore, the supervised learning method can be used to estimate parameters of HMM. The maximum likelihood estimation method is used, and the specific method is as follows:(1)
Estimate the transition probability. Assume that the frequency of the sample at timet in state i and transition to state j at time t + 1 is Aij, then the estimation of state transition probability aij is as follows:(6)aijˆ=Aij∑j=13Aij,i=1,2,3;j=1,2,3.(2)
Estimate the probability of observation. Assume that the frequency of the sample state isj and the observation is M is BjM, then the estimation of the probability bj(M) that the state is j and the observation is M is as follows:(7)bjMˆ=BjM∑M=18BjM,j=1,2,3;M=1,2,...,8.(3)
Estimate the initial state probability. The estimateπiˆ of the initial state probability πi is the frequency at which the initial state is hi in the sample.After the model parameters are determined, using the forward probability and the backward probability, given the modelλ and the observation O, the probability of being in the state hi at time t can be obtained:(8)γt=Pit=hi|O,λ=Pit=hi,O|λPO|λ.From the definition of forward probabilityαti and backward probability βti,(9)αtiβti=Pit=hi,O|λ.Then(10)γti=αtiβtiPO|λ=αtiβti∑j=13αtjβtj.


### 2.2.2. LSTM-Based Vehicle Trajectory Prediction

After judging the surrounding lane-change environment, the data suitable for lane change would be screened out to predict the lane-change behavior. An LSTM [30] model is designed to predict the current vehicle motion data. The structure of an LSTM block [31] is shown as Figure 5, in which fi is the input activation function, fo is the output activation function, and fg is the gate activation function. At time t, xt is the input, ht is the hidden layer state, it is the output state of input gate, ft is the output state of forget gate, and ot is the output state of output gate, which can be expressed as follows:(11)it=fgwxixt+whiht−1+bi,(12)ft=fgwxfxt+whfht−1+bf,(13)ot=fgwxoxt+whoht−1+bo,where wxi, wxf, and wxo are the input weight matrices; whi, whf, and who who are the feedback weight matrices; and bi, bf, and bo are the bias vectors.Figure 5
The structure of an LSTM block.The intermediate states at timet are as follows: the output state Cint corresponding to the input function, the output state Ct corresponding to the output function, and the output state ht corresponding to the hidden layer.(14)Cint=fiwxcxt+whcht−1+bCin,where wxc, whc, and bCin are the input weight matrix and the corresponding bias vector, respectively. Cint, as the input function, the output state at time t will participate in the overall update of the input state at time t together with the output state it of the input gate at time t. As the output state of the input function at time t, Cint participates in the overall update of the input state at time t together with the output state it of the input gate at time t.At timet, through the new input and state feedback at previous time, the entire LSTM unit is updated, including the update of Ct and ht:(15)Ct=ftCt−1+itCint,(16)ht=otfoCt.In the update process of each gate function and the output state of the entire unit, the key information in the input feature is retained and transferred through the forget gate function and the transfer of the state.


### 2.2.3. The Lane-Change Behavior Prediction Approach

The predicted data with conditions for lane change are used to determine whether the current vehicle will change lanes through the lane-change detection algorithm, which can be described as shown in Algorithm1. An optimal sampling interval length (δ sampling period) is obtained according to the data training, and then the velocity data on x-axis are used as input according to the obtained sampling interval length. In the process of lane-change detection, first, calculate all zero-crossing points in the input data and sort them by time; then, calculate the lateral velocity integral between all adjacent zero-crossing points; finally, train the calculation results based on the KNN [32, 33] method, and output lane-change detection result (the lane-change behavior category) and the start and end time of lane-change process.Algorithm 1: Lane-change detection algorithm.
Input: The velocity data on x-axis in δ sampling periods, Vx;Output: The predicted lane-change behavior, behavior; The beginning and ending time of the predicted lane-change behavior, t1, t2;(1)
fori = 1 to length do(2)
ifVxi≥0&&Vxi+1<0 or Vxi≤0&&Vxi+1>0then(3)
cj=i;j=j+1;(4)
end if(5)
end for(6)
for all zero crossing value m = 1 to length do(7)
Vm=∫cmcm+1Vxi;(8)
end for(9)
for all training object Vm,behavior∈I and test object z=Vm,∧behavior∧do(10)
Compute distanced=∑i=1nVmi∧−Vmi2 between z and every object Vm, behavior∈I;(11)
SelectIz⊆ I, the set of k closest training objects to z; returnbehavior∧=argvmax∑Vmi,behaviori∈IzFv=behaviori; t1 = c(m), t2 = c(m + 1);(12)
end forThe structure of proposed multimodel fusion lane-change behavior prediction approach can be described as shown in Figure6. First, the surrounding environment parameters related to the lane-change condition are used to judge the current lane-change condition through the HMM-based model. Then, the vehicle trajectory data which meet lane-change conditions are predicted by the LSTM-based model. Finally, the lane-change behavior is predicted by the proposed lane-change detection algorithm; the predicted lane-change behavior and the start and end time of lane-change process are output.Figure 6
The structure of lane-change behavior prediction approach.


## 2.2.1. HMM-Based Lane-Change Environment Judgment

HMM can be used to predict the probability of whether a vehicle changes lanes [27–29]. The vehicle and surrounding vehicles’ driving state determines to a large extent whether the vehicle has the conditions for changing lanes. In this paper, based on the analysis of vehicle lane-change characteristics, eight parameters, Dfl, Dfr, Dpfl, Dpfr, Dpl, Dpr, Dpc, and Vc, are selected as observations to judge the surrounding lane-change environment.As shown in Figure3, in the HMM model, the eight continuous observation values, Dfl, Dfr, Dpfl, Dpfr, Dpl, Dpr, Dpc, and Vc, are used as observation vectors. In order to simplify the model and facilitate implementation in practical applications, the continuous values of the observation vector are divided into eight segments according to the importance of each observation vector’s influence on lane-change behavior. The observation vector can be defined as follows:(1)VT=OM=DftT,DpftT,DptT,DpcT,VcT,where V = [V1, V2, …, VT] is the observation sequence, T is the sequence length, O = [O1, O2, …, OM] is the observation state, M is the distribution of the observable state, which is divided into 8 states according to the value of the observation vector and its importance [26], and M∈{1, 2, 3, 4, 5, 6, 7, 8}. The hidden states are the lane-change behaviors, including nonlane change, left-lane change, and right-lane change, which are represented as H1, H2, and H3, respectively.As shown in Figure4, the parameters of designed model can be defined as follows:(2)λ=A,B,π,where A means the state transition probability matrix, in which aij is the probability of transition to state hj at time T + 1 under the condition that time T is in state hi,(3)A=aij3×3,aij=PiT+1=hj|iT=hi,i=1,2,3;j=1,2,3.Figure 4
Schematic diagram of designed HMM.B represents the observation probability matrix, in which bj(M) is the probability of generating the observation OM under the condition that time T is in state hj:(4)B=bjM3×8,bjM=PVT=OM|iT=hj,M=1,2,...,8;j=1,2,3.π indicates the initial state probability distribution, in which πi is the probability of being in state hi at time t = 1:(5)π=πi,πi=Pi1=hi,i=1,2,3.In this paper, the dataset contains the observation sequence and the corresponding state sequence. Therefore, the supervised learning method can be used to estimate parameters of HMM. The maximum likelihood estimation method is used, and the specific method is as follows:(1)
Estimate the transition probability. Assume that the frequency of the sample at timet in state i and transition to state j at time t + 1 is Aij, then the estimation of state transition probability aij is as follows:(6)aijˆ=Aij∑j=13Aij,i=1,2,3;j=1,2,3.(2)
Estimate the probability of observation. Assume that the frequency of the sample state isj and the observation is M is BjM, then the estimation of the probability bj(M) that the state is j and the observation is M is as follows:(7)bjMˆ=BjM∑M=18BjM,j=1,2,3;M=1,2,...,8.(3)
Estimate the initial state probability. The estimateπiˆ of the initial state probability πi is the frequency at which the initial state is hi in the sample.After the model parameters are determined, using the forward probability and the backward probability, given the modelλ and the observation O, the probability of being in the state hi at time t can be obtained:(8)γt=Pit=hi|O,λ=Pit=hi,O|λPO|λ.From the definition of forward probabilityαti and backward probability βti,(9)αtiβti=Pit=hi,O|λ.Then(10)γti=αtiβtiPO|λ=αtiβti∑j=13αtjβtj.


## 2.2.2. LSTM-Based Vehicle Trajectory Prediction

After judging the surrounding lane-change environment, the data suitable for lane change would be screened out to predict the lane-change behavior. An LSTM [30] model is designed to predict the current vehicle motion data. The structure of an LSTM block [31] is shown as Figure 5, in which fi is the input activation function, fo is the output activation function, and fg is the gate activation function. At time t, xt is the input, ht is the hidden layer state, it is the output state of input gate, ft is the output state of forget gate, and ot is the output state of output gate, which can be expressed as follows:(11)it=fgwxixt+whiht−1+bi,(12)ft=fgwxfxt+whfht−1+bf,(13)ot=fgwxoxt+whoht−1+bo,where wxi, wxf, and wxo are the input weight matrices; whi, whf, and who who are the feedback weight matrices; and bi, bf, and bo are the bias vectors.Figure 5
The structure of an LSTM block.The intermediate states at timet are as follows: the output state Cint corresponding to the input function, the output state Ct corresponding to the output function, and the output state ht corresponding to the hidden layer.(14)Cint=fiwxcxt+whcht−1+bCin,where wxc, whc, and bCin are the input weight matrix and the corresponding bias vector, respectively. Cint, as the input function, the output state at time t will participate in the overall update of the input state at time t together with the output state it of the input gate at time t. As the output state of the input function at time t, Cint participates in the overall update of the input state at time t together with the output state it of the input gate at time t.At timet, through the new input and state feedback at previous time, the entire LSTM unit is updated, including the update of Ct and ht:(15)Ct=ftCt−1+itCint,(16)ht=otfoCt.In the update process of each gate function and the output state of the entire unit, the key information in the input feature is retained and transferred through the forget gate function and the transfer of the state.


## 2.2.3. The Lane-Change Behavior Prediction Approach

The predicted data with conditions for lane change are used to determine whether the current vehicle will change lanes through the lane-change detection algorithm, which can be described as shown in Algorithm1. An optimal sampling interval length (δ sampling period) is obtained according to the data training, and then the velocity data on x-axis are used as input according to the obtained sampling interval length. In the process of lane-change detection, first, calculate all zero-crossing points in the input data and sort them by time; then, calculate the lateral velocity integral between all adjacent zero-crossing points; finally, train the calculation results based on the KNN [32, 33] method, and output lane-change detection result (the lane-change behavior category) and the start and end time of lane-change process.Algorithm 1: Lane-change detection algorithm.
Input: The velocity data on x-axis in δ sampling periods, Vx;Output: The predicted lane-change behavior, behavior; The beginning and ending time of the predicted lane-change behavior, t1, t2;(1)
fori = 1 to length do(2)
ifVxi≥0&&Vxi+1<0 or Vxi≤0&&Vxi+1>0then(3)
cj=i;j=j+1;(4)
end if(5)
end for(6)
for all zero crossing value m = 1 to length do(7)
Vm=∫cmcm+1Vxi;(8)
end for(9)
for all training object Vm,behavior∈I and test object z=Vm,∧behavior∧do(10)
Compute distanced=∑i=1nVmi∧−Vmi2 between z and every object Vm, behavior∈I;(11)
SelectIz⊆ I, the set of k closest training objects to z; returnbehavior∧=argvmax∑Vmi,behaviori∈IzFv=behaviori; t1 = c(m), t2 = c(m + 1);(12)
end forThe structure of proposed multimodel fusion lane-change behavior prediction approach can be described as shown in Figure6. First, the surrounding environment parameters related to the lane-change condition are used to judge the current lane-change condition through the HMM-based model. Then, the vehicle trajectory data which meet lane-change conditions are predicted by the LSTM-based model. Finally, the lane-change behavior is predicted by the proposed lane-change detection algorithm; the predicted lane-change behavior and the start and end time of lane-change process are output.Figure 6
The structure of lane-change behavior prediction approach.


## 3. Results and Analysis

### 3.1. Evaluation Metrics

When evaluating the prediction results, the confusion matrix definition of the prediction results is shown in Table4. Accuracy, precision, recall, and F1 value are usually used as evaluation indicators [34] for learning-based classification and prediction models. Among them, the accuracy represents the proportion of the sample size correctly classified in the total sample size, which can be defined as follows:(17)ACC=TP+TNTP+TN+FN+FP.Table 4
Definition of the prediction confusion matrix.
PredictedTruthPositiveNegativeTrueTPTNFalseFPFNPrecision (P), which indicates the proportion of samples with the correct class label among the samples of a particular class found by the classifier, can be defined as follows:(18)P=TPTP+FP.The recall (R) represents the classifier’s ability to find samples of a certain category, which can be defined as follows:(19)R=TPTP+FN.TheF1 value is a comprehensive index that considers the balance between precision and recall, which can be defined as follows:(20)F1=2×P×RP+R.The closer theF1 value is to 1, the better the effect.


### 3.2. Experimental Results

The proposed prediction approach, including HMM-based lane-change condition judgment, LSTM-based vehicle lane change-related parameter prediction, and lane-change detection algorithm, is trained and tested on the established modified dataset to evaluate the performance.


#### 3.2.1. Performance of HMM-Based Model

In order to verify the judgment performance of the HMM-based model on the lane-change environment, parameters related to lane change, includingDfl, Dfr, Dpfl, Dpfr, Dpl, Dpr, Dpc, and Vc, are processed and then trained and tested. The results show that at all lane-change times, the lane-change probability of target lane is basically above 0.5. At the moment when the vehicle does not change lanes, some lane-change environments meet the lane-change conditions, and some do not. Therefore, the designed model can screen out the moments that do not meet the lane-change conditions and improve the prediction accuracy.The schematic fragment of the designed HMM-based judgment result of lane-change condition is shown in Figure7. In the figure, the data of the green line represent the truth of the lane-change behavior (1 means right-lane change, −1 means left-lane change, 0 means nonlane change), the data of the red line indicate the probability of the right-lane change calculated by the designed model, while the data of the blue line denote the probability of the left-lane change calculated by the designed model. It can be seen that the calculated right-lane-change probability at the time of right-lane change and the left-lane-change probability at the time of left-lane change in the figure are all greater than 0.5, which meets the lane-change conditions.Figure 7
Schematic fragment of the designed HMM-based judgment result of lane-change condition.


#### 3.2.2. Performance of LSTM-Based Model

In order to predict the lane-change behavior of the ICV, the LSTM-based model is designed to predict the lane-change-related motion data (lateral velocity) at the next moment. The dataset is divided into training set and test set at a ratio of 2 to 1 to verify the performance of the designed model. The loss curve of the training process is shown in Figure8, in which the loss value is stable at around 2.32E-05.Figure 8
The loss curve of LSTM.The prediction result of designed LSTM is shown in Figure9, in which the blue line represents original data and yellow line and green line indicate the prediction result of the training set and the test set, respectively. The root mean square error (RMSE) of the prediction is 0.37 m/s for the training set and 0.68 m/s for the test set. From the prediction results in the figure, it can be found that the overall prediction error of the designed model is small, and the prediction error is greater when the data have large and sudden changes than when the data are flat. The maximum prediction error of the dataset is 5.5668 m/s (the original data is 50.0055 m/s).Figure 9
The prediction result of designed LSTM.


#### 3.2.3. Performance of the Detection Algorithm

The designed lane-change behavior detection algorithm was performed on the established dataset to verify its detection effect on lane-change behavior. The dataset is divided into training set and test set at a ratio of 2 to 1, the experimental result of detection is shown in Table5, and the confusion matrix of it is shown in Figure 10.Table 5
Experimental result of detection algorithm.
MetricsCategoryP (%)R (%)F1 (%)Left-lane change10083.0590.74Nonlane change99.5510099.77Right-lane change90.9110095.24Figure 10
The confusion matrix of detection result.It can be seen from the experimental result of detection that theP of left-lane-change detection has reached 100% and R of it is 83.05%, the P of right-lane-change detection is 90.91% and R of it is 100%, while P and R of nonlane change are 99.55% and 100%, respectively. The F1 values of left-lane change, nonlane change, and right-lane change are 90.74%, 99.77%, and 95.24%, respectively.From the confusion matrix of detection result, it can be found that the accuracy of the detection algorithm is 99.56%. Among them, 10 samples in nonlane change are detected as left-lane change, and 1 sample is detected as right-lane change, while no lane-change behavior is detected as nonlane change and there is no error detection between left-lane change and right-lane change, which shows that the proposed detection algorithm could accurately detect lane-change information for the safe lane change of ICV.Taking entire driving process of vehicle 2458 as an example, the detection result is shown in Figure11. It can be seen from the figure that vehicle 2458 made a lane change at t = 600 during the whole process, and its lateral velocity has an obvious acceleration process. The designed detection algorithm accurately detects the lane-change behavior and calculates the lane-change process that is between t1 = 591 and t2 = 641.Figure 11
The detection result of vehicle 2458.


#### 3.2.4. Performance of the Lane-Change Behavior Prediction Approach

Finally, the proposed prediction approach is performed on the established dataset to verify the effect of the approach. The proposed lane-change behavior detection algorithm is performed on the filtered prediction data that meets the lane-change conditions. The dataset is also divided into training set and test set at a ratio of 2 to 1, the experimental result of prediction is shown in Table6, and the confusion matrix of prediction result is shown in Figure 12.Table 6
Experimental result of prediction approach.
MetricsCategoryP (%)R (%)F1 (%)Left-lane change89.3695.4592.31Nonlane change99.7299.5899.65Right-lane change90.0081.8285.71Figure 12
The confusion matrix of prediction result.From the experimental result of prediction, it can be seen thatP of left-lane-change detection is 89.36% and R of it is 95.45%, the P of right-lane-change detection is 90.00% and R of it is 81.82%, while P and R of nonlane change are 99.72% and 99.65%, respectively. The F1 values of left-lane change, nonlane change and right-lane change are 92.31%, 99.65%, and 85.71%, respectively.It can be found from the confusion matrix of prediction result that the accuracy of the prediction approach is 99.32%. Among them, 2 samples in nonlane change are predicted as left-lane change, and 2 samples are predicted as right-lane change. 5 samples in left-lane change are predicted as nonlane change, with a precision of 89.36%, and 1 sample in right-lane change is predicted as nonlane change, with a precision of 90.00%, while there is no error detection between left-lane change and right-lane change. The experimental result shows that the proposed prediction approach could effectively provide vehicle lane-change information to assist the ICV in safe lane change.Taking the entire driving process of vehicle 2458 as an example, the prediction result is shown in Figure13. It can be seen that the predicted lateral velocity and the truth of lateral velocity are basically consistent in value and trend. The designed prediction approach accurately predicts the lane-change behavior and calculates the lane-change process that is between t1 = 590 and t2 = 776. The predicted time interval of the lane-change process is longer than that calculated by the detection algorithm, which is because that when the lateral velocity value fluctuates around 0, the predicted lateral velocity value fluctuates slightly and is less than 0. The prediction approach can still accurately predict the lane-change behavior and the time interval of lane change.Figure 13
The prediction result of vehicle 2458.


## 3.1. Evaluation Metrics

When evaluating the prediction results, the confusion matrix definition of the prediction results is shown in Table4. Accuracy, precision, recall, and F1 value are usually used as evaluation indicators [34] for learning-based classification and prediction models. Among them, the accuracy represents the proportion of the sample size correctly classified in the total sample size, which can be defined as follows:(17)ACC=TP+TNTP+TN+FN+FP.Table 4
Definition of the prediction confusion matrix.
PredictedTruthPositiveNegativeTrueTPTNFalseFPFNPrecision (P), which indicates the proportion of samples with the correct class label among the samples of a particular class found by the classifier, can be defined as follows:(18)P=TPTP+FP.The recall (R) represents the classifier’s ability to find samples of a certain category, which can be defined as follows:(19)R=TPTP+FN.TheF1 value is a comprehensive index that considers the balance between precision and recall, which can be defined as follows:(20)F1=2×P×RP+R.The closer theF1 value is to 1, the better the effect.


## 3.2. Experimental Results

The proposed prediction approach, including HMM-based lane-change condition judgment, LSTM-based vehicle lane change-related parameter prediction, and lane-change detection algorithm, is trained and tested on the established modified dataset to evaluate the performance.


### 3.2.1. Performance of HMM-Based Model

In order to verify the judgment performance of the HMM-based model on the lane-change environment, parameters related to lane change, includingDfl, Dfr, Dpfl, Dpfr, Dpl, Dpr, Dpc, and Vc, are processed and then trained and tested. The results show that at all lane-change times, the lane-change probability of target lane is basically above 0.5. At the moment when the vehicle does not change lanes, some lane-change environments meet the lane-change conditions, and some do not. Therefore, the designed model can screen out the moments that do not meet the lane-change conditions and improve the prediction accuracy.The schematic fragment of the designed HMM-based judgment result of lane-change condition is shown in Figure7. In the figure, the data of the green line represent the truth of the lane-change behavior (1 means right-lane change, −1 means left-lane change, 0 means nonlane change), the data of the red line indicate the probability of the right-lane change calculated by the designed model, while the data of the blue line denote the probability of the left-lane change calculated by the designed model. It can be seen that the calculated right-lane-change probability at the time of right-lane change and the left-lane-change probability at the time of left-lane change in the figure are all greater than 0.5, which meets the lane-change conditions.Figure 7
Schematic fragment of the designed HMM-based judgment result of lane-change condition.


### 3.2.2. Performance of LSTM-Based Model

In order to predict the lane-change behavior of the ICV, the LSTM-based model is designed to predict the lane-change-related motion data (lateral velocity) at the next moment. The dataset is divided into training set and test set at a ratio of 2 to 1 to verify the performance of the designed model. The loss curve of the training process is shown in Figure8, in which the loss value is stable at around 2.32E-05.Figure 8
The loss curve of LSTM.The prediction result of designed LSTM is shown in Figure9, in which the blue line represents original data and yellow line and green line indicate the prediction result of the training set and the test set, respectively. The root mean square error (RMSE) of the prediction is 0.37 m/s for the training set and 0.68 m/s for the test set. From the prediction results in the figure, it can be found that the overall prediction error of the designed model is small, and the prediction error is greater when the data have large and sudden changes than when the data are flat. The maximum prediction error of the dataset is 5.5668 m/s (the original data is 50.0055 m/s).Figure 9
The prediction result of designed LSTM.


### 3.2.3. Performance of the Detection Algorithm

The designed lane-change behavior detection algorithm was performed on the established dataset to verify its detection effect on lane-change behavior. The dataset is divided into training set and test set at a ratio of 2 to 1, the experimental result of detection is shown in Table5, and the confusion matrix of it is shown in Figure 10.Table 5
Experimental result of detection algorithm.
MetricsCategoryP (%)R (%)F1 (%)Left-lane change10083.0590.74Nonlane change99.5510099.77Right-lane change90.9110095.24Figure 10
The confusion matrix of detection result.It can be seen from the experimental result of detection that theP of left-lane-change detection has reached 100% and R of it is 83.05%, the P of right-lane-change detection is 90.91% and R of it is 100%, while P and R of nonlane change are 99.55% and 100%, respectively. The F1 values of left-lane change, nonlane change, and right-lane change are 90.74%, 99.77%, and 95.24%, respectively.From the confusion matrix of detection result, it can be found that the accuracy of the detection algorithm is 99.56%. Among them, 10 samples in nonlane change are detected as left-lane change, and 1 sample is detected as right-lane change, while no lane-change behavior is detected as nonlane change and there is no error detection between left-lane change and right-lane change, which shows that the proposed detection algorithm could accurately detect lane-change information for the safe lane change of ICV.Taking entire driving process of vehicle 2458 as an example, the detection result is shown in Figure11. It can be seen from the figure that vehicle 2458 made a lane change at t = 600 during the whole process, and its lateral velocity has an obvious acceleration process. The designed detection algorithm accurately detects the lane-change behavior and calculates the lane-change process that is between t1 = 591 and t2 = 641.Figure 11
The detection result of vehicle 2458.


### 3.2.4. Performance of the Lane-Change Behavior Prediction Approach

Finally, the proposed prediction approach is performed on the established dataset to verify the effect of the approach. The proposed lane-change behavior detection algorithm is performed on the filtered prediction data that meets the lane-change conditions. The dataset is also divided into training set and test set at a ratio of 2 to 1, the experimental result of prediction is shown in Table6, and the confusion matrix of prediction result is shown in Figure 12.Table 6
Experimental result of prediction approach.
MetricsCategoryP (%)R (%)F1 (%)Left-lane change89.3695.4592.31Nonlane change99.7299.5899.65Right-lane change90.0081.8285.71Figure 12
The confusion matrix of prediction result.From the experimental result of prediction, it can be seen thatP of left-lane-change detection is 89.36% and R of it is 95.45%, the P of right-lane-change detection is 90.00% and R of it is 81.82%, while P and R of nonlane change are 99.72% and 99.65%, respectively. The F1 values of left-lane change, nonlane change and right-lane change are 92.31%, 99.65%, and 85.71%, respectively.It can be found from the confusion matrix of prediction result that the accuracy of the prediction approach is 99.32%. Among them, 2 samples in nonlane change are predicted as left-lane change, and 2 samples are predicted as right-lane change. 5 samples in left-lane change are predicted as nonlane change, with a precision of 89.36%, and 1 sample in right-lane change is predicted as nonlane change, with a precision of 90.00%, while there is no error detection between left-lane change and right-lane change. The experimental result shows that the proposed prediction approach could effectively provide vehicle lane-change information to assist the ICV in safe lane change.Taking the entire driving process of vehicle 2458 as an example, the prediction result is shown in Figure13. It can be seen that the predicted lateral velocity and the truth of lateral velocity are basically consistent in value and trend. The designed prediction approach accurately predicts the lane-change behavior and calculates the lane-change process that is between t1 = 590 and t2 = 776. The predicted time interval of the lane-change process is longer than that calculated by the detection algorithm, which is because that when the lateral velocity value fluctuates around 0, the predicted lateral velocity value fluctuates slightly and is less than 0. The prediction approach can still accurately predict the lane-change behavior and the time interval of lane change.Figure 13
The prediction result of vehicle 2458.


## 3.2.1. Performance of HMM-Based Model

In order to verify the judgment performance of the HMM-based model on the lane-change environment, parameters related to lane change, includingDfl, Dfr, Dpfl, Dpfr, Dpl, Dpr, Dpc, and Vc, are processed and then trained and tested. The results show that at all lane-change times, the lane-change probability of target lane is basically above 0.5. At the moment when the vehicle does not change lanes, some lane-change environments meet the lane-change conditions, and some do not. Therefore, the designed model can screen out the moments that do not meet the lane-change conditions and improve the prediction accuracy.The schematic fragment of the designed HMM-based judgment result of lane-change condition is shown in Figure7. In the figure, the data of the green line represent the truth of the lane-change behavior (1 means right-lane change, −1 means left-lane change, 0 means nonlane change), the data of the red line indicate the probability of the right-lane change calculated by the designed model, while the data of the blue line denote the probability of the left-lane change calculated by the designed model. It can be seen that the calculated right-lane-change probability at the time of right-lane change and the left-lane-change probability at the time of left-lane change in the figure are all greater than 0.5, which meets the lane-change conditions.Figure 7
Schematic fragment of the designed HMM-based judgment result of lane-change condition.


## 3.2.2. Performance of LSTM-Based Model

In order to predict the lane-change behavior of the ICV, the LSTM-based model is designed to predict the lane-change-related motion data (lateral velocity) at the next moment. The dataset is divided into training set and test set at a ratio of 2 to 1 to verify the performance of the designed model. The loss curve of the training process is shown in Figure8, in which the loss value is stable at around 2.32E-05.Figure 8
The loss curve of LSTM.The prediction result of designed LSTM is shown in Figure9, in which the blue line represents original data and yellow line and green line indicate the prediction result of the training set and the test set, respectively. The root mean square error (RMSE) of the prediction is 0.37 m/s for the training set and 0.68 m/s for the test set. From the prediction results in the figure, it can be found that the overall prediction error of the designed model is small, and the prediction error is greater when the data have large and sudden changes than when the data are flat. The maximum prediction error of the dataset is 5.5668 m/s (the original data is 50.0055 m/s).Figure 9
The prediction result of designed LSTM.


## 3.2.3. Performance of the Detection Algorithm

The designed lane-change behavior detection algorithm was performed on the established dataset to verify its detection effect on lane-change behavior. The dataset is divided into training set and test set at a ratio of 2 to 1, the experimental result of detection is shown in Table5, and the confusion matrix of it is shown in Figure 10.Table 5
Experimental result of detection algorithm.
MetricsCategoryP (%)R (%)F1 (%)Left-lane change10083.0590.74Nonlane change99.5510099.77Right-lane change90.9110095.24Figure 10
The confusion matrix of detection result.It can be seen from the experimental result of detection that theP of left-lane-change detection has reached 100% and R of it is 83.05%, the P of right-lane-change detection is 90.91% and R of it is 100%, while P and R of nonlane change are 99.55% and 100%, respectively. The F1 values of left-lane change, nonlane change, and right-lane change are 90.74%, 99.77%, and 95.24%, respectively.From the confusion matrix of detection result, it can be found that the accuracy of the detection algorithm is 99.56%. Among them, 10 samples in nonlane change are detected as left-lane change, and 1 sample is detected as right-lane change, while no lane-change behavior is detected as nonlane change and there is no error detection between left-lane change and right-lane change, which shows that the proposed detection algorithm could accurately detect lane-change information for the safe lane change of ICV.Taking entire driving process of vehicle 2458 as an example, the detection result is shown in Figure11. It can be seen from the figure that vehicle 2458 made a lane change at t = 600 during the whole process, and its lateral velocity has an obvious acceleration process. The designed detection algorithm accurately detects the lane-change behavior and calculates the lane-change process that is between t1 = 591 and t2 = 641.Figure 11
The detection result of vehicle 2458.


## 3.2.4. Performance of the Lane-Change Behavior Prediction Approach

Finally, the proposed prediction approach is performed on the established dataset to verify the effect of the approach. The proposed lane-change behavior detection algorithm is performed on the filtered prediction data that meets the lane-change conditions. The dataset is also divided into training set and test set at a ratio of 2 to 1, the experimental result of prediction is shown in Table6, and the confusion matrix of prediction result is shown in Figure 12.Table 6
Experimental result of prediction approach.
MetricsCategoryP (%)R (%)F1 (%)Left-lane change89.3695.4592.31Nonlane change99.7299.5899.65Right-lane change90.0081.8285.71Figure 12
The confusion matrix of prediction result.From the experimental result of prediction, it can be seen thatP of left-lane-change detection is 89.36% and R of it is 95.45%, the P of right-lane-change detection is 90.00% and R of it is 81.82%, while P and R of nonlane change are 99.72% and 99.65%, respectively. The F1 values of left-lane change, nonlane change and right-lane change are 92.31%, 99.65%, and 85.71%, respectively.It can be found from the confusion matrix of prediction result that the accuracy of the prediction approach is 99.32%. Among them, 2 samples in nonlane change are predicted as left-lane change, and 2 samples are predicted as right-lane change. 5 samples in left-lane change are predicted as nonlane change, with a precision of 89.36%, and 1 sample in right-lane change is predicted as nonlane change, with a precision of 90.00%, while there is no error detection between left-lane change and right-lane change. The experimental result shows that the proposed prediction approach could effectively provide vehicle lane-change information to assist the ICV in safe lane change.Taking the entire driving process of vehicle 2458 as an example, the prediction result is shown in Figure13. It can be seen that the predicted lateral velocity and the truth of lateral velocity are basically consistent in value and trend. The designed prediction approach accurately predicts the lane-change behavior and calculates the lane-change process that is between t1 = 590 and t2 = 776. The predicted time interval of the lane-change process is longer than that calculated by the detection algorithm, which is because that when the lateral velocity value fluctuates around 0, the predicted lateral velocity value fluctuates slightly and is less than 0. The prediction approach can still accurately predict the lane-change behavior and the time interval of lane change.Figure 13
The prediction result of vehicle 2458.


## 4. Conclusions

The paper proposed a novel intelligent approach to lane-change behavior prediction for ICVs, which combines the surrounding lane-change environment and the vehicle’s own motion parameters. A modified dataset is established based on the NGSIM dataset, and then the proposed approach is trained and tested. From the experimental results, the HMM-based model can make relatively accurate judgments on the lane-change environment, and its calculated lane-change probability at the time of lane change is above 0.5. The prediction RMSE of the vehicle lateral speed by the LSTM-based model is 0.37 m/s for the training set and 0.68 m/s for the test set. The proposed lane-change detection algorithm has an accuracy of 99.56% on the established dataset and can accurately calculate the time interval of the vehicle lane change. On the basis of the fusion of the above models and algorithm, the proposed intelligent prediction approach is completed, result shows that the accuracy of the prediction approach on the established dataset is 99.32%, and the time interval of the vehicle lane change can be calculated accurately. The experimental result indicates that the proposed prediction approach could effectively provide vehicle lane-change information to assist the ICV in safe lane-change and has the potentials for application in actual intelligent and connected environment for ICVs.Since the proposed approach is postprocessing with measured data, its real-time performance in practical applications needs to be further verified. In future work, the proposed approach can be deployed on mobile terminals for real-time testing, and its accuracy and real-time performance can be further improved.



---

*Source: 9516218-2022-01-17.xml*