# A Multiobjective Cooperative Driving Framework Based on Evolutionary Algorithm and Multitask Learning

Authors: Xia Jiang; Jian Zhang; Qing-yang Li; Tian-yi Chen
Journal: Journal of Advanced Transportation
 (2022)

---

## Abstract

The development of connected and automated vehicle (CAV) techniques brings an upcoming revolution to traffic management. The control of CAVs in potential conflict areas such as on-ramps and intersections will be complex to traffic management when considering their deployment. There is still a lack of a general framework for dispatching CAVs in these bottlenecks, which is expected to ensure safety, traffic efficiency, and energy consumption in real time. This study aimed to fill the technique gap, and a comprehensive cooperative intelligent driving framework is put forward to study the problem, which can be used in both on-ramp and intersection scenarios. Based on a multi-objective evolutionary algorithm, CAVs are denoted as a sequence to be searched in solution space, while a multitask learning neural network with adaptive loss function is implemented for optimization target feedback to surrogate the simulation test procedure. The simulation results show that the proposed framework can get satisfying performance with low time and energy consumption. It can reduce time consumption by up to 16.51% for the on-ramp scenario and 9.8% for the intersection scenario, while reducing energy consumption by up to 16.39% and 11.39% for the two scenarios. Meanwhile, an analysis of computation time is carried out, illuminating the flexibility and controllability of the new strategy.


---

## Body

## 1. Introduction

Connected and automated vehicles are considered to play an important role in improving traffic efficiency and saving energy [1]. The fickle driving behaviors can easily lead to a series of problems, including traffic congestion, energy consumption, and accident [2–4], but transport systems consisting of intelligent vehicles can make a difference using vehicle-to-everything (V2X) communication and advanced control techniques [5–7].The development of connected and automated vehicles (CAVs) brings both opportunities and challenges to traffic management. As the bottlenecks in traffic organization, intersection and on-ramp become the research hot spots in the domain [8–10]. Conventionally, the vehicles must adhere to the traffic signals in urban scenarios, and corresponding studies are proposed to optimize the trajectory of vehicles in this case [1, 11, 12]. Considering the traffic environment composed of CAVs, traffic signals can be eliminated because the information on the road can be fully obtained [13], while the vehicles on the road can be fully controlled. It is possible to implement cooperative control for CAVs through V2X communication [14]. Thus, the design of a cooperative driving strategy through the use of real-time traffic information becomes particularly important. Ann and Colombo [15] pointed out that an effective cooperative driving framework can work in different traffic scenarios such as intersections, merging roadways, and roundabouts. On account of the significance of cooperative driving, the researchers proposed many theoretical methods to solve the problem for different scenarios. Grand cooperative driving challenges were also organized to promote its development in practice [16].Some of the existing studies belong to the optimization-based method. Yan et al. [17] proposed a dynamic programming algorithm to evacuate vehicles at the intersection as soon as possible. Zhu and Ukkusuri [18] put forward a linear programming model to dispatch vehicles at autonomous intersections in order to minimize total travel time. Besides, mixed-integer linear programming (MILP) is widely used to obtain solutions [19–21]. However, Li and Wang [13] proposed a framework based on the optimization principle, which utilized a tree search algorithm to achieve the same purpose. All of the listed studies focus on searching for optimal solutions based on different prior hypotheses.Relevant studies pointed out that the key to solving the problem is determining the right-of-way for CAVs approaching the merging area [22–24]. In other words, the vehicles can be formulated as a passing sequence in the form of arrays, and the performance of the schedule strategy hinges on the way to generate the best passing order among a large number of possible solutions.In terms of generating passing orders, the existing studies can be classified into two categories. One is the rule-based strategy, which uses some heuristic rules to determine the passing order of vehicles. Dresner and Stone [25, 26] proposed a reservation-based system and assigned right-of-way to vehicles on a first-come-first-served (FCFS) basis. Although the effectiveness of the FCFS method can be proved [26, 27], its rule-based nature always leads to feasible but not optimal solutions. Moreover, the reservation-based strategy cannot outperform traditional signal control in some cases [28, 29]. While the rule-based strategies cannot always perform very well, the other approach to generate passing orders is introduced, called “planning-based strategy” [13]. Meng et al. demonstrated that the planning-based strategy could consistently outperform the FCFS method in intersection scenarios by comprehensive simulations [30]. Actually, the planning-based strategy is a framework that can search for optimal solutions in a huge solution space. The strategy is essentially a traversal problem with intolerable computational complexity. Therefore, consequent studies focused on the reduction in computing time. Xu et al. [31] proposed a grouping-based strategy, which groups CAVs to reduce the count of possible solutions. In their other study, a Monte Carlo tree is built to keep the trade-off between coordination performance and computation time [32]. Meanwhile, Zhang et al. [33] reported a framework that utilized a neural network to surrogate the simulation test process with the intent to reduce computation time. However, the only optimization target they considered is about traffic efficiency indexes such as passing time or total delay, while the value of other targets such as energy consumption or queue length is difficult to acquire. This is caused by the weakness of the trajectory interpretation algorithm in their studies.Therefore, there is still a lack of a real-time, multi-objective cooperative driving strategy that can be maneuverable and reliable. To this end, we design a multi-objective discrete evolutionary algorithm (MODEA) to search for (near) optimal passing orders, which combines the non-dominated sorting method [34] and state transition algorithm [35]. A multitask learning model is proposed to be a regressor, which can give feedback of objective values to MODEA. The scenario is simulated by Simulation of Urban MObility (SUMO) [36]. The simulation results indicate that the framework can be applied to different scenarios, performing well even under a high concurrency environment.The rest of the study is arranged as follows: Section “Problem Statement” gives the general form of cooperative driving problems and traffic scenarios the paper studied. Section “Methodology” presents the framework we proposed, including the MODEA and multitask learning method in detail. Section “Simulation and Analysis” provides the simulation results of a series of experiments. Finally, conclusions are given in Section “Conclusion.”


## 2. Problem Statement

Highway on-ramps and urban unsignalized intersections are two typical scenarios for cooperative driving (see Figure1). Rios-Torres and Malikopoulos [37] pointed out that the two mainstream frameworks in the cooperative driving field are centralized coordination and decentralized coordination, respectively, while the method proposed in this study belongs to the former. The centralized frameworks rely on a central controller responsible for computing and sending control commands. The controller has a communication range (CR) that defines the boundary of communication and control. This article denotes the CR as a circle, which is widely adopted in previous studies [30, 33, 38]. Only vehicles within the CR will communicate with the controller and be controlled.Figure 1
Two typical cooperative driving scenarios in general road networks. (a) The on-ramp in highway and (b) the unsignalized intersection in urban road.Some followed assumptions are listed to make the analysis and implementation easier:(i)
Lane-changing behaviors are prohibited in CR for safety consideration.(ii)
The system has no interference from pedestrians and non-motor vehicles.(iii)
All CAVs can transmit id, position, speed, and other precise information to the controller spontaneously.(iv)
The vehicles are homogeneous pure electric CAVs for estimating energy consumption. The energy model can be found in [39].The general form of the objective function in cooperative driving can be defined as follows:(1)minxFx,where F is the function that represents queue length, energy consumption, or traffic delay, and x is the independent variable that will give rise to the optimization target. In this study, two objects are considered: (a) the minimization of time consumption to evacuate all CAVs in CR and (b) electricity consumption for CAVs in the process of a scheduling scheme.The input of the functionx denotes a passing sequence, which can be denoted as follows:(2)x=CAV1,CAV2,…,CAVn,where n is the number of vehicles in CR. Let f1 be the time consumption to evacuate all CAVs in CR, and f2 be the corresponding electricity consumption, and (1) can be transformed to as follows:(3)minxf1x,f2x,Here, f1x can be denoted as follows:(4)f1x=teCAVn,where te represents the time when the vehicle CAVn exits from CR. f2x can be denoted as follows:(5)f2x=∑i=1n∫0teCAVieCAVi,i∈1,n,where e represents the energy consumption of CAVi in discrete time, and readers can refer to [39] for the stepwise energy consumption model.


## 3. Methodology

Figure2 illustrates the procedure of the framework this study proposed. The framework uses MODEA with non-dominated sorting and multitask neural network to reduce computation time and implement multi-objective optimization. A population-based evolutionary algorithm is used to search solutions in solution space, while the fitness value of every individual can be obtained from a neural network, which plays the role of target regressor. Then, the framework will be introduced in detail.Figure 2
Flowchart of the proposed framework.


### 3.1. Multitask Learning Model

It is found that carrying out learning for tasks jointly can improve the performance compared with conducting them individually [40]. Thus, in this study, a multitask deep learning model is trained to target the evolutionary algorithm’s feedback. Therefore, the task of the model is learning for target yield in each traffic state. Here, we consider the time consumption and electricity consumption as the targets defined in equations (4) and (5).For performing the regression task, the input should be appropriately expressed. As in equation (2), a passing sequence can be denoted as an array including CAV ids. We define the encoding of a single CAV as follows:(6)Ei=pi,vi,ai,encodelanei,where pi is the position of CAVi from the beginning of the lane, and vi is the speed, while ai represents the acceleration of CAVi, and pi,vi,ai will be normalized for input into the model.In addition,encodelanei is the encoding of the lane that the driving vehicle belongs to. The encoding method is different according to the different traffic scenarios. For the on-ramp scenario shown in Figure 1(a), the one-hot encoding is applied. However, in the intersection scenario, considering the spatial relationship, we combine with approach direction and driving direction. Figure 3 shows the encoding process that takes the scenarios in Figure 1 as an example. For instance, vehicle D is coming from the west approach, and it will turn left at the intersection, so the encoding of its lane is 1,0,0,0,0,1. Finally, a passing sequence can be formulated as the concatenation of encodings of CAVs.Figure 3
Encoding of lane of vehicles, which takes the state in Figure1 as an example. (a) Example of Figure 1(a) and (b) example of Figure 1(b).
(a)(b)When the vectorized representations of passing sequences are constructed, a neural network model can be built to take the vectors as input. Similar to TextCNN [41], we also use the convolutional neural network (CNN) to carry out the learning process, whereas CNN can extract the features from original data automatically [42]. The structure of the CNN-based multitask learning model is shown in Figure 4. The backbone part takes sequence vectors consisting of several CAV encodings as inputs and extracts latent feature representations for them; then, the specific task part takes the feature representations as input and output time consumption and energy consumption of the sequences in a specific traffic scenario. In the backbone part, one-dimensional convolution layers with different scales of kernel size are applied to extract features.Figure 4
Structure of the proposed neural network. The backbone layer is for feature extracting, and the specific task layer is for different regression tasks.After determining the basic structure of the neural network, the loss function should be specified to train the learning model towards the optimization goals. Here, considering the training process of two targets in two single-task models, the loss functions are considered as mean squared error (MSE), which is as follows:(7)MSE1=1n∑i=1nf^1xi−f1xi2,MSE2=1n∑i=1nf^2xi−f2xi2,where n is the count of test samples. f^1 and f^2 are predicting values, while f1 and f2 are ground truth. Generally, the loss function in the multitask learning model can be defined as the naive weighted sum of losses, which is as follows:(8)L=ω1MSE1+ω2MSE2,where the loss weights ω1 and ω2 are uniform or manually tuned. The performance of the model highly depends on the settings of the weight parameters. Cipolla et al. pointed out that the loss function can be calculated based on maximizing the Gaussian likelihood with homoscedastic uncertainty [43]. As a result, let fWx be the outputs of neural network with weights W, and the likelihood as a Gaussian can be defined as follows:(9)pfjx|fWx=ΝfWx,σ2j=1,2,where σ is a scalar that represents observation noise. Let f^jx be the sufficient statistics; then, the multitask likelihood can be derived from the following:(10)pf1x,f2x|fWx=pf1x|fWx⋅pf2x|fWx=Νf1x;fWx,σ12Νf2x;fWx,σ22.Taking logarithmic form, the new loss function can be defined as follows:(11)−logpf1x,f2x|fWx∝12σ12f1x−fWx2+12σ22f2x−fWx2+logσ1σ2=12σ12MSE1+12σ22MSE2+logσ1σ2.Notice thatσ1 and σ2 are the denominators in equation (11). To avoid division by zero errors, the logarithmic form is used for the actual training process:(12)ςj=logσj2j=1,2.Finally, the loss function is given in equation (13), which can be adaptive during the training process.(13)L=12e−ς1MSE1+e−ς2MSE2+ς1+ς2.


## 3.1. Multitask Learning Model

It is found that carrying out learning for tasks jointly can improve the performance compared with conducting them individually [40]. Thus, in this study, a multitask deep learning model is trained to target the evolutionary algorithm’s feedback. Therefore, the task of the model is learning for target yield in each traffic state. Here, we consider the time consumption and electricity consumption as the targets defined in equations (4) and (5).For performing the regression task, the input should be appropriately expressed. As in equation (2), a passing sequence can be denoted as an array including CAV ids. We define the encoding of a single CAV as follows:(6)Ei=pi,vi,ai,encodelanei,where pi is the position of CAVi from the beginning of the lane, and vi is the speed, while ai represents the acceleration of CAVi, and pi,vi,ai will be normalized for input into the model.In addition,encodelanei is the encoding of the lane that the driving vehicle belongs to. The encoding method is different according to the different traffic scenarios. For the on-ramp scenario shown in Figure 1(a), the one-hot encoding is applied. However, in the intersection scenario, considering the spatial relationship, we combine with approach direction and driving direction. Figure 3 shows the encoding process that takes the scenarios in Figure 1 as an example. For instance, vehicle D is coming from the west approach, and it will turn left at the intersection, so the encoding of its lane is 1,0,0,0,0,1. Finally, a passing sequence can be formulated as the concatenation of encodings of CAVs.Figure 3
Encoding of lane of vehicles, which takes the state in Figure1 as an example. (a) Example of Figure 1(a) and (b) example of Figure 1(b).
(a)(b)When the vectorized representations of passing sequences are constructed, a neural network model can be built to take the vectors as input. Similar to TextCNN [41], we also use the convolutional neural network (CNN) to carry out the learning process, whereas CNN can extract the features from original data automatically [42]. The structure of the CNN-based multitask learning model is shown in Figure 4. The backbone part takes sequence vectors consisting of several CAV encodings as inputs and extracts latent feature representations for them; then, the specific task part takes the feature representations as input and output time consumption and energy consumption of the sequences in a specific traffic scenario. In the backbone part, one-dimensional convolution layers with different scales of kernel size are applied to extract features.Figure 4
Structure of the proposed neural network. The backbone layer is for feature extracting, and the specific task layer is for different regression tasks.After determining the basic structure of the neural network, the loss function should be specified to train the learning model towards the optimization goals. Here, considering the training process of two targets in two single-task models, the loss functions are considered as mean squared error (MSE), which is as follows:(7)MSE1=1n∑i=1nf^1xi−f1xi2,MSE2=1n∑i=1nf^2xi−f2xi2,where n is the count of test samples. f^1 and f^2 are predicting values, while f1 and f2 are ground truth. Generally, the loss function in the multitask learning model can be defined as the naive weighted sum of losses, which is as follows:(8)L=ω1MSE1+ω2MSE2,where the loss weights ω1 and ω2 are uniform or manually tuned. The performance of the model highly depends on the settings of the weight parameters. Cipolla et al. pointed out that the loss function can be calculated based on maximizing the Gaussian likelihood with homoscedastic uncertainty [43]. As a result, let fWx be the outputs of neural network with weights W, and the likelihood as a Gaussian can be defined as follows:(9)pfjx|fWx=ΝfWx,σ2j=1,2,where σ is a scalar that represents observation noise. Let f^jx be the sufficient statistics; then, the multitask likelihood can be derived from the following:(10)pf1x,f2x|fWx=pf1x|fWx⋅pf2x|fWx=Νf1x;fWx,σ12Νf2x;fWx,σ22.Taking logarithmic form, the new loss function can be defined as follows:(11)−logpf1x,f2x|fWx∝12σ12f1x−fWx2+12σ22f2x−fWx2+logσ1σ2=12σ12MSE1+12σ22MSE2+logσ1σ2.Notice thatσ1 and σ2 are the denominators in equation (11). To avoid division by zero errors, the logarithmic form is used for the actual training process:(12)ςj=logσj2j=1,2.Finally, the loss function is given in equation (13), which can be adaptive during the training process.(13)L=12e−ς1MSE1+e−ς2MSE2+ς1+ς2.


## 4. Multi-Objective Discrete Evolutionary Algorithm

Generally, the average count of possible passing sequences in cooperative driving grows almost exponentially with the increase in numbers of CAVs in CR [30]. Thus, searching for the best solution is hard when the number of CAVs is large, so this study proposes a population-based evolutionary algorithm to obtain (near) optimal passing order from this perspective.In multi-objective optimization problems, the Pareto optimal solution is used to select according to the practical problem [44]. The conception of the Pareto optimal solution set is introduced as below. First, in this minimization problem, solution x0 Pareto dominates x1 only if:(14)fjx0≤fjx1,∀j=1,2fjx0<fjx1,∃j=1,2.We use the corresponding symbol to denote the domination relationship:(15)x0≻x1,which represents that x0 dominates x1. If there is not any solution that dominates x0, then x0 will be called the non-dominated solution. Accordingly, the Pareto optimal solution set Ps can be defined as the set consisting of all the non-dominated solutions. Therefore, the primary purpose of the algorithm is to search corresponding Pareto optimal solutions. If there is more than one element in the Pareto optimal solution set, two kinds of heuristic strategies can be used:(i)
Delay-first strategy (DFS): always choose the solution with minimal time consumption fromPs.(ii)
Energy-first strategy (EFS): always choose the solution with minimal energy consumption fromPs.The form of the candidate solutions in the algorithm is denoted as equation (2), while the initialization operation is generating n different integers with ranges from 1 to n. The feasible solutions make up a population in the evolutionary algorithm. Considering that lane-changing behavior is prohibited in CR, some solutions will be illegal. For example, in Figure 1(a), the passing order C,A,B cannot be accepted as candidate solution because A is supposed to be in front of C. Hence, a repair operation is applied to repair illegal sequence, which is defined as follows:(16)x˙=Mrx,where x˙ represents a passing sequence that can be a candidate, and Mr is a matrix that carries out the repair operation. The matrix is constructed according to the order of vehicles on the lanes. For unfeasible sequence 3,1,2, which represents “C-A-B” in Figure 1(a), Mr is as follows:(17)Mr=010100001.Then,3,1,2 will be transformed to 1,3,2, which represents “A-C-B,” and it will be legal.The proposed algorithm uses selection operation, crossover operation, state transition with swap operation, shift operation, and symmetry operation for population evolution. Corresponding operations can be described as follows.


### 4.1. Selection Operation

Non-dominated sorting technique is used for layering individuals. Algorithm1 shows the process of non-dominated sorting. In the algorithm, c is the non-dominated level, and XN is the set of all the non-dominated solutions in P; fitness represents the virtual value of individuals, which is used for selection operation. Eventually, the roulette wheel method is applied to choose individuals in the population; then, the crossover operation can be carried out. In the roulette wheel method, the selection probability of individual i is defined as follows:(18)Pri=cf−fitnessi∑j=1cf−1j,where cf is the value of c after iterations in Algorithm 1.Algorithm 1: Non-dominated sorting.
Input: Init population P(1)
:c⟵1(2)
: while length (P) > 0(3)
:B⟵XN(4)
:for each b∈B(5)
:     fitnessb⟵c(6)
:   end(7)
:c⟵c+1(8)
:   DeleteB from P(9)
: end


### 4.2. Crossover Operation

Tie-breaking crossover is introduced in this study [45]. This operation can prevent two identical orders from appearing in a sequence, and the procedure is indicated in Figure 5. The start positions and length of subsequences are generated randomly, so the results after crossover could be with duplicated items. A crossover map will also be generated, and the crossover map is actually a random order of integers 0,1,…,n−1. Accordingly, the new sequences after exchange can be transformed by multiplying the length of the sequence and adding the crossover map. Finally, as shown in Figure 5, offspring can be produced by sorting operation according to phase 3.Figure 5
Procedure of tie-breaking crossover.


### 4.3. State Transition

The state transition procedure is probabilistic in the light of predefined probability valuep. In this study, the value of p is set to 0.2 to keep the trade-off between exploration and exploitation. The state transition operations include swap, shift, and symmetry [35]. Swap transformation is used for randomly exchanging subsequences in passing sequences; shift transformation is used for subsequence translation, and symmetry transformation means two subsequences symmetrical about a selected central point exchange their values. These operations can be implemented by several matrixes, which can be denoted as follows:(19)xk+1=MsymmetryMshiftMswapxk,where xk+1 is a passing sequence after k iterations. Msymmetry, Mshift, and Mswap represent the matrix, which implements symmetry operation, shift operation, and swap operation, respectively. Figure 6 illustrates the three transformations. The length of subsequences is a hyperparameter for swap transformation and shift transformation. The values of these two operations are generated randomly according to the number of CAVs. While for symmetry operation, the length of subsequences and the position of the symmetry center can be generated randomly. Note that the boundary condition will be processed here when the indexes of elements may be out of bounds.Figure 6
Sketch of swap operation, shift operation, and symmetry operation. The length of subsequences in this figure is 1.


## 4.1. Selection Operation

Non-dominated sorting technique is used for layering individuals. Algorithm1 shows the process of non-dominated sorting. In the algorithm, c is the non-dominated level, and XN is the set of all the non-dominated solutions in P; fitness represents the virtual value of individuals, which is used for selection operation. Eventually, the roulette wheel method is applied to choose individuals in the population; then, the crossover operation can be carried out. In the roulette wheel method, the selection probability of individual i is defined as follows:(18)Pri=cf−fitnessi∑j=1cf−1j,where cf is the value of c after iterations in Algorithm 1.Algorithm 1: Non-dominated sorting.
Input: Init population P(1)
:c⟵1(2)
: while length (P) > 0(3)
:B⟵XN(4)
:for each b∈B(5)
:     fitnessb⟵c(6)
:   end(7)
:c⟵c+1(8)
:   DeleteB from P(9)
: end


## 4.2. Crossover Operation

Tie-breaking crossover is introduced in this study [45]. This operation can prevent two identical orders from appearing in a sequence, and the procedure is indicated in Figure 5. The start positions and length of subsequences are generated randomly, so the results after crossover could be with duplicated items. A crossover map will also be generated, and the crossover map is actually a random order of integers 0,1,…,n−1. Accordingly, the new sequences after exchange can be transformed by multiplying the length of the sequence and adding the crossover map. Finally, as shown in Figure 5, offspring can be produced by sorting operation according to phase 3.Figure 5
Procedure of tie-breaking crossover.


## 4.3. State Transition

The state transition procedure is probabilistic in the light of predefined probability valuep. In this study, the value of p is set to 0.2 to keep the trade-off between exploration and exploitation. The state transition operations include swap, shift, and symmetry [35]. Swap transformation is used for randomly exchanging subsequences in passing sequences; shift transformation is used for subsequence translation, and symmetry transformation means two subsequences symmetrical about a selected central point exchange their values. These operations can be implemented by several matrixes, which can be denoted as follows:(19)xk+1=MsymmetryMshiftMswapxk,where xk+1 is a passing sequence after k iterations. Msymmetry, Mshift, and Mswap represent the matrix, which implements symmetry operation, shift operation, and swap operation, respectively. Figure 6 illustrates the three transformations. The length of subsequences is a hyperparameter for swap transformation and shift transformation. The values of these two operations are generated randomly according to the number of CAVs. While for symmetry operation, the length of subsequences and the position of the symmetry center can be generated randomly. Note that the boundary condition will be processed here when the indexes of elements may be out of bounds.Figure 6
Sketch of swap operation, shift operation, and symmetry operation. The length of subsequences in this figure is 1.


## 5. Vehicle Control

When a passing order is determined, CAVs can move in the light of the sequence. First of all, the motion of vehicles needs to be constrained by the speed limit and acceleration ability:(20)0≤vi≤vmax,dmax≤ai≤amax,where vmax denotes the maximum speed limit on the road, and dmax is the maximum deceleration, while amax is the maximum acceleration constraint by vehicle dynamics.The virtual vehicle mapping method is used in the framework to ensure safety [46, 47]. Taking the case in Figure 1(a) as an example, if the passing order is “A-C-B,” then C will be mapped into lane1−1. CAV B will then follow a virtual vehicle mapped by CAV C, which means the mode of motion of CAVs will be divided into two cases: free driving and car following, respectively. The control process of the CAVs in sequences can be given by Algorithm 2. Accordingly, Conflict is a function to judge whether there are potential conflicts between xi and xj. Carfollowing is a function to guide vehicle xi to follow vehicle xk. The equation of Carfollowing can be denoted as follows:(21)min∫t=t0t1pk−pi−Δsdt,where t0 is the start time and t1 is the time when xi arrives at the conflict zone or stop line. In addition, Δs is the value of the safe gap between two consecutive CAVs. The gap here represents the distance from the front of the following vehicle to the rear of the leading vehicle. If xk is a real CAV, the value is set to Δs¨, and if xk is a virtual vehicle, a correction factor should be added to it, which can be denoted as follows:(22)Δs=Δs¨+b×LiM,where b is a bool variable, if xk is virtual, the value of b will be 1, and LM will be the distance for xi to cross the conflict zone.Algorithm 2: Simple sequence control.
Input: The passing sequencex(1)
: fori∈1:lengthx(2)
:   ifi=1(3)
:ai=vi−vmax&&amax(4)
else(5)
:k⟵1,rp1=p1−pj−Δs(6)
:     forj∈2:i(7)
:rpj=pi−pj−Δs(8)
:       ifConflictxi,xj&& rpj<rp1(9)
:k⟵j(10)
:       end(11)
:Carfollowingxi,xk(12)
:     end(13)
:   end(14)
: endUsing Algorithm2, the first CAV in sequence drives freely, and a CAV with a minimal relative distance with the first CAV in the rest of the sequence is chosen as car following target.Finally, if a passing sequence is determined, it will not be altered unless the set of CAVs in CR changes.


## 6. Simulation and Analysis

### 6.1. Simulation Platform and CNN Training

This study uses the microscopic traffic simulation software SUMO to study the cooperative driving strategy in two traffic scenarios in Figure1. Under the premise of comprehensive consideration of reality, the simulation settings are given in Table 1. The simulation step is set to 0.2s for smoother time-continuous control. The radius of CR in the on-ramp scenario is set to 1000m by considering the communication capability [38]. Meanwhile, we set the radius parameter to 200m in the urban intersection scenario because the speed of vehicles in this case is slow, while 200m is enough for vehicle braking.Table 1
Parameter configuration in simulation.
ItemValueUnitGeneric settingsMaximum acceleration of CAVs2.6m/s2Maximum deceleration of CAVs4.5m/s2Length of single lane1000.0mSize of single simulation step0.2sSafe gapΔs¨2.5mFor on-ramp scenarioRoad speed limit120.0km/hRadius of CR1000.0mFor intersection scenarioRoad speed limit60.0km/hRadius of CR200.0mFirst of all, more than 50000 records were collected in SUMO for each traffic scenario to serve as the training data. The records include encoding of passing sequence and the combination of two regression targets. We use message-digest algorithm 5 (MD5) to delete duplicated data to ensure the uniqueness of the records. Because the length of CAV encoding in the two scenarios is 5 and 9, respectively, the convolution kernel sizes are set to2,3,4 and 2,5,7 to extract different scales of features. The Adam optimizer is used to optimize the weights and biases for the network, and a step decay schedule for learning rate is implemented in the training process for better performance. Accordingly, the rest of the hyperparameters (e.g., batch size, the initial learning rate, and the scales of dense layers) were tuned automatically by applying tree-structured Parzen estimator (TPE), which can search significantly better results compared with random search methods [48].


### 6.2. Simulation Results

To evaluate the proposed strategy comprehensively, we carried out two kinds of simulations based on the pre-trained CNN model. One is adiscrete simulation, which is used for observing the performance of the framework under different static numbers of vehicles to be scheduled. The other is a continuous simulation, which is served to evaluate the framework in different traffic demand levels using the trace data exported from SUMO.We choose the FCFS strategy as a baseline, whereas it is generally used in the domain. The iteration step and population size in MODEA are set to 30 and 40, respectively. We generate different numbers of CAVs distributed in lanes randomly for the two scenarios, and the results of the discrete simulation are shown in Figure7. Obviously, the proposed method always has a better performance than the FCFS method. While in the on-ramp scenario, the gap between the two methods becomes more significant with the increase in CAVs. Thus, the capability of global optimization of MODEA can be verified, while the rule-based FCFS method is regarded as weak to get satisfying solutions. Meanwhile, when there is more than one solution in the Pareto front, the final sequence can be chosen manually according to specific requirements.Figure 7
Performance of the proposed framework compared with FCFS strategy under different CAV number circumstances. (a) For on-ramp scenario and (b) for intersection scenario.
(a)(b)As for continuous simulation, different arrival rates of CAVs are deployed for 2000 simulation steps, and the trace data are exported per 4 times steps. The trace datasets include the information of CAVs such as position, speed, and acceleration, and then, we reload these data in SUMO and carry out simulations. In other words, the same trace data are used for result comparison so that the randomness can be eliminated.All results presented are averaged over 10 independent runs, when the best results are shown in bold in Table2. According to Table 2, there is no significant difference between DFS and EFS, which may be caused by the regression error of the neural network. However, with the increase in CAV arrival rate, the difference in results between FCFS and the proposed framework gets more remarkable. It demonstrates that the MODEA can optimize the two objectives jointly.Table 2
Simulation result comparison.
FCFSMODEA-DFSMODEA-EFSOptimization Rate (%)Time cost (s)Energy cost (Wh)Time cost (s)Energy cost (Wh)Time cost (s)Energy cost (Wh)TimeEnergyOn-ramp scenario400veh/lane⋅h71.5±9.2302.9±88.071.1±9.1302.2±88.871.1±9.2298.8±86.80.56%1.35%600veh/lane⋅h75.4±7.7488.8±132.474.6±7.9479.0±129.575.0±7.6475.3±128.01.06%2.76%800veh/lane⋅h80.5±6.8679.8±172.879.0±7.0658.9±168.779.9±7.2653.0±167.21.86%3.94%1000veh/lane⋅h90.1±7.9933.5±239.282.8±6.6833.3±220.083.2±6.9831.7±210.58.10%10.91%1200veh/lane⋅h106.6±10.51276.3±324.189.0±8.41070.9±277.690.0±8.11067.1±274.916.51%16.39%Intersection scenario400veh/lane⋅h41.4±12.7140.2±49.541.0±12.9137.8±48.341.7±12.8140.0±49.40.10%1.71%600veh/lane⋅h58.6±15.6293.5±150.556.7±14.4287.3±141.357.1±14.5288.9±142.93.24%2.11%800veh/lane⋅h106.5±44.2951.5±661.398.2±37.4861.9±574.098.6±38.5861.4±572.17.79%9.47%1000veh/lane⋅h141.8±60.91468.5±829.0127.9±35.91323.5±639.4130.4±38.51301.3±610.29.80%11.39%1200veh/lane⋅h197.4±44.92572.5±812.1182.3±42.92384.1±780.9186.7±57.82352.6±779.87.65%8.55%


## 6.1. Simulation Platform and CNN Training

This study uses the microscopic traffic simulation software SUMO to study the cooperative driving strategy in two traffic scenarios in Figure1. Under the premise of comprehensive consideration of reality, the simulation settings are given in Table 1. The simulation step is set to 0.2s for smoother time-continuous control. The radius of CR in the on-ramp scenario is set to 1000m by considering the communication capability [38]. Meanwhile, we set the radius parameter to 200m in the urban intersection scenario because the speed of vehicles in this case is slow, while 200m is enough for vehicle braking.Table 1
Parameter configuration in simulation.
ItemValueUnitGeneric settingsMaximum acceleration of CAVs2.6m/s2Maximum deceleration of CAVs4.5m/s2Length of single lane1000.0mSize of single simulation step0.2sSafe gapΔs¨2.5mFor on-ramp scenarioRoad speed limit120.0km/hRadius of CR1000.0mFor intersection scenarioRoad speed limit60.0km/hRadius of CR200.0mFirst of all, more than 50000 records were collected in SUMO for each traffic scenario to serve as the training data. The records include encoding of passing sequence and the combination of two regression targets. We use message-digest algorithm 5 (MD5) to delete duplicated data to ensure the uniqueness of the records. Because the length of CAV encoding in the two scenarios is 5 and 9, respectively, the convolution kernel sizes are set to2,3,4 and 2,5,7 to extract different scales of features. The Adam optimizer is used to optimize the weights and biases for the network, and a step decay schedule for learning rate is implemented in the training process for better performance. Accordingly, the rest of the hyperparameters (e.g., batch size, the initial learning rate, and the scales of dense layers) were tuned automatically by applying tree-structured Parzen estimator (TPE), which can search significantly better results compared with random search methods [48].


## 6.2. Simulation Results

To evaluate the proposed strategy comprehensively, we carried out two kinds of simulations based on the pre-trained CNN model. One is adiscrete simulation, which is used for observing the performance of the framework under different static numbers of vehicles to be scheduled. The other is a continuous simulation, which is served to evaluate the framework in different traffic demand levels using the trace data exported from SUMO.We choose the FCFS strategy as a baseline, whereas it is generally used in the domain. The iteration step and population size in MODEA are set to 30 and 40, respectively. We generate different numbers of CAVs distributed in lanes randomly for the two scenarios, and the results of the discrete simulation are shown in Figure7. Obviously, the proposed method always has a better performance than the FCFS method. While in the on-ramp scenario, the gap between the two methods becomes more significant with the increase in CAVs. Thus, the capability of global optimization of MODEA can be verified, while the rule-based FCFS method is regarded as weak to get satisfying solutions. Meanwhile, when there is more than one solution in the Pareto front, the final sequence can be chosen manually according to specific requirements.Figure 7
Performance of the proposed framework compared with FCFS strategy under different CAV number circumstances. (a) For on-ramp scenario and (b) for intersection scenario.
(a)(b)As for continuous simulation, different arrival rates of CAVs are deployed for 2000 simulation steps, and the trace data are exported per 4 times steps. The trace datasets include the information of CAVs such as position, speed, and acceleration, and then, we reload these data in SUMO and carry out simulations. In other words, the same trace data are used for result comparison so that the randomness can be eliminated.All results presented are averaged over 10 independent runs, when the best results are shown in bold in Table2. According to Table 2, there is no significant difference between DFS and EFS, which may be caused by the regression error of the neural network. However, with the increase in CAV arrival rate, the difference in results between FCFS and the proposed framework gets more remarkable. It demonstrates that the MODEA can optimize the two objectives jointly.Table 2
Simulation result comparison.
FCFSMODEA-DFSMODEA-EFSOptimization Rate (%)Time cost (s)Energy cost (Wh)Time cost (s)Energy cost (Wh)Time cost (s)Energy cost (Wh)TimeEnergyOn-ramp scenario400veh/lane⋅h71.5±9.2302.9±88.071.1±9.1302.2±88.871.1±9.2298.8±86.80.56%1.35%600veh/lane⋅h75.4±7.7488.8±132.474.6±7.9479.0±129.575.0±7.6475.3±128.01.06%2.76%800veh/lane⋅h80.5±6.8679.8±172.879.0±7.0658.9±168.779.9±7.2653.0±167.21.86%3.94%1000veh/lane⋅h90.1±7.9933.5±239.282.8±6.6833.3±220.083.2±6.9831.7±210.58.10%10.91%1200veh/lane⋅h106.6±10.51276.3±324.189.0±8.41070.9±277.690.0±8.11067.1±274.916.51%16.39%Intersection scenario400veh/lane⋅h41.4±12.7140.2±49.541.0±12.9137.8±48.341.7±12.8140.0±49.40.10%1.71%600veh/lane⋅h58.6±15.6293.5±150.556.7±14.4287.3±141.357.1±14.5288.9±142.93.24%2.11%800veh/lane⋅h106.5±44.2951.5±661.398.2±37.4861.9±574.098.6±38.5861.4±572.17.79%9.47%1000veh/lane⋅h141.8±60.91468.5±829.0127.9±35.91323.5±639.4130.4±38.51301.3±610.29.80%11.39%1200veh/lane⋅h197.4±44.92572.5±812.1182.3±42.92384.1±780.9186.7±57.82352.6±779.87.65%8.55%


## 7. Discussion about Computation Time

In cooperative driving tasks, the computation time of algorithms is vital to ensure safety and efficiency. We focus on the time performance of the proposed framework in this part, and we only consider the on-ramp scenario for evaluating computation time because the time complexity of the algorithm in the two scenarios is equal. All experiments were conducted using Julia programming language on Windows 10 operating system with Intel CORE i7-10750H CPU. Meanwhile, BenchmarkTools.jl package is used to precisely evaluate the computation time performance [49].As Figure8 shows, the computation time of the proposed method mainly depends on the population size of MODEA, while the number of CAVs in CR has little effect on the computation complexity, which means that we can control the computation time flexibly by setting the population size of the algorithm manually.Figure 8
Average computation time of the proposed framework based on the different number of CAVs and population size in MODEA.Meanwhile, the influence of computation time on the traffic system should be discussed. First, safety is always the most primary goal to be achieved. The impact of computing time on safety considerations will be reflected in the safe gapΔs¨. The Δs¨ can be roughly revised with the time consumption td:(23)Δs'=Δs¨+td×vmax,where vmax is used for ensuring safety under any circumstance, so that Δs will be changed in simulations in terms of equation (22).Then, we carry out a series of simulations using the same trace data exported from SUMO to compare the performance of the control framework under different computation delays. In the test, the delay caused by computation varies from 0.1 s to 0.4 s, while DFS is chosen to get solutions. Figures9(a) and 9(b) show the time consumption and energy consumption under different circumstances. On average, the FCFS rule will outperform the proposed framework in the time consumption aspect when the computation delay reaches 0.3 s, and it will have almost identical performance in the electricity consumption aspect when the computation delay reaches 0.4 s.Figure 9
Comparison for values of objective function under different computation delays. (a) Time consumption and (b) electricity consumption.
(a)(b)


## 8. Conclusions

Over the last few years, many methods have been put forward in the cooperative driving field, but the controllability of optimization objectives and the efficiency of algorithms are still difficult to deal with. Based on the combination of evolutionary algorithm and machine learning technique, this study proposes an intelligent framework that considers both the delay and the energy consumption of vehicles. An encoding approach of CAVs is implemented, and a passing sequence of CAVs is approximately regarded as a sentence in natural language so that the TextCNN can be applied to extract features. Compared with other frameworks, it has some significant advantages:(i)
Controllability and flexibility: the optimization objectives and computation time can be adjusted manually, and it can be instrumental under different design requirements.(ii)
General applicability: similar to FCFS protocol, the framework can be applied in different cooperative driving scenarios such as intersection and on-ramps.In future research, a more concrete vehicle control method is supposed to be studied for practicability. Moreover, the neural network this study implements can only deal with a finite number of cases because the input length for the network is fixed. Therefore, the maximum number of CAVs must be assigned, and the zero paddings will be used if the number of CAVs is less than the predefined maximum length. Hence, the form of the neural network and CAV encodings can be further studied for better performance; for example, the encoder-decoder structure can be applied to study the cases of different numbers of CAVs. Finally, the lane-changing behavior of vehicles and pedestrian crossing rules can be considered in the system. However, a more complex but more realistic system will be put before us to study.



---

*Source: 6653598-2022-01-21.xml*