# Intelligent Research Based on Deep Learning Recognition Method in Vehicle-Road Cooperative Information Interaction System

Authors: Hongbin Jiao
Journal: Computational Intelligence and Neuroscience
 (2022)

---

## Abstract

The vehicle-road collaborative information interaction system is an emerging technology system that realizes the sharing of information between vehicles, vehicles and roads between traffic road information, and driving vehicle information. It is of positive significance for improving the urban transportation construction system and promoting urban economic development. This paper conducts intelligent research on the deep learning recognition method based on the vehicle-road collaborative information interaction system. First, this article comprehensively expounds the concept of the vehicle-road collaborative information interaction system and then introduces the specific components, functions, and applications of the system structure. Then, this article researches on deep learning recognition methods and introduces three deep learning recognition methods. They are background extraction method, YOLOv2 method, and DeepSORT method. Finally, this paper conducts simulation comparison experiments between deep learning algorithms and traditional algorithms. It evaluates the feasibility of the algorithm in the vehicle-road collaborative information interaction system in three aspects: vehicle target detection, vehicle flow identification, and emergency decision-making. The experimental results show that the value of the intersection ratio of vehicle target detection in the deep learning recognition method is 8.66% higher than that of the traditional algorithm, the recall rate is 7% higher than that of the traditional algorithm, and the vehicle flow recognition accuracy is 1.8% higher than that of the traditional algorithm. The early warning time in emergency decision-making is also shorter than that of traditional algorithms, which shows the unique superiority and feasibility of deep learning algorithms in the vehicle-road collaborative information interaction system.


---

## Body

## 1. Introduction

With the rapid improvement of the level of social and economic development, the popularity of urbanization is getting faster and faster, and the scope is getting wider and wider. At the same time, the problems of increasing urban population and lagging road construction continue to aggravate the contradiction between supply and demand of transportation in major cities. Frequent traffic accidents caused by traffic congestion and public environmental pollution not only hinder the construction of urban transportation, but also cause huge loss of life and property of the people, and directly have a negative impact on the development of the city’s economy. Taking reasonable measures to solve the contradiction between the increasing traffic demand and the imperfect road construction has become the primary task of modern urban development. However, the current traditional methods have greater limitations and passivity, and it is obvious that they cannot effectively solve the existing urban traffic problems.The change of social form has brought about the continuous progress of science and technology, and the intelligent transportation technology has brought new hope for the existing urban traffic problems. Relying on science and technology, an effective and safe vehicle-road collaborative information interaction system is established, which realizes the communication between vehicles and between vehicles and between vehicles and roads under the condition that vehicles and roads are coordinated. The deep learning recognition method based on the vehicle-road collaborative information interaction system can not only detect system functions, but also provide feedback on system performance. It can also use feedback information to further optimize the system, and intelligent research on it is conducive to the development and progress of the vehicle-road collaborative information interaction system. It has extremely important value and significance for reducing the probability of road traffic accidents, improving traffic efficiency, and promoting the sustainable development of urban transportation systems.In recent years, many scholars have focused on the research of vehicle-road collaborative information interaction systems. Duarte has developed a software tool to simulate and study Vehicle-Road Cooperative Information Interaction (VRI). He quantifies the energy released and absorbed by vehicles on the road in different sports scenes through road decelerators or specific energy harvesters. Software tools are designed to overcome the limitations of capability analysis and accurately quantify energy transfer. By evaluating different vehicle models and VRI models, it is found that the accuracy of the bicycle model is 60% higher than that of the quarter car model, and the accuracy of the contact surface analysis model is 67% higher. The software tools he developed have higher accuracy than existing tools in performing energy analysis and road deceleration applications [1]. Zhang et al. studied the information exchange protocol of the security-related services of the Vehicle-Road Cooperative Infrastructure System (CVIS) and optimized it from three aspects. They proposed an adaptive backoff algorithm that selects the appropriate competition window by considering the number of retransmissions and the busyness of the network. They established a mathematical analysis model to verify its performance improvement and finally used a network simulation tool to simulate different scene models of the vehicle ad hoc network (VANET). They studied the impact of different access methods on the quality of service (QoS). The simulation results verify that the improvement of the proposed algorithm is obvious, and the RTS/CTS access method can sacrifice a small delay to greatly increase the packet loss rate when there are many vehicle nodes [2]. Zhang et al. believe that with the advent of the era of big data, the application of vehicle-to-road technology can realize real-time information sharing between vehicles, traffic management departments, and enterprises. They designed a vehicle-road collaborative information interaction algorithm, which overcomes the problems of urban traffic and actual road network in traditional traffic and makes the delivery of goods faster. The goal of the path selection algorithm is to minimize the total cost and remove the larger V/C value during path selection. Finally, they showed through experiments that this algorithm solves the vehicle routing problem (VRP) under the soft time window constraint of multiple parking lots based on vehicle sharing and leasing and reduces the distribution cost from 17.64% to 14.85% [3]. Zhong and Yang created a bridge-vehicle model based on vehicle-road coordination to study the interaction response of the subsided bridge and the vehicle. They performed numerical simulations using Newmark’s β method. They studied the effects of settlement patterns, vehicle speed, road surface roughness, and boundary conditions. Theoretical and numerical results show that the settlement of the foundation has a significant impact on the influencing factors of the bridge at high speeds, and the road surface roughness may interact with the settlement to produce a coupling effect, which verifies the correctness and accuracy of the model [4]. Inturri utilizes vehicle-road collaborative information and communication technology (ICT) to provide transportation solutions ranging from flexible transportation to ride-sharing services. It provides real-time “on-demand” mobility through a fleet shared by different passengers, and it uses an agent-based model (ABM) fed by GIS data to explore different system configurations for specific types of DRST services (i.e., flexible transportation). And it estimates that the service’s feasible transportation demand and supply variables minimize the total unit cost index. This technology takes into account passenger travel time and vehicle operating costs. It provides useful suggestions for the correct planning, management, and optimization of DRST services by reproducing the microinteractions between demand and supply agents (i.e., passengers and vehicles) [5]. Reza uses sensor data to achieve collaborative information interaction between vehicles and roads to maintain a safe driving distance and prevent accidents, reducing the occurrence of road accidents. Sensor technology in connected cars can also improve the overall driving experience by using vehicle-to-infrastructure (V2I) interaction. This enables the vehicle to receive warnings from the roadside unit network and forward warning messages and information about availability. Finally, he proved through experiments that this type of vehicle-road interactive information is particularly beneficial to users in remote areas, who cannot obtain reliable information from traditional communication channels [6–8].This article is based on the intelligent research of the deep learning recognition method in the vehicle-road collaborative information interaction system. This paper studies the characteristics of the various elements of the road traffic system under vehicle-road coordination and provides a certain theoretical basis for the safe driving and walking of vehicles and pedestrians under vehicle-road coordination. It uses a deep learning-based target recognition method and never has a camera with a common field of view to collaboratively perceive vehicle information on the road. The mentioned technologies and methods can improve the safety of automobiles (connected cars) and pedestrians. It can also be applied to a road management platform to automatically and comprehensively obtain vehicle and pedestrian information for each road section, providing a basis for effective traffic guidance. The collaborative perception method of vehicles can also be applied to high-definition video bayonet, which provides an idea for the automatic identification and trajectory tracking of specific vehicles.


## 2. Vehicle-Road Collaborative Information Interaction System and Deep Recognition Method

### 2.1. Overview of Vehicle-Road Collaborative Information Interaction System

The vehicle-road collaboration system is a technical system that integrates a number of emerging sciences and technologies (information analysis and processing, satellite navigation, sensing technology, communication technology, artificial intelligence, and other technologies) to obtain road and vehicle information in real time [9–11]. The goal of the system is to achieve accurate and real-time information interaction between on-board equipment and roadside equipment, achieving a full range of network connections between the internal equipment of the car, between the car and the car, between the car and the road, between the car and the person, between the car and the cloud, and between the road and the cloud. It makes the more disorderly operation of the road system become more efficient and orderly and at the same time realizes road collaborative management and active vehicle safety control. It fully realizes the coordinated control between vehicles and roads [12, 13]. It provides all traffic participants with accurate and reliable traffic assistance information, realizing the full coordination of people, vehicles, and roads to form a safe, efficient, and environmentally friendly road traffic system, as shown in Figure 1.Figure 1
Schematic diagram of information interaction of the vehicle-road collaboration system.


#### 2.1.1. Vehicle-Road Cooperative System Structure

The vehicle-road collaboration system mainly includes intelligent roadside systems, intelligent vehicle systems, signal equipment, and traffic information processing and management systems. The system structure is shown in Figure2.Figure 2
Vehicle-road cooperative system structure.The vehicle-road collaboration system is divided into three parts: the vehicle-mounted system, the cloud service system, and the drive test system. The vehicle on-board system mainly performs calculations and decisions on vehicle safety, such as vehicle emergency avoidance warning, emergency braking, and other operations [14–16]. The cloud service system has a high level of computing and is used to process autonomous driving applications that require timeliness and moderate calculations, such as blind zone warning and green wave speed guidance. The coverage and computing capacity of the drive test system is the largest among the three systems. The structure is shown in Table 1. It is mainly used to deal with the calculation content that is not sensitive to time delay, such as road path planning, macro traffic guidance, global high-precision map management, and so on. In fact, the real-time intelligent optimization of road traffic flow at the macro level is now. The entire vehicle-road collaboration system needs to receive data information through multiple channels. It includes information about the vehicle’s own state (brake, steering, pressure, and temperature), driving environment (obstacles, spacing), road surface (road surface, geometric conditions), traffic flow (vehicle volume, vehicle speed, and occupancy rate), and other information. It reuses navigation and positioning technology and limited or wireless communication technology to send these information data to the cloud server. The cloud server comprehensively processes these information data to form effective information feedback and publish the information feedback. This coherent system operation can also be summarized as the interaction between roadside system information and vehicle-mounted system information and between vehicle-mounted system information and vehicle-mounted system information.Table 1
Structural composition of drive test system.
Component equipmentThe main functionDrive test communication unitCombine the perception information generated by the perception unit for a more comprehensive road information perceptionTraffic management unitSensor unitObtain road and vehicle information and realize different information processingDrive test sensing unit


#### 2.1.2. Application and Function of Vehicle-Road Coordination System

(1)
At present, the vehicle-road coordination system can be used for traffic safety assurance, traffic planning and decision-making, and environmental protection. Traffic safety guarantee is the most important basic function of the vehicle-road coordination system. When the vehicle brakes or turns due to unforeseen emergencies or accidents, the vehicle-road coordination system will use its own communication technology to enable surrounding vehicles to obtain early warning information related to the accident so that the driver can respond in time and avoid the occurrence of dangerous traffic accidents. Compared with smart vehicles equipped with advanced sensing equipment, drivers take longer to respond to emergencies and accidents. Smart vehicles without communication technology cannot respond to emergencies and accidents within the blind zone when the driver is driving the vehicle and make emergency braking or turning on the vehicle [17–19]. A smart car with communication technology can obtain content that it cannot perceive through the information interaction between the roadside system and the on-board system and between the on-board system and the on-board system. It greatly reduces the occurrence of vehicle safety accidents and guarantees traffic safety.(2)
The intelligent vehicle-road collaboration system has more accurate road traffic planning and decision-making capabilities. Because the in-vehicle system has the ability to communicate with the roadside system, it can obtain and share the dynamic information (vehicle position, vehicle speed, and acceleration) of the vehicles driving on each section of the road and the environmental information collected by the roadside facilities. With richer perception information, a decision-making plan that is conducive to the optimal vehicle operation in the overall situation is calculated. It enables vehicles to travel in formation safely under a smaller distance between vehicles, increasing the average running speed and reducing the running time. When a road accident occurs, vehicles driven on other roads can obtain early warning information, and at the same time, they will continuously broadcast their own relevant information. It is beneficial to realize multivehicle coordinated lane change in a very short time, provide driving assistance information for surrounding vehicles, and cooperate with road condition information provided by roadside equipment to avoid traffic jams and improve traffic efficiency.(3)
The vehicle-road coordination system helps reduce energy consumption. The urban traffic management department obtains the traffic operation information of each urban road with the assistance of the vehicle-road coordination system. As a result, traffic can be cleared and directed more effectively, and energy loss caused by road congestion can be reduced [20]. Intelligent vehicles need to rely on high-performance on-board computers for road condition information collection, analysis, and decision-making. The main computing modules are installed in the car, and the on-board computer takes up a certain amount of space and has a large power consumption. Most of the calculation modules of the vehicle-road coordination system can be transferred to the roadside system. This can greatly save vehicle space, reducing vehicle weight, energy consumption, and exhaust emissions. It has great positive significance in environmental protection.


### 2.2. Deep Learning Recognition Methods

Deep learning recognition is one of the most effective perception methods in the vehicle-road collaboration system. It can obtain more comprehensive and accurate vehicle information under different angles and different fields of view according to the deep learning recognition method and provide the most intuitive and reliable information judgment for the vehicle-road collaborative system. At the same time, it also provides relevant basis for the system’s reasoning decision and execution [21]. It conducts research on the deep learning recognition method in the intelligent vehicle-road collaboration system and understands its ability to detect and recognize vehicle targets in video images, as well as its ability to count vehicle traffic.


#### 2.2.1. Deep Learning Recognition Algorithm Based on Background Extraction

Vehicle-road collaboration collects video through the vision system to obtain road pedestrian and vehicle movement information. The detection methods of pedestrians and vehicles in this environment most often use background extraction algorithms; the background extraction algorithm extracts objects with relatively small changes in pixel values from the video, that is, the background. This algorithm mainly finds the background value of each point in the image based on the video image sequence, and the commonly used background extraction algorithms are subdivided into background difference method and interframe difference method.Background Difference Method. The background difference method is a general method for motion segmentation of still scenes. It uses the difference operation between the acquired image frame and the background image to obtain the grayscale image of the target moving area. Thresholding the gray image to extract the moving area, in order to avoid the influence of environmental lighting changes, the background image needs to be updated according to the currently acquired image frame [22]. The process is shown in Figure 3.Figure 3
Background difference method process.UseFx,y to represent the current frame, Ax,y to represent the currently acquired background frame, the difference between the current frame and the background frame to get the difference image Dx,y; then there is [23](1)Dx,y=Fx,y−Ax,y.UseT to represent the threshold of image binarization, and perform a binarization operation on the difference image to extract the area higher than the threshold T, thereby extracting the target moving on the road; that is, [24](2)Mx,y=Fx,y,Dx,y≥T,0,Dx,y<T.Among them,Mx,y represents the movement goal.The algorithm of the background difference method is simple, but it will also be disturbed by the scene, such as the shaking of branches, and the construction and update of the background must not include moving targets. The quality of background modeling directly affects the performance of the algorithm.Interframe Difference Method. The interframe difference method is to subtract the pixel values of two adjacent frames or two images separated by a few frames in the video stream and threshold the subtracted image to extract the motion area in the image. The process of the interframe difference method is shown in Figure 4.Figure 4
Interframe difference method process.The images of the adjacentjth and j+1th frames are represented as Fjx,y and Fj+1x,y, the binarization threshold in the differential image is T, and the differential image is represented as Dx,y; then there are [25](3)Dx,y=1,Fj+1x,y−Fjx,y>T,0,Fj+1x,y−Fjx,y≤T.The interframe difference method does not need to consider the construction of the background model, the algorithm is simple, fast, real-time, and insensitive to the overall lighting changes in the environment. However, the interframe difference method is easy to detect the edges of moving objects. When the difference between the adjacent frames of the moving object is not large, the extracted moving object will produce holes, which is not good for the recognition of the target. The use of interframe difference method is also easy to produce ghost regions; that is, when a stationary object starts to move, the area when the object is stationary is detected as motion after using the interframe difference method. For the same reason, when the moving object starts to enter the static state, it is easy to appear this kind of area that is wrongly detected as moving.


#### 2.2.2. Recognition Algorithm Based on Deep Learning

YOLO algorithm is called You only look once in full; that is, it only needs to perform convolutional neural network calculations under a unified framework to achieve end-to-end real-time target location prediction and recognition. The YOLO algorithm is divided into YOLOv1, YOLOv2, and YOLOv3 according to the time it was launched. The performance varies between different versions. YOLOv2 has made relevant improvements on the basis of YOLOv1, which effectively improves the recognition type, accuracy, recognition speed, and positioning accuracy. The accuracy of YOLOv3 has a certain improvement compared with YOLOv2. The network adopts the class ResNet structure, the network structure is more complicated, and the real-time performance has been slightly reduced. Therefore, the use of network YOLOv2 for target detection is more suitable for the urban road traffic environment with higher real-time requirements, and the YOLOv2 network divides the input image into S×S grids. Each grid predicts the confidence of B bounding boxes and bounding boxes. The bounding box information includes the center coordinates of the bounding box, the width and height values, and the confidence score [26]. The confidence information includes the possibility of the target existence of the bounding box and the accuracy of the position of the bounding box, which is defined as(4)Confidence=Probjece∗IOUpredtruth.Among them,Probjece∈0,1, if the target is in the grid, then Probjece=1. On the contrary, Probjece=0. The intersection ratio IOUpredtruth is used to describe the accuracy of the bounding box position, and the ratio of the bounding box intersection and union of the predicted position A of the detection target and the actual position B; namely,(5)IOUpredtruth=SA∩BSA∪B.If the predicted position completely coincides with the actual position, thenIOU=1.Therefore, the final output dimension of theYOLOv2 network is S∗S∗B∗5+C, where C is the number of target categories. For example, in a target detection for urban road traffic, the number of C is 5 (pedestrians, cars, buses, trucks, and bicycles).Same asYOLOv1, YOLOv2 uses the mean square error as the loss function. The loss function is composed of three parts: positioning error, confidence error, and classification error. The relationship between the loss function and the three parts is as follows [27]:(6)loss=Ecoord+EIOU+Eclass.The positioning error is the deviation of the position and size of the predicted bounding box from the actual; that is, there is the following relationship:(7)Ecoord=λcoord∑i=0S2∑j=0BIijobjxi−xi^2+yi−yi^2+λcoord∑i=0S2∑j=0BIijobjwi−wi^2+hi−hi^22.Among them,xi, yi, wi, and hi represent the predicted value of the model to the target, xi^, yi^, wi^, and hi^ represent the actual value of the target, and λcoord represents the weight value of the coordinate error, which is generally 0.5. Iijobj indicates that there is a target in the ith grid, and the jth box in the grid is responsible for the prediction of the target.For the confidence error, there is the following relationship:(8)EIOU=∑i=0S2∑j=0BIijobjCi−Ci^2+λno obj∑i=0S2∑j=oBIijno objCi−Ci^2.Among them, the weightλno obj generally takes the value 0.5, and the values of Ci and Ci^ have the following relationship [28]:(9)Ci=IOUpredtruth,Ci^=1,Iiobj=1,0,Iino obj=1.If the object is detected, the classification error of each grid is the square error of the conditional probability of each type; namely,(10)Eclass=∑i=0S2Iiobj∑c∈classpic−pi^c.Among them,pic and pi^c, respectively, represent the predicted category probability and the actual category probability in the grid.In terms of network structure,YOLOv2 uses the Darknet 19 classification model. It draws on the VGG network, uses more 3×3 convolution kernels, and compresses the features through the 1×1 convolution kernel. Darknet 19 contains nineteen convolutional layers and five layers Maxpool. The network parameters of Darknet 19 are shown in Table 2.Table 2
Convolutional layer network parameters.
TypeNumber of filtersFilter size/step sizeOutputConvolutional layer323 × 3224 × 224MaxPool2 × 2/2112 × 112Convolutional layer643 × 3112 × 112MaxPool2 × 2/256 × 56Convolutional layer1283 × 356 × 56Convolutional layer641 × 156 × 56Convolutional layer1283 × 356 × 56MaxPool2 × 2/228 × 28Convolutional layer2563 × 328 × 28Convolutional layer1281 × 128 × 28Convolutional layer2563 × 328 × 28MaxPool2 × 2/214 × 14Convolutional layer5123 × 314 × 14Convolutional layer2561 × 114 × 14Convolutional layer5123 × 314 × 14Convolutional layer2561 × 114 × 14Convolutional layer5123 × 314 × 14MaxPool2 × 2/27 × 7Convolutional layer10243 × 37 × 7Convolutional layer5121 × 17 × 7Convolutional layer10243 × 37 × 7Convolutional layer5121 × 17 × 7Convolutional layer10243 × 37 × 7The offset of the predicted bounding box relative to the a priori box is used, as shown in Figure5. In order to make the center point fall in the current grid, a function is used to constrain the predicted offset value so that the offset value is within (0, 1).Figure 5
Schematic diagram of the position of the bounding box.According to Figure5, the following relationships exist:(11)bx=σtx+cx,(12)by=σty+cy,(13)bw=pwetw,(14)bh=pheth,(15)Probjece∗IOUobjectb=σt0.The meaning of each parameter of the formulae is shown in Table3Table 3
The meaning of each parameter in the formulae.
parameterMeaningbxThe prediction results of the horizontal and vertical coordinates of the center of the bounding box and the width and height valuesbybwbhσSigmoid functionProbjece∗IOUobjectbConfidence of bounding boxcx,cyRelative to the coordinates of the upper left corner of the targettxThe parameters learned by the network are used to predict the center, width and height, and confidence of the border.tytwtht0YOLOv2 does not have a fully connected layer, so it does not require the input image to have a fixed size. Every 10 times of training, different sizes will be randomly selected for training, which improves the robustness to images of different sizes.


#### 2.2.3. DeepSORT-Based Deep Learning Recognition Algorithm

The DeepSORT deep learning recognition algorithm fully combines the measurement of target motion information and appearance information and uses theCNN network to train on large-scale personnel re-recognition data sets. By extracting depth information, it improves the robustness against target loss and occlusion [29]. The algorithm flow is shown in Figure 6.Figure 6
Flowchart based on DeepSORT target tracking.State Estimation and Trajectory Processing. In the state estimation of DeepSORT, an eight-dimensional state space x is used to describe the target state in a certain period of time. The state space is as follows:(16)x=u,v,γ,h,u,˙v˙,γ˙,h˙.In formula (16), u is the abscissa of the center of the bounding box in the pixel coordinate system, v is the ordinate of the center of the bounding box in the pixel coordinate system, and γ is the aspect ratio of the bounding box. h is the height of the bounding box, u˙ is the speed of the center of the bounding box along the x-axis in the image coordinate system, and v˙ is the speed of the center of the bounding box along the y-axis in the image coordinate system. γ˙ represents the change in the aspect ratio of the bounding box in adjacent frames, and h˙ represents the change in the height of the bounding box in adjacent frames.DeepSORT uses a Kalman filter model with uniform motion and linear observation model to predict and update the state of the object, and the observation variable isu,v,γ,h.In terms of trajectory processing, the algorithm builds target and predicted frames by counting the number of frames associated with each target. If the number of frames is greater than the thresholdAmax, it is regarded as the end of tracking. The detection that cannot be successfully associated with the current trajectory is regarded as a new trajectory. In order to avoid false positive trajectories, the new trajectory must be associated in three consecutive frames, and vice versa.The Problem of Matching the Target and the Detection Frame. On the basis of the original sort algorithm that only used the Hungarian method to match the Kalman state of the target with the newly generated state matching problem, the DeepSORT algorithm uses a method that combines the target’s motion characteristics and surface characteristics to solve the problem.For the motion characteristics of the target, the Mahalanobis distance between the predicted Kalman state and the newly generated state is collected for evaluation [30]:(17)d1i,j=dj−yiTSi−1dj−yi.In formula (17), d1i,j represents the degree of correlation between the jth target detection and the ith trajectory, and Si represents the covariance matrix of the Kalman filter predicting the current state space. yi is the current observation of the target motion trajectory, and dj is the current state of the target when the jth target is detected [31].In the process of target movement, in order to avoid obvious misassociation of the target, 95% of the chi-square distribution is used as the threshold,t1=9.4877 is used for screening, and the following threshold function is set:(18)bij1=Id1i,j≤t1.In formula (18), bij1 is expressed as the result between the jth target detection and the ith trajectory. If the distance between the target detection and the trajectory is less than the threshold t1, the result is true [32].But the Mahalanobis distance is not very practical, when the uncertainty of the movement of the target is low, the Mahalanobis distance has a better correlation measure. However, in the actual environment, the movement of the camera makes it difficult to correlate the Mahalanobis distance. So, the second measurement method is proposed. For each bounding boxdj, calculate its surface feature descriptor rj, and satisfy rj=1. Create a set of feature vectors corresponding to the bounding box after tracking the object Lk=100 times; namely, Rk=rkik=1Lk. Use the minimum cosine distance between the ith target trajectory and the jth detection frame as the second correlation metric; namely [33],(19)d2i,j=min1−rjTrki|rki∈Ri.Taking the advantage of Mahalanobis distance in short-term target prediction and the advantage of cosine distance in regaining tracking after the target has been occluded for a long time, the two relationship measures are combined by weighting [34]:(20)ci,j=λd1i,j+1−λd2i,j.In formula (20), λ is expressed as the weight of Mahalanobis distance, which satisfies λ∈0,1.The effect of target tracking can be improved by changing the weightλ. If the camera has obvious movement, λ=0 will have better performance.


## 2.1. Overview of Vehicle-Road Collaborative Information Interaction System

The vehicle-road collaboration system is a technical system that integrates a number of emerging sciences and technologies (information analysis and processing, satellite navigation, sensing technology, communication technology, artificial intelligence, and other technologies) to obtain road and vehicle information in real time [9–11]. The goal of the system is to achieve accurate and real-time information interaction between on-board equipment and roadside equipment, achieving a full range of network connections between the internal equipment of the car, between the car and the car, between the car and the road, between the car and the person, between the car and the cloud, and between the road and the cloud. It makes the more disorderly operation of the road system become more efficient and orderly and at the same time realizes road collaborative management and active vehicle safety control. It fully realizes the coordinated control between vehicles and roads [12, 13]. It provides all traffic participants with accurate and reliable traffic assistance information, realizing the full coordination of people, vehicles, and roads to form a safe, efficient, and environmentally friendly road traffic system, as shown in Figure 1.Figure 1
Schematic diagram of information interaction of the vehicle-road collaboration system.


### 2.1.1. Vehicle-Road Cooperative System Structure

The vehicle-road collaboration system mainly includes intelligent roadside systems, intelligent vehicle systems, signal equipment, and traffic information processing and management systems. The system structure is shown in Figure2.Figure 2
Vehicle-road cooperative system structure.The vehicle-road collaboration system is divided into three parts: the vehicle-mounted system, the cloud service system, and the drive test system. The vehicle on-board system mainly performs calculations and decisions on vehicle safety, such as vehicle emergency avoidance warning, emergency braking, and other operations [14–16]. The cloud service system has a high level of computing and is used to process autonomous driving applications that require timeliness and moderate calculations, such as blind zone warning and green wave speed guidance. The coverage and computing capacity of the drive test system is the largest among the three systems. The structure is shown in Table 1. It is mainly used to deal with the calculation content that is not sensitive to time delay, such as road path planning, macro traffic guidance, global high-precision map management, and so on. In fact, the real-time intelligent optimization of road traffic flow at the macro level is now. The entire vehicle-road collaboration system needs to receive data information through multiple channels. It includes information about the vehicle’s own state (brake, steering, pressure, and temperature), driving environment (obstacles, spacing), road surface (road surface, geometric conditions), traffic flow (vehicle volume, vehicle speed, and occupancy rate), and other information. It reuses navigation and positioning technology and limited or wireless communication technology to send these information data to the cloud server. The cloud server comprehensively processes these information data to form effective information feedback and publish the information feedback. This coherent system operation can also be summarized as the interaction between roadside system information and vehicle-mounted system information and between vehicle-mounted system information and vehicle-mounted system information.Table 1
Structural composition of drive test system.
Component equipmentThe main functionDrive test communication unitCombine the perception information generated by the perception unit for a more comprehensive road information perceptionTraffic management unitSensor unitObtain road and vehicle information and realize different information processingDrive test sensing unit


### 2.1.2. Application and Function of Vehicle-Road Coordination System

(1)
At present, the vehicle-road coordination system can be used for traffic safety assurance, traffic planning and decision-making, and environmental protection. Traffic safety guarantee is the most important basic function of the vehicle-road coordination system. When the vehicle brakes or turns due to unforeseen emergencies or accidents, the vehicle-road coordination system will use its own communication technology to enable surrounding vehicles to obtain early warning information related to the accident so that the driver can respond in time and avoid the occurrence of dangerous traffic accidents. Compared with smart vehicles equipped with advanced sensing equipment, drivers take longer to respond to emergencies and accidents. Smart vehicles without communication technology cannot respond to emergencies and accidents within the blind zone when the driver is driving the vehicle and make emergency braking or turning on the vehicle [17–19]. A smart car with communication technology can obtain content that it cannot perceive through the information interaction between the roadside system and the on-board system and between the on-board system and the on-board system. It greatly reduces the occurrence of vehicle safety accidents and guarantees traffic safety.(2)
The intelligent vehicle-road collaboration system has more accurate road traffic planning and decision-making capabilities. Because the in-vehicle system has the ability to communicate with the roadside system, it can obtain and share the dynamic information (vehicle position, vehicle speed, and acceleration) of the vehicles driving on each section of the road and the environmental information collected by the roadside facilities. With richer perception information, a decision-making plan that is conducive to the optimal vehicle operation in the overall situation is calculated. It enables vehicles to travel in formation safely under a smaller distance between vehicles, increasing the average running speed and reducing the running time. When a road accident occurs, vehicles driven on other roads can obtain early warning information, and at the same time, they will continuously broadcast their own relevant information. It is beneficial to realize multivehicle coordinated lane change in a very short time, provide driving assistance information for surrounding vehicles, and cooperate with road condition information provided by roadside equipment to avoid traffic jams and improve traffic efficiency.(3)
The vehicle-road coordination system helps reduce energy consumption. The urban traffic management department obtains the traffic operation information of each urban road with the assistance of the vehicle-road coordination system. As a result, traffic can be cleared and directed more effectively, and energy loss caused by road congestion can be reduced [20]. Intelligent vehicles need to rely on high-performance on-board computers for road condition information collection, analysis, and decision-making. The main computing modules are installed in the car, and the on-board computer takes up a certain amount of space and has a large power consumption. Most of the calculation modules of the vehicle-road coordination system can be transferred to the roadside system. This can greatly save vehicle space, reducing vehicle weight, energy consumption, and exhaust emissions. It has great positive significance in environmental protection.


## 2.1.1. Vehicle-Road Cooperative System Structure

The vehicle-road collaboration system mainly includes intelligent roadside systems, intelligent vehicle systems, signal equipment, and traffic information processing and management systems. The system structure is shown in Figure2.Figure 2
Vehicle-road cooperative system structure.The vehicle-road collaboration system is divided into three parts: the vehicle-mounted system, the cloud service system, and the drive test system. The vehicle on-board system mainly performs calculations and decisions on vehicle safety, such as vehicle emergency avoidance warning, emergency braking, and other operations [14–16]. The cloud service system has a high level of computing and is used to process autonomous driving applications that require timeliness and moderate calculations, such as blind zone warning and green wave speed guidance. The coverage and computing capacity of the drive test system is the largest among the three systems. The structure is shown in Table 1. It is mainly used to deal with the calculation content that is not sensitive to time delay, such as road path planning, macro traffic guidance, global high-precision map management, and so on. In fact, the real-time intelligent optimization of road traffic flow at the macro level is now. The entire vehicle-road collaboration system needs to receive data information through multiple channels. It includes information about the vehicle’s own state (brake, steering, pressure, and temperature), driving environment (obstacles, spacing), road surface (road surface, geometric conditions), traffic flow (vehicle volume, vehicle speed, and occupancy rate), and other information. It reuses navigation and positioning technology and limited or wireless communication technology to send these information data to the cloud server. The cloud server comprehensively processes these information data to form effective information feedback and publish the information feedback. This coherent system operation can also be summarized as the interaction between roadside system information and vehicle-mounted system information and between vehicle-mounted system information and vehicle-mounted system information.Table 1
Structural composition of drive test system.
Component equipmentThe main functionDrive test communication unitCombine the perception information generated by the perception unit for a more comprehensive road information perceptionTraffic management unitSensor unitObtain road and vehicle information and realize different information processingDrive test sensing unit


## 2.1.2. Application and Function of Vehicle-Road Coordination System

(1)
At present, the vehicle-road coordination system can be used for traffic safety assurance, traffic planning and decision-making, and environmental protection. Traffic safety guarantee is the most important basic function of the vehicle-road coordination system. When the vehicle brakes or turns due to unforeseen emergencies or accidents, the vehicle-road coordination system will use its own communication technology to enable surrounding vehicles to obtain early warning information related to the accident so that the driver can respond in time and avoid the occurrence of dangerous traffic accidents. Compared with smart vehicles equipped with advanced sensing equipment, drivers take longer to respond to emergencies and accidents. Smart vehicles without communication technology cannot respond to emergencies and accidents within the blind zone when the driver is driving the vehicle and make emergency braking or turning on the vehicle [17–19]. A smart car with communication technology can obtain content that it cannot perceive through the information interaction between the roadside system and the on-board system and between the on-board system and the on-board system. It greatly reduces the occurrence of vehicle safety accidents and guarantees traffic safety.(2)
The intelligent vehicle-road collaboration system has more accurate road traffic planning and decision-making capabilities. Because the in-vehicle system has the ability to communicate with the roadside system, it can obtain and share the dynamic information (vehicle position, vehicle speed, and acceleration) of the vehicles driving on each section of the road and the environmental information collected by the roadside facilities. With richer perception information, a decision-making plan that is conducive to the optimal vehicle operation in the overall situation is calculated. It enables vehicles to travel in formation safely under a smaller distance between vehicles, increasing the average running speed and reducing the running time. When a road accident occurs, vehicles driven on other roads can obtain early warning information, and at the same time, they will continuously broadcast their own relevant information. It is beneficial to realize multivehicle coordinated lane change in a very short time, provide driving assistance information for surrounding vehicles, and cooperate with road condition information provided by roadside equipment to avoid traffic jams and improve traffic efficiency.(3)
The vehicle-road coordination system helps reduce energy consumption. The urban traffic management department obtains the traffic operation information of each urban road with the assistance of the vehicle-road coordination system. As a result, traffic can be cleared and directed more effectively, and energy loss caused by road congestion can be reduced [20]. Intelligent vehicles need to rely on high-performance on-board computers for road condition information collection, analysis, and decision-making. The main computing modules are installed in the car, and the on-board computer takes up a certain amount of space and has a large power consumption. Most of the calculation modules of the vehicle-road coordination system can be transferred to the roadside system. This can greatly save vehicle space, reducing vehicle weight, energy consumption, and exhaust emissions. It has great positive significance in environmental protection.


## 2.2. Deep Learning Recognition Methods

Deep learning recognition is one of the most effective perception methods in the vehicle-road collaboration system. It can obtain more comprehensive and accurate vehicle information under different angles and different fields of view according to the deep learning recognition method and provide the most intuitive and reliable information judgment for the vehicle-road collaborative system. At the same time, it also provides relevant basis for the system’s reasoning decision and execution [21]. It conducts research on the deep learning recognition method in the intelligent vehicle-road collaboration system and understands its ability to detect and recognize vehicle targets in video images, as well as its ability to count vehicle traffic.


### 2.2.1. Deep Learning Recognition Algorithm Based on Background Extraction

Vehicle-road collaboration collects video through the vision system to obtain road pedestrian and vehicle movement information. The detection methods of pedestrians and vehicles in this environment most often use background extraction algorithms; the background extraction algorithm extracts objects with relatively small changes in pixel values from the video, that is, the background. This algorithm mainly finds the background value of each point in the image based on the video image sequence, and the commonly used background extraction algorithms are subdivided into background difference method and interframe difference method.Background Difference Method. The background difference method is a general method for motion segmentation of still scenes. It uses the difference operation between the acquired image frame and the background image to obtain the grayscale image of the target moving area. Thresholding the gray image to extract the moving area, in order to avoid the influence of environmental lighting changes, the background image needs to be updated according to the currently acquired image frame [22]. The process is shown in Figure 3.Figure 3
Background difference method process.UseFx,y to represent the current frame, Ax,y to represent the currently acquired background frame, the difference between the current frame and the background frame to get the difference image Dx,y; then there is [23](1)Dx,y=Fx,y−Ax,y.UseT to represent the threshold of image binarization, and perform a binarization operation on the difference image to extract the area higher than the threshold T, thereby extracting the target moving on the road; that is, [24](2)Mx,y=Fx,y,Dx,y≥T,0,Dx,y<T.Among them,Mx,y represents the movement goal.The algorithm of the background difference method is simple, but it will also be disturbed by the scene, such as the shaking of branches, and the construction and update of the background must not include moving targets. The quality of background modeling directly affects the performance of the algorithm.Interframe Difference Method. The interframe difference method is to subtract the pixel values of two adjacent frames or two images separated by a few frames in the video stream and threshold the subtracted image to extract the motion area in the image. The process of the interframe difference method is shown in Figure 4.Figure 4
Interframe difference method process.The images of the adjacentjth and j+1th frames are represented as Fjx,y and Fj+1x,y, the binarization threshold in the differential image is T, and the differential image is represented as Dx,y; then there are [25](3)Dx,y=1,Fj+1x,y−Fjx,y>T,0,Fj+1x,y−Fjx,y≤T.The interframe difference method does not need to consider the construction of the background model, the algorithm is simple, fast, real-time, and insensitive to the overall lighting changes in the environment. However, the interframe difference method is easy to detect the edges of moving objects. When the difference between the adjacent frames of the moving object is not large, the extracted moving object will produce holes, which is not good for the recognition of the target. The use of interframe difference method is also easy to produce ghost regions; that is, when a stationary object starts to move, the area when the object is stationary is detected as motion after using the interframe difference method. For the same reason, when the moving object starts to enter the static state, it is easy to appear this kind of area that is wrongly detected as moving.


### 2.2.2. Recognition Algorithm Based on Deep Learning

YOLO algorithm is called You only look once in full; that is, it only needs to perform convolutional neural network calculations under a unified framework to achieve end-to-end real-time target location prediction and recognition. The YOLO algorithm is divided into YOLOv1, YOLOv2, and YOLOv3 according to the time it was launched. The performance varies between different versions. YOLOv2 has made relevant improvements on the basis of YOLOv1, which effectively improves the recognition type, accuracy, recognition speed, and positioning accuracy. The accuracy of YOLOv3 has a certain improvement compared with YOLOv2. The network adopts the class ResNet structure, the network structure is more complicated, and the real-time performance has been slightly reduced. Therefore, the use of network YOLOv2 for target detection is more suitable for the urban road traffic environment with higher real-time requirements, and the YOLOv2 network divides the input image into S×S grids. Each grid predicts the confidence of B bounding boxes and bounding boxes. The bounding box information includes the center coordinates of the bounding box, the width and height values, and the confidence score [26]. The confidence information includes the possibility of the target existence of the bounding box and the accuracy of the position of the bounding box, which is defined as(4)Confidence=Probjece∗IOUpredtruth.Among them,Probjece∈0,1, if the target is in the grid, then Probjece=1. On the contrary, Probjece=0. The intersection ratio IOUpredtruth is used to describe the accuracy of the bounding box position, and the ratio of the bounding box intersection and union of the predicted position A of the detection target and the actual position B; namely,(5)IOUpredtruth=SA∩BSA∪B.If the predicted position completely coincides with the actual position, thenIOU=1.Therefore, the final output dimension of theYOLOv2 network is S∗S∗B∗5+C, where C is the number of target categories. For example, in a target detection for urban road traffic, the number of C is 5 (pedestrians, cars, buses, trucks, and bicycles).Same asYOLOv1, YOLOv2 uses the mean square error as the loss function. The loss function is composed of three parts: positioning error, confidence error, and classification error. The relationship between the loss function and the three parts is as follows [27]:(6)loss=Ecoord+EIOU+Eclass.The positioning error is the deviation of the position and size of the predicted bounding box from the actual; that is, there is the following relationship:(7)Ecoord=λcoord∑i=0S2∑j=0BIijobjxi−xi^2+yi−yi^2+λcoord∑i=0S2∑j=0BIijobjwi−wi^2+hi−hi^22.Among them,xi, yi, wi, and hi represent the predicted value of the model to the target, xi^, yi^, wi^, and hi^ represent the actual value of the target, and λcoord represents the weight value of the coordinate error, which is generally 0.5. Iijobj indicates that there is a target in the ith grid, and the jth box in the grid is responsible for the prediction of the target.For the confidence error, there is the following relationship:(8)EIOU=∑i=0S2∑j=0BIijobjCi−Ci^2+λno obj∑i=0S2∑j=oBIijno objCi−Ci^2.Among them, the weightλno obj generally takes the value 0.5, and the values of Ci and Ci^ have the following relationship [28]:(9)Ci=IOUpredtruth,Ci^=1,Iiobj=1,0,Iino obj=1.If the object is detected, the classification error of each grid is the square error of the conditional probability of each type; namely,(10)Eclass=∑i=0S2Iiobj∑c∈classpic−pi^c.Among them,pic and pi^c, respectively, represent the predicted category probability and the actual category probability in the grid.In terms of network structure,YOLOv2 uses the Darknet 19 classification model. It draws on the VGG network, uses more 3×3 convolution kernels, and compresses the features through the 1×1 convolution kernel. Darknet 19 contains nineteen convolutional layers and five layers Maxpool. The network parameters of Darknet 19 are shown in Table 2.Table 2
Convolutional layer network parameters.
TypeNumber of filtersFilter size/step sizeOutputConvolutional layer323 × 3224 × 224MaxPool2 × 2/2112 × 112Convolutional layer643 × 3112 × 112MaxPool2 × 2/256 × 56Convolutional layer1283 × 356 × 56Convolutional layer641 × 156 × 56Convolutional layer1283 × 356 × 56MaxPool2 × 2/228 × 28Convolutional layer2563 × 328 × 28Convolutional layer1281 × 128 × 28Convolutional layer2563 × 328 × 28MaxPool2 × 2/214 × 14Convolutional layer5123 × 314 × 14Convolutional layer2561 × 114 × 14Convolutional layer5123 × 314 × 14Convolutional layer2561 × 114 × 14Convolutional layer5123 × 314 × 14MaxPool2 × 2/27 × 7Convolutional layer10243 × 37 × 7Convolutional layer5121 × 17 × 7Convolutional layer10243 × 37 × 7Convolutional layer5121 × 17 × 7Convolutional layer10243 × 37 × 7The offset of the predicted bounding box relative to the a priori box is used, as shown in Figure5. In order to make the center point fall in the current grid, a function is used to constrain the predicted offset value so that the offset value is within (0, 1).Figure 5
Schematic diagram of the position of the bounding box.According to Figure5, the following relationships exist:(11)bx=σtx+cx,(12)by=σty+cy,(13)bw=pwetw,(14)bh=pheth,(15)Probjece∗IOUobjectb=σt0.The meaning of each parameter of the formulae is shown in Table3Table 3
The meaning of each parameter in the formulae.
parameterMeaningbxThe prediction results of the horizontal and vertical coordinates of the center of the bounding box and the width and height valuesbybwbhσSigmoid functionProbjece∗IOUobjectbConfidence of bounding boxcx,cyRelative to the coordinates of the upper left corner of the targettxThe parameters learned by the network are used to predict the center, width and height, and confidence of the border.tytwtht0YOLOv2 does not have a fully connected layer, so it does not require the input image to have a fixed size. Every 10 times of training, different sizes will be randomly selected for training, which improves the robustness to images of different sizes.


### 2.2.3. DeepSORT-Based Deep Learning Recognition Algorithm

The DeepSORT deep learning recognition algorithm fully combines the measurement of target motion information and appearance information and uses theCNN network to train on large-scale personnel re-recognition data sets. By extracting depth information, it improves the robustness against target loss and occlusion [29]. The algorithm flow is shown in Figure 6.Figure 6
Flowchart based on DeepSORT target tracking.State Estimation and Trajectory Processing. In the state estimation of DeepSORT, an eight-dimensional state space x is used to describe the target state in a certain period of time. The state space is as follows:(16)x=u,v,γ,h,u,˙v˙,γ˙,h˙.In formula (16), u is the abscissa of the center of the bounding box in the pixel coordinate system, v is the ordinate of the center of the bounding box in the pixel coordinate system, and γ is the aspect ratio of the bounding box. h is the height of the bounding box, u˙ is the speed of the center of the bounding box along the x-axis in the image coordinate system, and v˙ is the speed of the center of the bounding box along the y-axis in the image coordinate system. γ˙ represents the change in the aspect ratio of the bounding box in adjacent frames, and h˙ represents the change in the height of the bounding box in adjacent frames.DeepSORT uses a Kalman filter model with uniform motion and linear observation model to predict and update the state of the object, and the observation variable isu,v,γ,h.In terms of trajectory processing, the algorithm builds target and predicted frames by counting the number of frames associated with each target. If the number of frames is greater than the thresholdAmax, it is regarded as the end of tracking. The detection that cannot be successfully associated with the current trajectory is regarded as a new trajectory. In order to avoid false positive trajectories, the new trajectory must be associated in three consecutive frames, and vice versa.The Problem of Matching the Target and the Detection Frame. On the basis of the original sort algorithm that only used the Hungarian method to match the Kalman state of the target with the newly generated state matching problem, the DeepSORT algorithm uses a method that combines the target’s motion characteristics and surface characteristics to solve the problem.For the motion characteristics of the target, the Mahalanobis distance between the predicted Kalman state and the newly generated state is collected for evaluation [30]:(17)d1i,j=dj−yiTSi−1dj−yi.In formula (17), d1i,j represents the degree of correlation between the jth target detection and the ith trajectory, and Si represents the covariance matrix of the Kalman filter predicting the current state space. yi is the current observation of the target motion trajectory, and dj is the current state of the target when the jth target is detected [31].In the process of target movement, in order to avoid obvious misassociation of the target, 95% of the chi-square distribution is used as the threshold,t1=9.4877 is used for screening, and the following threshold function is set:(18)bij1=Id1i,j≤t1.In formula (18), bij1 is expressed as the result between the jth target detection and the ith trajectory. If the distance between the target detection and the trajectory is less than the threshold t1, the result is true [32].But the Mahalanobis distance is not very practical, when the uncertainty of the movement of the target is low, the Mahalanobis distance has a better correlation measure. However, in the actual environment, the movement of the camera makes it difficult to correlate the Mahalanobis distance. So, the second measurement method is proposed. For each bounding boxdj, calculate its surface feature descriptor rj, and satisfy rj=1. Create a set of feature vectors corresponding to the bounding box after tracking the object Lk=100 times; namely, Rk=rkik=1Lk. Use the minimum cosine distance between the ith target trajectory and the jth detection frame as the second correlation metric; namely [33],(19)d2i,j=min1−rjTrki|rki∈Ri.Taking the advantage of Mahalanobis distance in short-term target prediction and the advantage of cosine distance in regaining tracking after the target has been occluded for a long time, the two relationship measures are combined by weighting [34]:(20)ci,j=λd1i,j+1−λd2i,j.In formula (20), λ is expressed as the weight of Mahalanobis distance, which satisfies λ∈0,1.The effect of target tracking can be improved by changing the weightλ. If the camera has obvious movement, λ=0 will have better performance.


## 2.2.1. Deep Learning Recognition Algorithm Based on Background Extraction

Vehicle-road collaboration collects video through the vision system to obtain road pedestrian and vehicle movement information. The detection methods of pedestrians and vehicles in this environment most often use background extraction algorithms; the background extraction algorithm extracts objects with relatively small changes in pixel values from the video, that is, the background. This algorithm mainly finds the background value of each point in the image based on the video image sequence, and the commonly used background extraction algorithms are subdivided into background difference method and interframe difference method.Background Difference Method. The background difference method is a general method for motion segmentation of still scenes. It uses the difference operation between the acquired image frame and the background image to obtain the grayscale image of the target moving area. Thresholding the gray image to extract the moving area, in order to avoid the influence of environmental lighting changes, the background image needs to be updated according to the currently acquired image frame [22]. The process is shown in Figure 3.Figure 3
Background difference method process.UseFx,y to represent the current frame, Ax,y to represent the currently acquired background frame, the difference between the current frame and the background frame to get the difference image Dx,y; then there is [23](1)Dx,y=Fx,y−Ax,y.UseT to represent the threshold of image binarization, and perform a binarization operation on the difference image to extract the area higher than the threshold T, thereby extracting the target moving on the road; that is, [24](2)Mx,y=Fx,y,Dx,y≥T,0,Dx,y<T.Among them,Mx,y represents the movement goal.The algorithm of the background difference method is simple, but it will also be disturbed by the scene, such as the shaking of branches, and the construction and update of the background must not include moving targets. The quality of background modeling directly affects the performance of the algorithm.Interframe Difference Method. The interframe difference method is to subtract the pixel values of two adjacent frames or two images separated by a few frames in the video stream and threshold the subtracted image to extract the motion area in the image. The process of the interframe difference method is shown in Figure 4.Figure 4
Interframe difference method process.The images of the adjacentjth and j+1th frames are represented as Fjx,y and Fj+1x,y, the binarization threshold in the differential image is T, and the differential image is represented as Dx,y; then there are [25](3)Dx,y=1,Fj+1x,y−Fjx,y>T,0,Fj+1x,y−Fjx,y≤T.The interframe difference method does not need to consider the construction of the background model, the algorithm is simple, fast, real-time, and insensitive to the overall lighting changes in the environment. However, the interframe difference method is easy to detect the edges of moving objects. When the difference between the adjacent frames of the moving object is not large, the extracted moving object will produce holes, which is not good for the recognition of the target. The use of interframe difference method is also easy to produce ghost regions; that is, when a stationary object starts to move, the area when the object is stationary is detected as motion after using the interframe difference method. For the same reason, when the moving object starts to enter the static state, it is easy to appear this kind of area that is wrongly detected as moving.


## 2.2.2. Recognition Algorithm Based on Deep Learning

YOLO algorithm is called You only look once in full; that is, it only needs to perform convolutional neural network calculations under a unified framework to achieve end-to-end real-time target location prediction and recognition. The YOLO algorithm is divided into YOLOv1, YOLOv2, and YOLOv3 according to the time it was launched. The performance varies between different versions. YOLOv2 has made relevant improvements on the basis of YOLOv1, which effectively improves the recognition type, accuracy, recognition speed, and positioning accuracy. The accuracy of YOLOv3 has a certain improvement compared with YOLOv2. The network adopts the class ResNet structure, the network structure is more complicated, and the real-time performance has been slightly reduced. Therefore, the use of network YOLOv2 for target detection is more suitable for the urban road traffic environment with higher real-time requirements, and the YOLOv2 network divides the input image into S×S grids. Each grid predicts the confidence of B bounding boxes and bounding boxes. The bounding box information includes the center coordinates of the bounding box, the width and height values, and the confidence score [26]. The confidence information includes the possibility of the target existence of the bounding box and the accuracy of the position of the bounding box, which is defined as(4)Confidence=Probjece∗IOUpredtruth.Among them,Probjece∈0,1, if the target is in the grid, then Probjece=1. On the contrary, Probjece=0. The intersection ratio IOUpredtruth is used to describe the accuracy of the bounding box position, and the ratio of the bounding box intersection and union of the predicted position A of the detection target and the actual position B; namely,(5)IOUpredtruth=SA∩BSA∪B.If the predicted position completely coincides with the actual position, thenIOU=1.Therefore, the final output dimension of theYOLOv2 network is S∗S∗B∗5+C, where C is the number of target categories. For example, in a target detection for urban road traffic, the number of C is 5 (pedestrians, cars, buses, trucks, and bicycles).Same asYOLOv1, YOLOv2 uses the mean square error as the loss function. The loss function is composed of three parts: positioning error, confidence error, and classification error. The relationship between the loss function and the three parts is as follows [27]:(6)loss=Ecoord+EIOU+Eclass.The positioning error is the deviation of the position and size of the predicted bounding box from the actual; that is, there is the following relationship:(7)Ecoord=λcoord∑i=0S2∑j=0BIijobjxi−xi^2+yi−yi^2+λcoord∑i=0S2∑j=0BIijobjwi−wi^2+hi−hi^22.Among them,xi, yi, wi, and hi represent the predicted value of the model to the target, xi^, yi^, wi^, and hi^ represent the actual value of the target, and λcoord represents the weight value of the coordinate error, which is generally 0.5. Iijobj indicates that there is a target in the ith grid, and the jth box in the grid is responsible for the prediction of the target.For the confidence error, there is the following relationship:(8)EIOU=∑i=0S2∑j=0BIijobjCi−Ci^2+λno obj∑i=0S2∑j=oBIijno objCi−Ci^2.Among them, the weightλno obj generally takes the value 0.5, and the values of Ci and Ci^ have the following relationship [28]:(9)Ci=IOUpredtruth,Ci^=1,Iiobj=1,0,Iino obj=1.If the object is detected, the classification error of each grid is the square error of the conditional probability of each type; namely,(10)Eclass=∑i=0S2Iiobj∑c∈classpic−pi^c.Among them,pic and pi^c, respectively, represent the predicted category probability and the actual category probability in the grid.In terms of network structure,YOLOv2 uses the Darknet 19 classification model. It draws on the VGG network, uses more 3×3 convolution kernels, and compresses the features through the 1×1 convolution kernel. Darknet 19 contains nineteen convolutional layers and five layers Maxpool. The network parameters of Darknet 19 are shown in Table 2.Table 2
Convolutional layer network parameters.
TypeNumber of filtersFilter size/step sizeOutputConvolutional layer323 × 3224 × 224MaxPool2 × 2/2112 × 112Convolutional layer643 × 3112 × 112MaxPool2 × 2/256 × 56Convolutional layer1283 × 356 × 56Convolutional layer641 × 156 × 56Convolutional layer1283 × 356 × 56MaxPool2 × 2/228 × 28Convolutional layer2563 × 328 × 28Convolutional layer1281 × 128 × 28Convolutional layer2563 × 328 × 28MaxPool2 × 2/214 × 14Convolutional layer5123 × 314 × 14Convolutional layer2561 × 114 × 14Convolutional layer5123 × 314 × 14Convolutional layer2561 × 114 × 14Convolutional layer5123 × 314 × 14MaxPool2 × 2/27 × 7Convolutional layer10243 × 37 × 7Convolutional layer5121 × 17 × 7Convolutional layer10243 × 37 × 7Convolutional layer5121 × 17 × 7Convolutional layer10243 × 37 × 7The offset of the predicted bounding box relative to the a priori box is used, as shown in Figure5. In order to make the center point fall in the current grid, a function is used to constrain the predicted offset value so that the offset value is within (0, 1).Figure 5
Schematic diagram of the position of the bounding box.According to Figure5, the following relationships exist:(11)bx=σtx+cx,(12)by=σty+cy,(13)bw=pwetw,(14)bh=pheth,(15)Probjece∗IOUobjectb=σt0.The meaning of each parameter of the formulae is shown in Table3Table 3
The meaning of each parameter in the formulae.
parameterMeaningbxThe prediction results of the horizontal and vertical coordinates of the center of the bounding box and the width and height valuesbybwbhσSigmoid functionProbjece∗IOUobjectbConfidence of bounding boxcx,cyRelative to the coordinates of the upper left corner of the targettxThe parameters learned by the network are used to predict the center, width and height, and confidence of the border.tytwtht0YOLOv2 does not have a fully connected layer, so it does not require the input image to have a fixed size. Every 10 times of training, different sizes will be randomly selected for training, which improves the robustness to images of different sizes.


## 2.2.3. DeepSORT-Based Deep Learning Recognition Algorithm

The DeepSORT deep learning recognition algorithm fully combines the measurement of target motion information and appearance information and uses theCNN network to train on large-scale personnel re-recognition data sets. By extracting depth information, it improves the robustness against target loss and occlusion [29]. The algorithm flow is shown in Figure 6.Figure 6
Flowchart based on DeepSORT target tracking.State Estimation and Trajectory Processing. In the state estimation of DeepSORT, an eight-dimensional state space x is used to describe the target state in a certain period of time. The state space is as follows:(16)x=u,v,γ,h,u,˙v˙,γ˙,h˙.In formula (16), u is the abscissa of the center of the bounding box in the pixel coordinate system, v is the ordinate of the center of the bounding box in the pixel coordinate system, and γ is the aspect ratio of the bounding box. h is the height of the bounding box, u˙ is the speed of the center of the bounding box along the x-axis in the image coordinate system, and v˙ is the speed of the center of the bounding box along the y-axis in the image coordinate system. γ˙ represents the change in the aspect ratio of the bounding box in adjacent frames, and h˙ represents the change in the height of the bounding box in adjacent frames.DeepSORT uses a Kalman filter model with uniform motion and linear observation model to predict and update the state of the object, and the observation variable isu,v,γ,h.In terms of trajectory processing, the algorithm builds target and predicted frames by counting the number of frames associated with each target. If the number of frames is greater than the thresholdAmax, it is regarded as the end of tracking. The detection that cannot be successfully associated with the current trajectory is regarded as a new trajectory. In order to avoid false positive trajectories, the new trajectory must be associated in three consecutive frames, and vice versa.The Problem of Matching the Target and the Detection Frame. On the basis of the original sort algorithm that only used the Hungarian method to match the Kalman state of the target with the newly generated state matching problem, the DeepSORT algorithm uses a method that combines the target’s motion characteristics and surface characteristics to solve the problem.For the motion characteristics of the target, the Mahalanobis distance between the predicted Kalman state and the newly generated state is collected for evaluation [30]:(17)d1i,j=dj−yiTSi−1dj−yi.In formula (17), d1i,j represents the degree of correlation between the jth target detection and the ith trajectory, and Si represents the covariance matrix of the Kalman filter predicting the current state space. yi is the current observation of the target motion trajectory, and dj is the current state of the target when the jth target is detected [31].In the process of target movement, in order to avoid obvious misassociation of the target, 95% of the chi-square distribution is used as the threshold,t1=9.4877 is used for screening, and the following threshold function is set:(18)bij1=Id1i,j≤t1.In formula (18), bij1 is expressed as the result between the jth target detection and the ith trajectory. If the distance between the target detection and the trajectory is less than the threshold t1, the result is true [32].But the Mahalanobis distance is not very practical, when the uncertainty of the movement of the target is low, the Mahalanobis distance has a better correlation measure. However, in the actual environment, the movement of the camera makes it difficult to correlate the Mahalanobis distance. So, the second measurement method is proposed. For each bounding boxdj, calculate its surface feature descriptor rj, and satisfy rj=1. Create a set of feature vectors corresponding to the bounding box after tracking the object Lk=100 times; namely, Rk=rkik=1Lk. Use the minimum cosine distance between the ith target trajectory and the jth detection frame as the second correlation metric; namely [33],(19)d2i,j=min1−rjTrki|rki∈Ri.Taking the advantage of Mahalanobis distance in short-term target prediction and the advantage of cosine distance in regaining tracking after the target has been occluded for a long time, the two relationship measures are combined by weighting [34]:(20)ci,j=λd1i,j+1−λd2i,j.In formula (20), λ is expressed as the weight of Mahalanobis distance, which satisfies λ∈0,1.The effect of target tracking can be improved by changing the weightλ. If the camera has obvious movement, λ=0 will have better performance.


## 3. Deep Learning Recognition Simulation Experiment of Vehicle-Road Collaborative System

Through the intelligent research of the depth recognition method in the vehicle-road collaborative system, this paper designs a simulation experiment. Test the depth recognition method for vehicle target detection, vehicle flow recognition, and conflict decision-making capabilities under different vehicle densities and different driving speeds. And it compares it with the traditional recognition algorithm, and analyzes its superiority and feasibility in the vehicle-road collaborative information interaction system. The experimental equipment is shown in Table4.Table 4
Experiment preparation.
Experiment apparatusSpecific configurationProcessorIntel Core i7-4790KRAM16 GBGraphics cardNVIDIA GTX1060Operating systemUbuntu 16.04Programming languagePython 3.7Deep learning frameworkTensorFlow 1.13


### 3.1. Vehicle Target Detection

This article selects the KITTI data set (one of the world’s largest computer vision algorithm evaluation data sets in autonomous driving scenarios) as the simulation experiment data set. In a complex vehicle-road environment, the intersection ratio and recall rate of the two recognition algorithms for different target detections (pedestrians, signal lights, vehicles, lanes, and traffic signs) are shown in Figure7.Figure 7
Cross-combination ratio and recall rate of target detection.
(a)(b)Figure7(a) shows the intersection ratio and recall rate under the deep recognition algorithm.Figure7(b) shows the intersection ratio and recall rate under the traditional recognition algorithm.According to Figure7, the intersection of pedestrians, signal lights, vehicles, lanes, and traffic signs under the deep recognition algorithm is about 81.39% of the overall level, and the overall level of recall is about 84.51%. Under the traditional recognition algorithm, the intersection of pedestrians, signal lights, vehicles, lanes, and traffic signs is about 72.72% of the overall level, and the overall level of recall is about 77.51%.


### 3.2. Vehicle Flow Identification

The vehicle flow identification test sample comes from the traffic flow of five videos collected at different time periods on a pedestrian overpass in a certain section of the city center. Each video is 3 minutes long. In order to verify the validity of the calculation of the traffic volume, the five groups of test sample videos are compared with the traditional recognition algorithm and the deep learning recognition method based on target tracking. Figure8 shows the comparison of the recognition results and accuracy of the two algorithms.Figure 8
The difference between deep recognition algorithm and traditional algorithm.
(a)(b)Figure8(a) shows the sample recognition results of the deep recognition algorithm and the traditional algorithm.Figure8(b) shows the recognition accuracy of the deep recognition algorithm and the traditional algorithm.The total value of traffic flow in each video is 450. According to the recognition results of the deep recognition algorithm and the traditional algorithm for each segment of the sample, the recognition accuracy of each segment of the sample is calculated. The recognition accuracy rate of the deep recognition algorithm is maintained at about 86.62%, and the recognition accuracy rate of the traditional algorithm is maintained at about 84.82%.


### 3.3. Conflict Decision

The vehicle-road collaborative utilization system algorithm can detect whether there are dynamic obstacles during road driving. If there is an obstacle, an early warning will be issued to remind the driver to make decisions (slow down the speed and maintain the distance between cars) in response to the conflict. This article uses this as a control parameter to test the early warning distance between the vehicle and the early warning vehicle under different vehicle densities and different driving speeds. The early warning distance is defined as the time difference between the two vehicles after the early warning vehicle sends out the information to the time the test vehicle receives the information. The local simulation parameters of the scene are shown in Table5, and the simulation results are shown in Figures 9 and 10.Table 5
Scene parameters.
ParameterValueVehicle density50–300Vehicle knots100 vehiclesVehicle speed10 km/h–60 km/hTotal length of road2 kmDynamic obstacle2 pcsFigure 9
Results of vehicle warning time under different vehicle densities.
(a)(b)Figure 10
Vehicle warning time results at different driving speeds.
(a)(b)Figure9(a) shows the early warning time of the deep learning recognition algorithm under different vehicle densities.Figure9(b) shows the warning time of the traditional algorithm under different vehicle densities.According to Figure9, the overall average value of the early warning time of the deep learning recognition algorithm under different vehicle densities is about 0.102 seconds, and the overall average value of the early warning time of the traditional algorithm under different vehicle densities is about 0.127 seconds.Figure10(a) shows the early warning time of the deep learning recognition algorithm at different driving speeds.Figure10(b) shows the warning time of the traditional algorithm at different driving speeds.According to Figure10, the overall average value of the early warning time of the deep learning recognition algorithm at different driving speeds (10 km/h–60 km/h) is about 0.121 seconds. The overall average value of the warning time of the traditional algorithm at different driving speeds (10 km/h–60 km/h) is about 0.129 seconds.


## 3.1. Vehicle Target Detection

This article selects the KITTI data set (one of the world’s largest computer vision algorithm evaluation data sets in autonomous driving scenarios) as the simulation experiment data set. In a complex vehicle-road environment, the intersection ratio and recall rate of the two recognition algorithms for different target detections (pedestrians, signal lights, vehicles, lanes, and traffic signs) are shown in Figure7.Figure 7
Cross-combination ratio and recall rate of target detection.
(a)(b)Figure7(a) shows the intersection ratio and recall rate under the deep recognition algorithm.Figure7(b) shows the intersection ratio and recall rate under the traditional recognition algorithm.According to Figure7, the intersection of pedestrians, signal lights, vehicles, lanes, and traffic signs under the deep recognition algorithm is about 81.39% of the overall level, and the overall level of recall is about 84.51%. Under the traditional recognition algorithm, the intersection of pedestrians, signal lights, vehicles, lanes, and traffic signs is about 72.72% of the overall level, and the overall level of recall is about 77.51%.


## 3.2. Vehicle Flow Identification

The vehicle flow identification test sample comes from the traffic flow of five videos collected at different time periods on a pedestrian overpass in a certain section of the city center. Each video is 3 minutes long. In order to verify the validity of the calculation of the traffic volume, the five groups of test sample videos are compared with the traditional recognition algorithm and the deep learning recognition method based on target tracking. Figure8 shows the comparison of the recognition results and accuracy of the two algorithms.Figure 8
The difference between deep recognition algorithm and traditional algorithm.
(a)(b)Figure8(a) shows the sample recognition results of the deep recognition algorithm and the traditional algorithm.Figure8(b) shows the recognition accuracy of the deep recognition algorithm and the traditional algorithm.The total value of traffic flow in each video is 450. According to the recognition results of the deep recognition algorithm and the traditional algorithm for each segment of the sample, the recognition accuracy of each segment of the sample is calculated. The recognition accuracy rate of the deep recognition algorithm is maintained at about 86.62%, and the recognition accuracy rate of the traditional algorithm is maintained at about 84.82%.


## 3.3. Conflict Decision

The vehicle-road collaborative utilization system algorithm can detect whether there are dynamic obstacles during road driving. If there is an obstacle, an early warning will be issued to remind the driver to make decisions (slow down the speed and maintain the distance between cars) in response to the conflict. This article uses this as a control parameter to test the early warning distance between the vehicle and the early warning vehicle under different vehicle densities and different driving speeds. The early warning distance is defined as the time difference between the two vehicles after the early warning vehicle sends out the information to the time the test vehicle receives the information. The local simulation parameters of the scene are shown in Table5, and the simulation results are shown in Figures 9 and 10.Table 5
Scene parameters.
ParameterValueVehicle density50–300Vehicle knots100 vehiclesVehicle speed10 km/h–60 km/hTotal length of road2 kmDynamic obstacle2 pcsFigure 9
Results of vehicle warning time under different vehicle densities.
(a)(b)Figure 10
Vehicle warning time results at different driving speeds.
(a)(b)Figure9(a) shows the early warning time of the deep learning recognition algorithm under different vehicle densities.Figure9(b) shows the warning time of the traditional algorithm under different vehicle densities.According to Figure9, the overall average value of the early warning time of the deep learning recognition algorithm under different vehicle densities is about 0.102 seconds, and the overall average value of the early warning time of the traditional algorithm under different vehicle densities is about 0.127 seconds.Figure10(a) shows the early warning time of the deep learning recognition algorithm at different driving speeds.Figure10(b) shows the warning time of the traditional algorithm at different driving speeds.According to Figure10, the overall average value of the early warning time of the deep learning recognition algorithm at different driving speeds (10 km/h–60 km/h) is about 0.121 seconds. The overall average value of the warning time of the traditional algorithm at different driving speeds (10 km/h–60 km/h) is about 0.129 seconds.


## 4. Discussion

According to the comparison of the simulation experiment results, we can know that the interaction ratio and recall rate of the deep learning recognition method for vehicle target detection are both greater than that of the traditional algorithm for vehicle target detection. This shows that the deep learning recognition method is more accurate and complete in target detection. It also has a high accuracy rate in terms of traffic flow statistics; the accuracy of the overall identification statistics can reach 86.62%, with a small gap between the statistical results and the actual traffic flow value, which meets the road’s requirements for traffic flow perception. In the sudden decision simulation experiment, the deep learning recognition algorithm is better than the traditional algorithm whether it is under different vehicle densities or different driving speeds. Deep learning can send out early warning messages in a short period of time. This means that the early warning information of the vehicle-road coordination system can be transmitted to surrounding vehicles faster under this algorithm. This allows the driver to have a longer actual braking or lane-changing operation time, which ensures the safety of the vehicle in the danger warning scene.


## 5. Conclusion

The continuous increase of urban population and the number of cars has made urban traffic problems increasingly prominent and hindered the sustainable development of urbanization. As one of the key technologies in urban transportation construction, the vehicle-road collaborative information interaction system plays an extremely important role in alleviating the problems of urban transportation contradictions. The system uses deep learning recognition methods to detect and track the traffic environment, road conditions, and vehicle implementation. And it will share and interact with effective information feedback. In the event of an unexpected accident, the driver will immediately issue early warning messages to avoid traffic accidents and improve traffic safety. The article verifies that the deep learning recognition method has a positive role in promoting the development and maturity of the vehicle-road collaborative information interaction system through simulation experiments. However, there are still many shortcomings in this study. The depth and breadth of the article research is not enough. The operation of the information interaction system of the vehicle-road collaboration system is extremely complicated, and the network conditions and influencing factors are also diverse. The simulation experiments in this article are all carried out under ideal conditions, without considering the actual changing factors. Coupled with the uncertainty of the traffic environment, the simulation experiment of sudden decision-making only considers the two indicators of vehicle density and driving speed. It makes the traffic scenes applicable to deep learning recognition methods have limitations. These two points need to be improved and deepened in future research work. It is believed that with the further improvement of the level of technological development, the accuracy of deep learning recognition algorithms will become higher and higher, and the vehicle-road collaborative information interaction system will be more complete.



---

*Source: 4921211-2022-06-29.xml*