# Theory and Method of Data Collection for Mixed Traffic Flow Based on Image Processing Technology

Authors: Dong-Yuan Ge; Xi-Fan Yao; Wen-Jiang Xiang; En-Chen Liu; Zhi-Bin Xu
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

As a key element of ITS (intelligent traffic systems), traffic information collection facilities play a key role, with ITS being able to analyze the state of mixed traffic more appropriately and can provide effective technical support for the design, management, and the evaluation of constructions.Traffic Infrastructure. Focusing on image processing technology, this study takes pedestrians, electric motor, and vehicles in mixed traffic flow as the research object, and Gaussian mixed model, Kalman filtering, and Fisher linear discriminant are introduced in the recognition system. On this basis, the mixed motion flow data acquisition framework model is elaborated in detail, which includes attribute extraction, object recognition, and object tracking. Given the difficulty in capturing reliable images of objects in real traffic scenes, this study adopted a novel background and foreground classification method with region proposal network so as to decrease the number of regions proposal from 2000 to 300, which can detect objects fast and accurately. Experiments demonstrate that the designed programme can collect the flow data by detecting and tracking moving object in the surveillance video for mixed traffic. Further integration of various modules to achieve integrated collection is another important task for further research and development. In the future, research on dynamic calibration of monocular vision will be carried out for distance measurement and speed measurement of vehicles and pedestrians.


---

## Body

## 1. Introduction

In mixed urban road traffic, pedestrians and electric vehicles have a major impact on driving, which not only threatens road safety but also leads to increased delays and reduced traffic capacity. How to manage pedestrians and traffic of electric motors and vehicles through traffic management and control, effectively improve the capacity of urban road networks, especially at intersections, reduce travel time for travelers, and improve passenger safety has become one of the primary problems facing urban transport in China. Therefore, more and more intelligent traffic control systems have been developed and applied in actual traffic management and control. As the primary element of ITS, traffic information collection facilities play a key role in many ITS systems.Road pricing is one of the most important ways to reduce the loss of traffic distribution efficiency. Kumar et al. studied the loss of efficiency of the multi-level traffic balance distribution with elastic demand at road prices [1]. Barbosa et al. proposed a novel vehicle detection model named Priority Vehicle Image Detection Network based on YOLOV3, for which a lightweight design strategy is adopted to decrease the execution time of the proposed model [2]. Hu et al. proposed a RepNet network for feature extraction of vehicle, the focal loss function is adopted to reduce the weight of simple samples, and the cosine similarity function is used to judge the similarity between images [3]. The study of Pang et al. helps to improve the existing condition of intersections and provides guidelines for providing adequate pedestrian facilities at signalized intersections for safe and comfortable crossing of pedestrian crosswalk [4–6].Through field investigation of typical signalized intersections at commercial hubs in Calcutta, the characteristics of pedestrian movement are described. This analysis takes into account several attributes, such as the width of the road, the age and gender of pedestrians, and whether they carry any luggage. The study found that pedestrians’ age and gender had an impact on their speed; however, children were observed to walk faster because they were accompanied by their parents in most cases [7–10]. At crossroads, panicked pedestrians like to run fairly fast on zebra signs. Interestingly, it was found that the effect of carrying luggage on walking speed was not significant at the study site. Therefore, current research attempts to further investigate this fact by conducting informal public opinion polls. The survey of about 50 road users showed that because most people walk towards offices or business centers, they usually carry lighter luggage and are often forced to walk very fast. In addition, at the crosswalk, the speed will not change significantly with the increase of traffic flow, which is due to the unrestricted traffic flow. The observed flow parameters are plotted, and the scatter diagram indicates a wide range of data points that mainly follow the Greenberg logarithmic model.In this study, a frame model of automatic pedestrian and nonmotor vehicle flow detection using image processing technology is designed. On the basis of vehicle detection and vehicle tracking modules commonly used in traditional vehicle video acquisition system, for the convenience of data acquisition of mixed traffic flow, four modules including “feature extraction,” “object recognition,” “object detection,” and “object tracking” are developed for pedestrians, electric motors, and vehicles in mixed traffic flow, and automatic detection system of flow data. And a case of intersection of Liugong Avenue and Heping Road was conducted to evaluate the effect of intelligent traffic systems.


## 2. Proposed Method

### 2.1. Object Detection Based on Gaussian Mixed Model

The basic idea of the Gauss hybrid model is to use multiple Gaussian models as a pixel location model, in order to improve the model solid on the multimodal background. Regarding the background of waving leaves, when the leaves move outside a specific location, the pixel information on the site is represented by a Gauss model. When the leaves are suspended at the site, the other is used [11–13]. The Gaussian model represents the pixel information of the location, so that the pixels in the new picture will be regarded as the background regardless of the matching with the Gaussian model, which can avoid the model taking the shaking leaves as a moving target and increasing the robustness of the model.The basic steps of the hybrid Gaussian model algorithm are as follows.


#### 2.1.1. The Definition of the Pixel Model

(1)Pp=wix,y,t,uix,y,t,σix,y,t2,(2)I=fWeD1+δe,(3)Da=gWdI+δd.Each pixel is described by a number of single models:i=1,2,…,k. The value of k is generally between 3 and 5, which indicates the number of single models in the mixture Gaussian model, wix,y,t represents the weight of each model [14], uix,y,t is the ratio of the height part of the model to the lower part, and σix,y,t represents the correlation between the models.(4)∑i=1kwix,y,t=1.Three parameters (weight, mean, and variance) determine a single model.


#### 2.1.2. Updating the Parameters and Performing Foreground Detection

Step 1: if the pixel value of the picture∑i=1kwix,y,t=1 in the newly read video image sequence matches the feature in the training model library, the new pixel i=1,2,…,k matches Ix,y,t−uix,y,t≤λ⋅σix,y,t the single model. If there is a single model that matches the new pixel, it is judged that the point is the background and enters Step 2; if there is no model matching the new pixel, the point is identified as the foreground and enters Step 3.Step 2: modify the weight of the single model matched with the new pixel,dw=α⋅1−wix,y,t−1, and the weight increment is expressed as follows [15, 16]:(5)wix,y,t=wix,y,t−1+dw,wheredw=α⋅1−wix,y,t−1, and α is weighting factor.Modify the mean and variance of the single model matching the new pixels, as in the single Gaussian model.While Step 2 is completed, the program directly enters Step 4.Step 3: if the new pixel does not match any model and if the current number of individual models has reached the maximum number allowed, then the single model with the least value in the current set of multiple models is removed. Then, delete the original sample attribute that entered the corresponding library so that the new sample attribute remains in the specimen library.A new single model is added. The weight of the new model is a smaller value (0.001 in experiment), the mean value is the new pixel value, and the variance is a given larger value.Step 4: weighting normalization is carried as follows:(6)wix,y,t=wix,y,t∑j=1kwix,y,t,i=1,2,…,k.


#### 2.1.3. Sorting and Deleting of Multiple Single Gaussian Models

In the mixed Gauss background model, each pixel model is composed of multiple single Gaussian models [17–19]. In order to improve the efficiency of the algorithm, we need to sort the single Gauss model according to the importance and delete the nonbackground model in time.We assume that the background model has the following characteristics: heavy weight with high frequency of background occurrence and small variance being with little change in pixel value. Accordingly, we let(7)sort_key=wix,y,tσix,y,t.The process of sorting and deleting is carried out as follows: for each single model, first rank according to the weight of the feature (sort_key). If the weights of the first N single models are satisfied ∑i=1Nwix,y,t>T, then only N single models are used as background models, and other models are deleted; generally, T = 0.7.


### 2.2. Object Tracking Based on Kalman Filtering

In the process of tracking a moving target by a mobile robot, the movement of the target in a unit of time can be thought of as uniform motion, so that the position and speed of a target at a given time can be used to represent the target motion state. To simplify the computational complexity of the algorithm, two Kalman filters can be designed to describe changes in target position and velocity in theX-axis and Y-axis directions, respectively. Next, the application of the Kalman filter in the direction of the X-axis is discussed and the same applies to the direction of the Y-axis.The motion equation of the object is as follows:(8)xk+1=xk+vkT,vk+1=vk+akT.The variables of which are the location, speed, and acceleration of the target in theX-axis direction at t = k. xk indicates the moving distance of the vehicle, vk represents the instantaneous speed of the vehicle, and ak is acceleration. T as the time interval between k frame image and k + 1 frame image can be treated as change value. Equation (8) can be described with matrix as follows: (9)xk+1=xk+vk−1+ak−1TT.The equation of state of the system is as follows:(10)Xk+1=HkXk+Wk.Among them,Xk=xk+vkT, the state vectors of the Kalman filter system are as follows:(11)Hk=1T01,wk=0akT,which are the dynamic noise vector of the system. According to the observation equation, the observation noise is 0, so it is 0. After establishing the state equation and observation equation of the above system, we can use Kalman filtering equation to predict the position of the target in the next frame by recursion method. At t = k time, the target position identified by the target recognition algorithm in the K frame image is recorded [20–22]. When the target appears for the first time, it initializes the filter = [0], according to the observed position of the target.The initial state vector covariance matrix of the system can get a larger value on the diagonal line, and the value is obtained according to the actual measurement situation. However, after a period of filtering start-up, the influence is not large.The predicted positionX˜1′ of the target in the next frame image is calculated by formula (1). In the vicinity of the location, the local image of the next frame is searched, and the centroid position is identified Z1. By updating formula (2) to formula (5), we can update the covariance matrix of the state vector and the state vector, prepare for the next step prediction of the target position, and get the new prediction location. X˜2′ Local search is carried out to get the new centroid position Z2 of the target, which is calculated iteratively to achieve the tracking of the target object.


### 2.3. Feature Extraction Based on Fisher Linear Discriminant Analysis

The basic idea of Fisher linear discriminant analysis (FLD) is to find a projection direction, so that when the training sample is projected to this direction, the maximum interclass distance and minimum intraclass distance can be as large as possible. Later, the FLD method of two kinds of problems was extended to many kinds of cases. Let the pattern categories havec: w1,w2,…,wc, each category has ni training samples; X is the collection of N training samples, X1,…,Xn. The mean subordinates of each category and the mean values of the total sample are, respectively, as follows [23, 24]:(12)μi=1ni∑xk∈ωiXk,(13)μ=1n∑k=1nXk.The within-class scatter matrixSw of the samples is as follows:(14)Sw=∑i=1cSi,where Si=∑xk∈ωiXk−μiXk−μiT is the discrete-time matrix.The between-class scatter matrixSb of the samples is as follows:(15)Sb=∑i=1cμi−μμi−μT.Fisher discriminant function is defined as follows:(16)JFw=wTSbwwTSww,where w denotes the transformation vector from the original sample space to Fisher space. So the system can attain maximum separability between different classes while minimizing the within-class scatter by solving the optimization problem.


## 2.1. Object Detection Based on Gaussian Mixed Model

The basic idea of the Gauss hybrid model is to use multiple Gaussian models as a pixel location model, in order to improve the model solid on the multimodal background. Regarding the background of waving leaves, when the leaves move outside a specific location, the pixel information on the site is represented by a Gauss model. When the leaves are suspended at the site, the other is used [11–13]. The Gaussian model represents the pixel information of the location, so that the pixels in the new picture will be regarded as the background regardless of the matching with the Gaussian model, which can avoid the model taking the shaking leaves as a moving target and increasing the robustness of the model.The basic steps of the hybrid Gaussian model algorithm are as follows.


### 2.1.1. The Definition of the Pixel Model

(1)Pp=wix,y,t,uix,y,t,σix,y,t2,(2)I=fWeD1+δe,(3)Da=gWdI+δd.Each pixel is described by a number of single models:i=1,2,…,k. The value of k is generally between 3 and 5, which indicates the number of single models in the mixture Gaussian model, wix,y,t represents the weight of each model [14], uix,y,t is the ratio of the height part of the model to the lower part, and σix,y,t represents the correlation between the models.(4)∑i=1kwix,y,t=1.Three parameters (weight, mean, and variance) determine a single model.


### 2.1.2. Updating the Parameters and Performing Foreground Detection

Step 1: if the pixel value of the picture∑i=1kwix,y,t=1 in the newly read video image sequence matches the feature in the training model library, the new pixel i=1,2,…,k matches Ix,y,t−uix,y,t≤λ⋅σix,y,t the single model. If there is a single model that matches the new pixel, it is judged that the point is the background and enters Step 2; if there is no model matching the new pixel, the point is identified as the foreground and enters Step 3.Step 2: modify the weight of the single model matched with the new pixel,dw=α⋅1−wix,y,t−1, and the weight increment is expressed as follows [15, 16]:(5)wix,y,t=wix,y,t−1+dw,wheredw=α⋅1−wix,y,t−1, and α is weighting factor.Modify the mean and variance of the single model matching the new pixels, as in the single Gaussian model.While Step 2 is completed, the program directly enters Step 4.Step 3: if the new pixel does not match any model and if the current number of individual models has reached the maximum number allowed, then the single model with the least value in the current set of multiple models is removed. Then, delete the original sample attribute that entered the corresponding library so that the new sample attribute remains in the specimen library.A new single model is added. The weight of the new model is a smaller value (0.001 in experiment), the mean value is the new pixel value, and the variance is a given larger value.Step 4: weighting normalization is carried as follows:(6)wix,y,t=wix,y,t∑j=1kwix,y,t,i=1,2,…,k.


### 2.1.3. Sorting and Deleting of Multiple Single Gaussian Models

In the mixed Gauss background model, each pixel model is composed of multiple single Gaussian models [17–19]. In order to improve the efficiency of the algorithm, we need to sort the single Gauss model according to the importance and delete the nonbackground model in time.We assume that the background model has the following characteristics: heavy weight with high frequency of background occurrence and small variance being with little change in pixel value. Accordingly, we let(7)sort_key=wix,y,tσix,y,t.The process of sorting and deleting is carried out as follows: for each single model, first rank according to the weight of the feature (sort_key). If the weights of the first N single models are satisfied ∑i=1Nwix,y,t>T, then only N single models are used as background models, and other models are deleted; generally, T = 0.7.


## 2.1.1. The Definition of the Pixel Model

(1)Pp=wix,y,t,uix,y,t,σix,y,t2,(2)I=fWeD1+δe,(3)Da=gWdI+δd.Each pixel is described by a number of single models:i=1,2,…,k. The value of k is generally between 3 and 5, which indicates the number of single models in the mixture Gaussian model, wix,y,t represents the weight of each model [14], uix,y,t is the ratio of the height part of the model to the lower part, and σix,y,t represents the correlation between the models.(4)∑i=1kwix,y,t=1.Three parameters (weight, mean, and variance) determine a single model.


## 2.1.2. Updating the Parameters and Performing Foreground Detection

Step 1: if the pixel value of the picture∑i=1kwix,y,t=1 in the newly read video image sequence matches the feature in the training model library, the new pixel i=1,2,…,k matches Ix,y,t−uix,y,t≤λ⋅σix,y,t the single model. If there is a single model that matches the new pixel, it is judged that the point is the background and enters Step 2; if there is no model matching the new pixel, the point is identified as the foreground and enters Step 3.Step 2: modify the weight of the single model matched with the new pixel,dw=α⋅1−wix,y,t−1, and the weight increment is expressed as follows [15, 16]:(5)wix,y,t=wix,y,t−1+dw,wheredw=α⋅1−wix,y,t−1, and α is weighting factor.Modify the mean and variance of the single model matching the new pixels, as in the single Gaussian model.While Step 2 is completed, the program directly enters Step 4.Step 3: if the new pixel does not match any model and if the current number of individual models has reached the maximum number allowed, then the single model with the least value in the current set of multiple models is removed. Then, delete the original sample attribute that entered the corresponding library so that the new sample attribute remains in the specimen library.A new single model is added. The weight of the new model is a smaller value (0.001 in experiment), the mean value is the new pixel value, and the variance is a given larger value.Step 4: weighting normalization is carried as follows:(6)wix,y,t=wix,y,t∑j=1kwix,y,t,i=1,2,…,k.


## 2.1.3. Sorting and Deleting of Multiple Single Gaussian Models

In the mixed Gauss background model, each pixel model is composed of multiple single Gaussian models [17–19]. In order to improve the efficiency of the algorithm, we need to sort the single Gauss model according to the importance and delete the nonbackground model in time.We assume that the background model has the following characteristics: heavy weight with high frequency of background occurrence and small variance being with little change in pixel value. Accordingly, we let(7)sort_key=wix,y,tσix,y,t.The process of sorting and deleting is carried out as follows: for each single model, first rank according to the weight of the feature (sort_key). If the weights of the first N single models are satisfied ∑i=1Nwix,y,t>T, then only N single models are used as background models, and other models are deleted; generally, T = 0.7.


## 2.2. Object Tracking Based on Kalman Filtering

In the process of tracking a moving target by a mobile robot, the movement of the target in a unit of time can be thought of as uniform motion, so that the position and speed of a target at a given time can be used to represent the target motion state. To simplify the computational complexity of the algorithm, two Kalman filters can be designed to describe changes in target position and velocity in theX-axis and Y-axis directions, respectively. Next, the application of the Kalman filter in the direction of the X-axis is discussed and the same applies to the direction of the Y-axis.The motion equation of the object is as follows:(8)xk+1=xk+vkT,vk+1=vk+akT.The variables of which are the location, speed, and acceleration of the target in theX-axis direction at t = k. xk indicates the moving distance of the vehicle, vk represents the instantaneous speed of the vehicle, and ak is acceleration. T as the time interval between k frame image and k + 1 frame image can be treated as change value. Equation (8) can be described with matrix as follows: (9)xk+1=xk+vk−1+ak−1TT.The equation of state of the system is as follows:(10)Xk+1=HkXk+Wk.Among them,Xk=xk+vkT, the state vectors of the Kalman filter system are as follows:(11)Hk=1T01,wk=0akT,which are the dynamic noise vector of the system. According to the observation equation, the observation noise is 0, so it is 0. After establishing the state equation and observation equation of the above system, we can use Kalman filtering equation to predict the position of the target in the next frame by recursion method. At t = k time, the target position identified by the target recognition algorithm in the K frame image is recorded [20–22]. When the target appears for the first time, it initializes the filter = [0], according to the observed position of the target.The initial state vector covariance matrix of the system can get a larger value on the diagonal line, and the value is obtained according to the actual measurement situation. However, after a period of filtering start-up, the influence is not large.The predicted positionX˜1′ of the target in the next frame image is calculated by formula (1). In the vicinity of the location, the local image of the next frame is searched, and the centroid position is identified Z1. By updating formula (2) to formula (5), we can update the covariance matrix of the state vector and the state vector, prepare for the next step prediction of the target position, and get the new prediction location. X˜2′ Local search is carried out to get the new centroid position Z2 of the target, which is calculated iteratively to achieve the tracking of the target object.


## 2.3. Feature Extraction Based on Fisher Linear Discriminant Analysis

The basic idea of Fisher linear discriminant analysis (FLD) is to find a projection direction, so that when the training sample is projected to this direction, the maximum interclass distance and minimum intraclass distance can be as large as possible. Later, the FLD method of two kinds of problems was extended to many kinds of cases. Let the pattern categories havec: w1,w2,…,wc, each category has ni training samples; X is the collection of N training samples, X1,…,Xn. The mean subordinates of each category and the mean values of the total sample are, respectively, as follows [23, 24]:(12)μi=1ni∑xk∈ωiXk,(13)μ=1n∑k=1nXk.The within-class scatter matrixSw of the samples is as follows:(14)Sw=∑i=1cSi,where Si=∑xk∈ωiXk−μiXk−μiT is the discrete-time matrix.The between-class scatter matrixSb of the samples is as follows:(15)Sb=∑i=1cμi−μμi−μT.Fisher discriminant function is defined as follows:(16)JFw=wTSbwwTSww,where w denotes the transformation vector from the original sample space to Fisher space. So the system can attain maximum separability between different classes while minimizing the within-class scatter by solving the optimization problem.


## 3. Key Technology and System Design

Based on the tracking results of the traffic flow data collection from video and image editing, the integrated mixed traffic flow collection framework is proposed according to the traffic flow collection workflow and the characteristics of the mixed traffic objects. Its structure is shown in Figure1; based on the object detection and monitoring unit, the feature extraction unit and the object recognition unit are used to identify pedestrians, motorcycles and vehicles and improve adaptive background extraction and object detection information, as well as obstruction and interference in the monitoring of mixed motion objects, as shown in Figure 1.Figure 1
Structure diagram of vehicle information recognition.


### 3.1. The Characteristic Expression of Mixed Traffic Moving Targets

Effective expression of moving target features is a prerequisite for target recognition and classification. The quality of feature expression not only determines the construction and performance of the classifier model in the subsequent recognition process but also relates to the correctness of the classification output. Good feature attributes should be able to increase the differences between different target categories and narrow the differences between the same categories. How to extract stable features reflecting the nature of the target region from the moving region as input parameters of the recognition system is the key to the study of feature expression.In order to design a video detection algorithm suitable for mixed traffic conditions, the classification between motor vehicles and nonmotor vehicles must be considered. Although the 3D feature classification effect is good, the algorithm complexity is high and the calculation time is long. It is difficult to meet the needs of real-time detection. The plane image feature extraction algorithm is simple and can meet the actual needs of real-time detection of mixed traffic flow.Based on this, this study proposes a feature expression method based on eccentricity vector for mixed traffic flow. In view of the specific problems of event recognition, the morphological characteristics and motion characteristics of the target are taken into account, respectively, and the form and motion characteristics of the target are expressed in order to achieve better target recognition results. As the movement of objects can cause the translation and stretching changes of the features, it will seriously affect the shape recognition of objects. Therefore, it is particularly important to establish a morphological feature representation method with translation, expansion, and rotation invariance. At the same time, in view of the dynamic state of moving objects in event recognition system, we choose the motion on the target time series. Characteristic, further constraints are added to target recognition. After preprocessing the video image, the foreground object is extracted, and the object forms a relatively complete contour. We define the distance between the point on the contour and the center of gravity of the object as the eccentricity and use a set of vectors on the contour as the recognition feature according to the counterclockwise sequence.


### 3.2. Object Tracking Model of Kalman Filtering

Filter is an efficient recursive filter, which is often used for moving target tracking. It is a data processing algorithm based on observation information to derive optimal autoregression for optimal state estimation and state observation, as shown in Figure2. First, a time varying transcendental model is established; then the observation model is established through observation information [25].Figure 2
Object tracking of Kalman filtering.In summary, the implementation of the filter in moving target tracking is as follows.First, initialize the Kalman filter, include the initial position of the moving target, measurement matrix, error covariance, state transition matrix, and noise covariance, and predict the state variables of the moving target. The state variables and observation variables on the moving target are used in the Kalman filter equation set to update the error covariance, gain and predict the position of the current target, and update and iterate the state of the Kalman filter.


### 3.3. Image Classification Model Based on Region Proposal Network

As can be seen from Figure3, the object detector by embedding fully convolutional network in Fast R-CNN is designed, which achieves state-of-the-art intelligent transportation, which decreases the number of region proposal greatly [26–29]. The designed detector consists of four parts.(1)
Convolution layer, VGG-16 network is adopted which include 13 convolution layers, 13 ReLU layers, and 4 pooling, the layer’s input is any size image, its output, i.e., feature map’s size is (M/16) × (N/16), and its number is 512.(2)
Region proposal network layer, that is fully convolutional network which can share weight of CNN, whose input are feature maps, and the region proposals are obtained, where anchors’ numberk is 9 for each sliding position, which are obtained according to 3 scales and 3 aspect ratios, the reg layer has 4k output, i.e., the coordinates of k boxes, and cls layer outputs 2k scores to estimate each proposal being foreground or background.While the RPN is trained, we assign a binary class label (of foreground or background) to each anchor; if IoU overlap of an anchor’ is higher than 0.7, let it be positive; if its IoU ratio is lower than 0.3, assign a negative label; other anchors (0.3 < IoU < 0.7) do not contribute to the training objective.The adopted loss function for RPN is multitask loss, consisting of the outputs of the cls and reg layers, i.e., 2-class softmax loss for classification, whereLcls is log loss over two classes (foreground vs background), and the smooth L1 loss for regression is Lreg (ti, ti∗), which is written as follows:(17)Lpi,ti=1Ncls∑iLclspi,pi∗+λ1Nreg∑ipi∗Lregti,ti∗,wherei is the index of an anchor and pi is the predicted probability of anchor i being an object. If the anchor is positive, pi∗ = 1, and if the anchor is negative, pi∗ = 0. ti is 4 coordinates of the predicted bounding box, and ti∗ is that of the ground-truth box associated with a positive anchor. The second term pi∗Lreg means the regression loss is activated only for positive anchors. And in order to have the both cls and reg terms with equally weight during the training, let λ = 10 in the research [30–32].(3)
RoI pooling layer, whose inputs are feature maps and proposal, and convert input of different sizes proposals to fixed length representations (7 × 7).(4)
The classification and regression layer, whose inputs are proposal feature maps, and whose outputs are the classes and the positions of the proposal regions in the image.Figure 3
Image classification based on region proposal networks.


## 3.1. The Characteristic Expression of Mixed Traffic Moving Targets

Effective expression of moving target features is a prerequisite for target recognition and classification. The quality of feature expression not only determines the construction and performance of the classifier model in the subsequent recognition process but also relates to the correctness of the classification output. Good feature attributes should be able to increase the differences between different target categories and narrow the differences between the same categories. How to extract stable features reflecting the nature of the target region from the moving region as input parameters of the recognition system is the key to the study of feature expression.In order to design a video detection algorithm suitable for mixed traffic conditions, the classification between motor vehicles and nonmotor vehicles must be considered. Although the 3D feature classification effect is good, the algorithm complexity is high and the calculation time is long. It is difficult to meet the needs of real-time detection. The plane image feature extraction algorithm is simple and can meet the actual needs of real-time detection of mixed traffic flow.Based on this, this study proposes a feature expression method based on eccentricity vector for mixed traffic flow. In view of the specific problems of event recognition, the morphological characteristics and motion characteristics of the target are taken into account, respectively, and the form and motion characteristics of the target are expressed in order to achieve better target recognition results. As the movement of objects can cause the translation and stretching changes of the features, it will seriously affect the shape recognition of objects. Therefore, it is particularly important to establish a morphological feature representation method with translation, expansion, and rotation invariance. At the same time, in view of the dynamic state of moving objects in event recognition system, we choose the motion on the target time series. Characteristic, further constraints are added to target recognition. After preprocessing the video image, the foreground object is extracted, and the object forms a relatively complete contour. We define the distance between the point on the contour and the center of gravity of the object as the eccentricity and use a set of vectors on the contour as the recognition feature according to the counterclockwise sequence.


## 3.2. Object Tracking Model of Kalman Filtering

Filter is an efficient recursive filter, which is often used for moving target tracking. It is a data processing algorithm based on observation information to derive optimal autoregression for optimal state estimation and state observation, as shown in Figure2. First, a time varying transcendental model is established; then the observation model is established through observation information [25].Figure 2
Object tracking of Kalman filtering.In summary, the implementation of the filter in moving target tracking is as follows.First, initialize the Kalman filter, include the initial position of the moving target, measurement matrix, error covariance, state transition matrix, and noise covariance, and predict the state variables of the moving target. The state variables and observation variables on the moving target are used in the Kalman filter equation set to update the error covariance, gain and predict the position of the current target, and update and iterate the state of the Kalman filter.


## 3.3. Image Classification Model Based on Region Proposal Network

As can be seen from Figure3, the object detector by embedding fully convolutional network in Fast R-CNN is designed, which achieves state-of-the-art intelligent transportation, which decreases the number of region proposal greatly [26–29]. The designed detector consists of four parts.(1)
Convolution layer, VGG-16 network is adopted which include 13 convolution layers, 13 ReLU layers, and 4 pooling, the layer’s input is any size image, its output, i.e., feature map’s size is (M/16) × (N/16), and its number is 512.(2)
Region proposal network layer, that is fully convolutional network which can share weight of CNN, whose input are feature maps, and the region proposals are obtained, where anchors’ numberk is 9 for each sliding position, which are obtained according to 3 scales and 3 aspect ratios, the reg layer has 4k output, i.e., the coordinates of k boxes, and cls layer outputs 2k scores to estimate each proposal being foreground or background.While the RPN is trained, we assign a binary class label (of foreground or background) to each anchor; if IoU overlap of an anchor’ is higher than 0.7, let it be positive; if its IoU ratio is lower than 0.3, assign a negative label; other anchors (0.3 < IoU < 0.7) do not contribute to the training objective.The adopted loss function for RPN is multitask loss, consisting of the outputs of the cls and reg layers, i.e., 2-class softmax loss for classification, whereLcls is log loss over two classes (foreground vs background), and the smooth L1 loss for regression is Lreg (ti, ti∗), which is written as follows:(17)Lpi,ti=1Ncls∑iLclspi,pi∗+λ1Nreg∑ipi∗Lregti,ti∗,wherei is the index of an anchor and pi is the predicted probability of anchor i being an object. If the anchor is positive, pi∗ = 1, and if the anchor is negative, pi∗ = 0. ti is 4 coordinates of the predicted bounding box, and ti∗ is that of the ground-truth box associated with a positive anchor. The second term pi∗Lreg means the regression loss is activated only for positive anchors. And in order to have the both cls and reg terms with equally weight during the training, let λ = 10 in the research [30–32].(3)
RoI pooling layer, whose inputs are feature maps and proposal, and convert input of different sizes proposals to fixed length representations (7 × 7).(4)
The classification and regression layer, whose inputs are proposal feature maps, and whose outputs are the classes and the positions of the proposal regions in the image.Figure 3
Image classification based on region proposal networks.


## 4. Experiments Conclusions

### 4.1. Target Quantity Statistics

When the number of moving targets is counted, the object detect is carried out, which can be seen from Figure4. The number of moving targets is counted by the vehicle information feature matching method, and the detection line is displayed at the appropriate position of the video image. When two monitoring frames appear on both sides of the detection line, the distance between the vehicles is large enough to be sure to identify the two vehicles, which increases the number of moving targets. This system represents the vehicle with a blue rectangular frame and displays vehicle information around the rectangular frame. Real scenes often contain complex features, such as pedestrians and people pushing cars. The recognition rate of this system is not very high and needs to be strengthened. Table 1 is multiline traffic statistics. From the statistical results, the marking method can accurately measure the number of moving targets based on multitarget tracking, as shown in Table 1.Figure 4
Moving target detection. (a) Cars. (b) Motor bicycles. (c) Pedestrians.
(a)(b)(c)Table 1
Multilane traffic statistics.
Video dataVideo length (minutes)Equipment acquisition (vehicle)Artificial acquisition (feature number)Multiple inspection (frequency)Leak detection (frequency)Accuracy rate (%)Video 15.8669620186.9Video 28.641872010592.3Video 33.951631801184.2


### 4.2. Target Density

Density is an important parameter for traffic management because it can describe the quality of traffic operation and the proximity between the target and the target. The density of traffic flow is the number of moving targets on the driveway in a unit length, and it can also be expressed indirectly by the occupancy rate of vehicles. The results of the test are shown in Table2.Table 2
Test results of moving target density.
Frame numberVehicle number (cars)Car density (vehicles/sec)Pedestrian density (people/sec)Truck density (vehicles/sec)Bicycle density (vehicles/sec)Motorcycle density (vehicles/sec)3829.945.112.685.463.2610932.562.633.22.75.2715720.922.331.181.511.1923929.653.83.843.415.49The detection location is multilane one-way lane, and the time is daylight. The width of each lane is meters, and the length of each lane is meters. According to the statistical method in the previous section, the number of vehicles is obtained and the density calculation is realized.


### 4.3. Target Speed

Velocity calculation of moving target: before moving target detection and tracking, we need to calibrate the camera. The formula for calculating the velocity of a moving target is as follows:(18)v=st.Among them, the moving time and the moving distance are considered, so we must find out the moving distance of the target in the specified time. Pixel 640∗480 for video capture: we need to calculate the actual distance of each row of pixels in the image. w1,w2,w3,…,wN: for each row of pixels after camera imaging, the distance is not equal in the actual detection scene. But after the camera imaging, they should have the same image distance H in the image. Therefore, we need to map each row of pixels to the actual distance according to the actual situation of the test scenario.


## 4.1. Target Quantity Statistics

When the number of moving targets is counted, the object detect is carried out, which can be seen from Figure4. The number of moving targets is counted by the vehicle information feature matching method, and the detection line is displayed at the appropriate position of the video image. When two monitoring frames appear on both sides of the detection line, the distance between the vehicles is large enough to be sure to identify the two vehicles, which increases the number of moving targets. This system represents the vehicle with a blue rectangular frame and displays vehicle information around the rectangular frame. Real scenes often contain complex features, such as pedestrians and people pushing cars. The recognition rate of this system is not very high and needs to be strengthened. Table 1 is multiline traffic statistics. From the statistical results, the marking method can accurately measure the number of moving targets based on multitarget tracking, as shown in Table 1.Figure 4
Moving target detection. (a) Cars. (b) Motor bicycles. (c) Pedestrians.
(a)(b)(c)Table 1
Multilane traffic statistics.
Video dataVideo length (minutes)Equipment acquisition (vehicle)Artificial acquisition (feature number)Multiple inspection (frequency)Leak detection (frequency)Accuracy rate (%)Video 15.8669620186.9Video 28.641872010592.3Video 33.951631801184.2


## 4.2. Target Density

Density is an important parameter for traffic management because it can describe the quality of traffic operation and the proximity between the target and the target. The density of traffic flow is the number of moving targets on the driveway in a unit length, and it can also be expressed indirectly by the occupancy rate of vehicles. The results of the test are shown in Table2.Table 2
Test results of moving target density.
Frame numberVehicle number (cars)Car density (vehicles/sec)Pedestrian density (people/sec)Truck density (vehicles/sec)Bicycle density (vehicles/sec)Motorcycle density (vehicles/sec)3829.945.112.685.463.2610932.562.633.22.75.2715720.922.331.181.511.1923929.653.83.843.415.49The detection location is multilane one-way lane, and the time is daylight. The width of each lane is meters, and the length of each lane is meters. According to the statistical method in the previous section, the number of vehicles is obtained and the density calculation is realized.


## 4.3. Target Speed

Velocity calculation of moving target: before moving target detection and tracking, we need to calibrate the camera. The formula for calculating the velocity of a moving target is as follows:(18)v=st.Among them, the moving time and the moving distance are considered, so we must find out the moving distance of the target in the specified time. Pixel 640∗480 for video capture: we need to calculate the actual distance of each row of pixels in the image. w1,w2,w3,…,wN: for each row of pixels after camera imaging, the distance is not equal in the actual detection scene. But after the camera imaging, they should have the same image distance H in the image. Therefore, we need to map each row of pixels to the actual distance according to the actual situation of the test scenario.


## 5. Conclusions

With the continuous development of urbanization and the continuous growth of people’s travel demand, the travel problem becomes more and more important to people’s daily life. There are still many problems to be further studied, including the following aspects. Although there are many image processing methods, most of them are applied to vehicle volume acquisition. Therefore, how to learn more and better experience from vehicle image detection technology and improve the function of hybrid traffic flow acquisition system based on image processing becomes one of the tasks of the next stage of research work. Because this research involves a lot of content, the goal of this study is to propose a feasible theory and method of video mixed traffic flow data acquisition. How to develop a more robust shadow removal algorithm and hybrid traffic object detection method in high density still needs to be further studied.In this article, the framework of the mixed traffic flow data acquisition system is proposed and the operation of each module is performed. However, this study only provides the theoretical methods and basis for implementing mixed motion video traffic, and there is still a gap with the more mature trading system. Therefore, it is necessary to further integrate all the modules and make a complete acquisition, which is another important task for further research and development. We will conduct research on the dynamic calibration of one-eyed vision to measure the range and speed of vehicles and pedestrians in the future.



---

*Source: 9966494-2021-06-22.xml*