# Short-Term Traffic Flow Prediction with Weather Conditions: Based on Deep Learning Algorithms and Data Fusion

Authors: Yue Hou; Zhiyuan Deng; Hanke Cui
Journal: Complexity
 (2021)

---

## Abstract

Short-term traffic flow prediction is an effective means for intelligent transportation system (ITS) to mitigate traffic congestion. However, traffic flow data with temporal features and periodic characteristics are vulnerable to weather effects, making short-term traffic flow prediction a challenging issue. However, the existing models do not consider the influence of weather changes on traffic flow, leading to poor performance under some extreme conditions. In view of the rich features of traffic data and the characteristic of being vulnerable to external weather conditions, the prediction model based on traffic data has certain limitations, so it is necessary to conduct research studies on traffic flow prediction driven by both the traffic data and weather data. This paper proposes a combined framework of stacked autoencoder (SAE) and radial basis function (RBF) neural network to predict traffic flow, which can effectively capture the temporal correlation and periodicity of traffic flow data and disturbance of weather factors. Firstly, SAE is used to process the traffic flow data in multiple time slices to acquire a preliminary prediction. Then, RBF is used to capture the relation between weather disturbance and periodicity of traffic flow so as to gain another prediction. Finally, another RBF is used for the fusion of the above two predictions on decision level, obtaining a reconstructed prediction with higher accuracy. The effectiveness and robustness of the proposed model are verified by experiments.


---

## Body

## 1. Introduction

In recent years, with the continuous increase of vehicle ownership, the conflict between road resources and travel demand is becoming increasingly acute, which leads to the increasingly serious traffic congestion and even hinders the development of social economy. ITS is an effective means to alleviate traffic congestion, and short-term traffic flow prediction is the key to it. Accurate and timely prediction of traffic flow provides reliable basis of traffic control for governors and meanwhile offers appropriate travel advice for travelers so as to optimize road network and reduce traffic congestion. However, traffic prediction is a sophisticated and nonlinear problem. Traffic flow in reality has obvious temporal correlation and periodicity, but it may evolve in an irregular way under disturbance of weather changes, which makes this problem more challenging.The existing short-term traffic flow prediction models can be mainly divided into 3 categories: statistical models, traditional machine learning models, and deep learning models.Statistical models include historical average (HA) and autoregressive integrated moving average (ARIMA). The former takes the statistical average value at a certain time slip in the past as the predicted value, while the latter establishes mathematical model based on the time series. This kind of method has been widely used for a long time because it can reveal the periodic changes of traffic flow data. In 1970s, Ahmed et al. [1] firstly applied ARIMA on short-term traffic flow prediction problem. After that, some improvements had been made on the ARIMA model. Voort et al. [2] combined Kohonen maps with ARIMA and proposed the KARIMA method to forecast traffic flow. Williams et al. [3] proposed seasonal ARIMA for traffic flow prediction on expressway. Min et al. [4] proposed GSTARIMA for short-term traffic flow prediction in urban network. For further extraction on spatiotemporal correlation, Duan et al. [5] proposed an extended space-time ARIMA for short-term traffic flow estimation. However, statistical models use empirical data for parameter calculation based on transcendental knowledge, which is not suitable to reveal the nonlinearity and uncertainty of traffic flow.Compared with the statistical methods, traditional machine learning methods like support vector machine (SVM) and support vector regression machine (SVR) show stronger function fitting ability in complex and nonlinear traffic flow prediction problem. The essential idea of this kind of method is to transform low-dimensional and linearly inseparable traffic data into high-dimensional and linearly separable expression through kernel function. Hong et al. [6] proposed a SVR traffic flow prediction model employing the hybrid genetic algorithm to determine the suitable combination of parameters. Lou et al. [7] presented the least square SVR algorithm for short-term traffic flow forecasting. Hu et al. [8] used particle swarm optimization (PSO) to determine optimal parameters for SVR for higher precision in short-term traffic flow forecasting problem. Ling et al. [9] proposed multikernel SVM and used adaptive particle swarm optimization (APSO) to improve it. Feng et al. [10] proposed a novel SVM with adaptive multikernel (AMSVM). Although there are a lot of optimization studies on this kind of method, limitation in the regression problem and lack of the ability of knowledge mining for large-scale traffic data still constrain the prediction performance.With the emergence of traffic big data [11], short-term traffic prediction becomes more challenging and complex, which put forward higher requirements for data modeling. Deep learning models, with the effectiveness for high-dimensional space modeling and the ability to extract features of parameters through hierarchical representation, have become the mainstream technology of traffic flow prediction. Deep artificial neural network (ANN) [12], deep belief network (DBN) [13, 14] based on restricted Boltzmann machine, and long short-term memory (LSTM) network [15] for time series problems have been studied and applied to some extent. In addition, Xiao et al. [16] presented a short-term multistep freeway traffic flow prediction model with RBF whose center position of the hidden layer is determined by the fuzzy c-means clustering algorithm. Lv et al. [17] firstly used stacked autoencoder to learn the representation of traffic flow features for prediction. Abdi et al. [18] proposed a novel temporal difference backpropagation (TDBP) method in the training of RBF, which improved the short-term traffic flow prediction accuracy. Dai et al. [19] combined the spatiotemporal analysis with GRU to predict short-term traffic flow.However, single model still has the limitation on the process of complex data. In order to integrate the advantages of single model to achieve more accurate traffic flow forecasting, a variety of combined models have emerged. Hong et al. [20] proposed the ARIMA-ANN combination model, using ARIMA to deal with the linear part of the historical data and ANN for the nonlinear part. Li et al. [21] combined the ARIMA model with the RBF model to capture the different aspects of the underlying patterns of traffic flow. Du et al. [22] proposed a hybrid deep learning framework based on RNN and CNN, which can capture the spatiotemporal dependencies of the traffic flow.The existing short-term traffic flow prediction methods mainly aim at data modeling for traffic flow, and little research has been done on the effect of external conditions like weather on traffic flow. Hall et al. [23] discussed how adverse weather affects traffic flow. Holdener et al. [24] considered the effects of weather condition on rural freeway flow. Jian et al. [25] investigated the microscopic traffic flow parameters under rainy environment. These research studies did reveal part of the correlation between traffic flow and weather conditions, but the conclusions have not been applied to the prediction problem. Koesdwiady et al. [26] incorporated DBNs for more accurate prediction based on flow data and traffic data, and decision-level data fusion of traffic flow and weather data had been realized. Zheng et al. [27] proposed a combined architecture of embedded components, LSTM and CNN, to capture the relationship between traffic flow and weather. However, Koesdwiady et al. [26] did not have a further consideration on the weather decision (traffic prediction based on weather data only), making it hard to achieve high performance in decision-level data fusion. Reference [27] used embedding components to extract the weather disturbance but lacked the analysis and processing of weather parameters.Based on the deficiency of existing methods, research studies on traffic flow prediction driven by both the traffic data and weather data are very important for mining data characteristics of traffic flow and improving the accuracy in prediction. This paper proposes a novel combined framework of SAE and RBF based on traffic flow and weather data. The main contributions are as follows:(1)
Corresponding data processing according to the characteristics of the data: in terms of the non-numerical weather type parameter, we firstly use one-hot coding for the original expression. Then, with an embedding component, the explicable expression is learned. To deal with numerous weather parameters, the Pearson correlation coefficient (PCC) is calculated to find out the flow-related parameters and with principal component analysis (PCA), the selected parameters are processed to be a new parameter with higher correlation. In addition, to integrate time periodicity into prediction, HA is used to construct time expression based on historical traffic flow data.(2)
Incorporating the SAE and RBF to capture the features of traffic flow and weather conditions: considering the effectiveness of combination modeling based on deep learning, we use SAE to learn the temporal correlation in traffic flow, RBF to learn the periodic evolution under weather disturbance, and another RBF to realize the decision-level data fusion of the former models. This combined framework can effectively learn the periodicity and temporal correlation of traffic flow and the disturbance of weather conditions so as to improve the accuracy and robustness of the prediction model.


## 2. Problem Description

Traffic prediction in our research is based on the former parameters of 12 consecutive 5-minute intervals to predict the output flow in any subsequent time slice. The output targety of the prediction model can be expressed by the following formula:(1)y=fX1,X2,X3,…,X10,X11,X12.Considering that the evolution of traffic flow is not only restricted by its own regularity but also disturbed by external weather conditions, the input parameters of the model need to include external weather factors in addition.Xi represents the dataset on the time slice i, including flow xiflow, time expression xitimecode, vector representation of embedding xiembedding, and weather parameters xiweathercondition; Xi can be expressed as(2)Xi=xiflow,xitimecode,xiembedding,xiweathercondition,where yflowrepresents the flow prediction based on traffic sequence and yweather&time represents the flow prediction based on weather and time periodicity. From the perspective of decision-level data fusion, the final flow prediction value is the fusion value of two decisions, so the output y of the combined model can also be expressed as follows:(3)y=ffusionyflow,yweather&time.In multistep prediction,y is represented by yi, and i is the step size. As shown in Figure 1(a), in the sequence-to-sequence framework, the former output is also the input of the next unit, and in this way, the prediction results can be extended to any one time slice. At the beginning, this framework was proposed for machine translation [28]. As shown in Figure 1(b), the proposed model in this paper referred to sequence-to-sequence model for multistep prediction. Considering that the input of the prediction model includes parameters of different types (weather and traffic flow), some of the modules are adjusted to satisfy the modeling demand of multivariate data.Figure 1
Multistep prediction based on sequence-to-sequence framework. (a) Sequence-to-sequence model. (b) Sequence-to-sequence proposed method.
(a)(b)


## 3. Data Process

The data from January 12, 2018, to June 11, 2018, are selected as the training set, and the data from June 17, 2018, to January 12, 2019, are selected as the test set. The primary objective of data mining in our research is to study the data rules from 6:00 to 21:00, which is the peak period of the day.


### 3.1. Traffic Data

The traffic dataset of metro freeways in the Twin Cities is from the Regional Transportation Management Center (https://www.d.umn.edu/tdrl/traffic/). The original data are collected at a 30-second interval from more than 4,500 loop detectors. The No. 644 detector data from January 12, 2018, to January 12, 2019, with less errors and omissions are selected. In the data preprocessing stage, the data are processed into a table with 5-minute interval. Meanwhile, the omissions and error are corrected by using the principle of time similarity. Part of the processed traffic data is shown in Table1.Table 1
Traffic flow dataset (processed).
TimeFlow2018/1/12 6:00672018/1/12 6:0586............2018/6/12 8:051632018/6/12 8:10167............2019/1/12 21:0047To reveal the periodicity of traffic data under weather disturbance, we construct time-flow correlation expression by the HA method. The training set is divided into working days and nonworking days, and the average flow in every certain time slice is counted, which is taken as the representation of time slice. The time-flow correlation expression on time slice can be expressed as follows:(4)xttimecode=1n∑j=0nxi,jflow,where xi,jflow represents the flow of time slice i on day j.


### 3.2. Weather Data

The weather dataset is from the National Oceanic and Atmospheric Administration (https://gis.ncdc.noaa.gov/maps/ncei/lcd). On the map, we select the site with the closest location to detector 644 and the final selection is No. 72658414927. The collection time of weather data corresponds to traffic data. After data preprocessing, part of the weather data is shown in Table2.Table 2
Weather dataset (processed).
TimeWeather typeHourly dew-point temperatureHourly dry-bulb temperatureHourly precipitation..............................2018/1/14 8:15−Sn−6−202018/1/14 8:20−Sn−6−20..............................2018/6/2 15:00−RA61640.01..............................Hourly relative humidityHourly visibilityHourly wet-bulb temperatureHourly wind speed........................836−38836−38........................882.5623........................Weather type is a non-numerical parameter, so we use one-hot coding for the preliminary treatment. However, the sparse representation of one-hot coding cannot reflect the correlation between weather types, resulting in the model not effectively extracting its rich features during training. To solve this problem, an embedding component has been applied to extract the expression of higher dimension of weather type. The embedding vector of weather type can be expressed by the following formula:(5)xembedding=fembeddingxone−hot,where xembedding is the trained embedding vector of weather type, while xone−hot is the one-hot expression; the relation between them is shown in Figure 2.Figure 2
Relation between one-hot coding and embedding.Except for weather type, there are still 7 types of weather parameters. To select the parameters relevant to traffic flow, Pearson correlation coefficientρ as formula (6) is calculated. X and Y stand for two target variables involved in the operation. The PCC of traffic flow and weather parameters is shown in Table 3, and the corresponding heat map is shown in Figure 3.(6)ρX,Y=covX,YσXσY=EX−μXY−μYσXσY.Table 3
PCC of weather parameters and traffic flow.
FlowHourly dew-point temperature0.108352Hourly dry-bulb temperature0.234864Hourly precipitation−0.057385Hourly relative humidity−0.372337Hourly visibility0.143758Hourly wet-bulb temperature0.190240Hourly wind speed0.140027Figure 3
Heat map of PCC of weather parameters and traffic flow.From Table3, hourly dew-point temperature and hourly precipitation have the minimum Pearson correlation coefficient, so only the other five parameters are retained. For further extraction of weather parameters, the mathematical PCA method is used for feature-level data fusion. By linear transformation, the original set of variables with certain correlation is reformed into a new set of independent variables to replace the original ones. By this way, the original information can be retained and the similar information can be removed. Formula (7) is the original matrix A of weather parameters:(7)A=x1DBx1RHx1Visx1WBx1WSx2DBx2RHx2Visx2WBx2WS……………xn−1DBxn−1RHxn−1Visxn−1WBxn−1WSxnDBxnRHxnVisxnWBxnWS,where xDB、 xRH、 xVis、 xWB, and xWS are the selected parameters.After being processed by PCA, the new matrixP is generated as follows:(8)P=fPCAA=x1pcax2pca…xn−1pcaxnpca,where xpca is the fusion value of the selected weather parameters processed by PCA.


## 3.1. Traffic Data

The traffic dataset of metro freeways in the Twin Cities is from the Regional Transportation Management Center (https://www.d.umn.edu/tdrl/traffic/). The original data are collected at a 30-second interval from more than 4,500 loop detectors. The No. 644 detector data from January 12, 2018, to January 12, 2019, with less errors and omissions are selected. In the data preprocessing stage, the data are processed into a table with 5-minute interval. Meanwhile, the omissions and error are corrected by using the principle of time similarity. Part of the processed traffic data is shown in Table1.Table 1
Traffic flow dataset (processed).
TimeFlow2018/1/12 6:00672018/1/12 6:0586............2018/6/12 8:051632018/6/12 8:10167............2019/1/12 21:0047To reveal the periodicity of traffic data under weather disturbance, we construct time-flow correlation expression by the HA method. The training set is divided into working days and nonworking days, and the average flow in every certain time slice is counted, which is taken as the representation of time slice. The time-flow correlation expression on time slice can be expressed as follows:(4)xttimecode=1n∑j=0nxi,jflow,where xi,jflow represents the flow of time slice i on day j.


## 3.2. Weather Data

The weather dataset is from the National Oceanic and Atmospheric Administration (https://gis.ncdc.noaa.gov/maps/ncei/lcd). On the map, we select the site with the closest location to detector 644 and the final selection is No. 72658414927. The collection time of weather data corresponds to traffic data. After data preprocessing, part of the weather data is shown in Table2.Table 2
Weather dataset (processed).
TimeWeather typeHourly dew-point temperatureHourly dry-bulb temperatureHourly precipitation..............................2018/1/14 8:15−Sn−6−202018/1/14 8:20−Sn−6−20..............................2018/6/2 15:00−RA61640.01..............................Hourly relative humidityHourly visibilityHourly wet-bulb temperatureHourly wind speed........................836−38836−38........................882.5623........................Weather type is a non-numerical parameter, so we use one-hot coding for the preliminary treatment. However, the sparse representation of one-hot coding cannot reflect the correlation between weather types, resulting in the model not effectively extracting its rich features during training. To solve this problem, an embedding component has been applied to extract the expression of higher dimension of weather type. The embedding vector of weather type can be expressed by the following formula:(5)xembedding=fembeddingxone−hot,where xembedding is the trained embedding vector of weather type, while xone−hot is the one-hot expression; the relation between them is shown in Figure 2.Figure 2
Relation between one-hot coding and embedding.Except for weather type, there are still 7 types of weather parameters. To select the parameters relevant to traffic flow, Pearson correlation coefficientρ as formula (6) is calculated. X and Y stand for two target variables involved in the operation. The PCC of traffic flow and weather parameters is shown in Table 3, and the corresponding heat map is shown in Figure 3.(6)ρX,Y=covX,YσXσY=EX−μXY−μYσXσY.Table 3
PCC of weather parameters and traffic flow.
FlowHourly dew-point temperature0.108352Hourly dry-bulb temperature0.234864Hourly precipitation−0.057385Hourly relative humidity−0.372337Hourly visibility0.143758Hourly wet-bulb temperature0.190240Hourly wind speed0.140027Figure 3
Heat map of PCC of weather parameters and traffic flow.From Table3, hourly dew-point temperature and hourly precipitation have the minimum Pearson correlation coefficient, so only the other five parameters are retained. For further extraction of weather parameters, the mathematical PCA method is used for feature-level data fusion. By linear transformation, the original set of variables with certain correlation is reformed into a new set of independent variables to replace the original ones. By this way, the original information can be retained and the similar information can be removed. Formula (7) is the original matrix A of weather parameters:(7)A=x1DBx1RHx1Visx1WBx1WSx2DBx2RHx2Visx2WBx2WS……………xn−1DBxn−1RHxn−1Visxn−1WBxn−1WSxnDBxnRHxnVisxnWBxnWS,where xDB、 xRH、 xVis、 xWB, and xWS are the selected parameters.After being processed by PCA, the new matrixP is generated as follows:(8)P=fPCAA=x1pcax2pca…xn−1pcaxnpca,where xpca is the fusion value of the selected weather parameters processed by PCA.


## 4. Proposed Methodology

To capture the features of traffic flow and weather conditions, we propose a combined framework of SAE and RBF, as shown in Figure4. The combined framework can be divided into three modules: the flow prediction module (FPM), the weather and periodicity module (WPM), and the decision-level data fusion module (DDFM).Figure 4
The combined framework of SAE and RBF.


### 4.1. FPM Using SAE

FPM uses SAE to extract the temporal correlation of traffic flow, as shown in Figure5. The stacking of multiple hidden layers can improve the function fitting ability of the neural network for complicated issues. Weights of the former three hidden layers are trained by three different autoencoders (AEs) using backpropagation algorithm (BP algorithm). With these autoencoders, the input is reproduced and SAE learns multiple expressions of original data layer by layer. The input of the SAE is a sequence of flow data in continuous segments, which is denoted as(9)Xflow=xi+1flow,xi+2flow,…,xi+11flow,xi+12flow.Figure 5
SAE structure in FPM.Ignoring the calculation process of the model and focusing on the input and output, the output target of the model can be expressed as(10)yflow=fSAEXflow.To train SAE, firstly, an autoencoder encodes the inputXflow to a sparse representation as shown in the following formula:(11)Xhidden1=Whidden1Xflow,where Xhidden1 is the learned sparse representation of the original input data and Whidden1 is the weight matrix between the input layer and the first hidden layer.Then, with the other two autoencoders, the input is reconstructed again, as shown in formulas (12) and (13).(12)Xhidden2=Whidden2Xhidden1,(13)Xhidden3=Whidden3Xhidden2,where Whidden1, Whidden2, and Whidden3 are transferred to SAE as the weights of first three layers. The weights of the last layer of the SAE are trained separately in all with BP algorithm after weight transfer. In this way, after training, SAE can capture the time correlation of traffic flow data according to the former traffic flow which is close to the predicted value in time.


### 4.2. WPM Using RBF

As shown in Figure6, RBF is a three-layer neural network, consisting of an input layer, hidden layer, and output layer. Unlike general ANN, the transformation of RBF from input space to hidden layer space is nonlinear, while that from hidden layer space to output layer space is linear. The training of the RBF can be summarized with two stages: (a) determine the center of basis function of the hidden layer with unsupervised learning like clustering algorithm and (b) train weights between hidden layer and output layer with supervised learning.Figure 6
The structure of RBF in WPM.The purpose of WPM is to generate the flow prediction with three processed parameters includingxembedding, xtimecode, and xpca, so the function of RBF can be described as follows:(14)yweather&time=fRBFxembedding,xtimecode,xpca.In part (a), the K-means clustering algorithm is applied to find outm cluster centers of the input data as the radial basis of Gaussian kernel function. This procedure is summarized in Algorithm 1.Algorithm 1:K-means clustering.
Input: given the training set ofxembedding,xtimecode,xpca, the number of cluster centers mProcedure:Step 1:m points were selected randomly as clustering centers.Step 2: calculate the distance between every point and them clustering centers, and then the points are divided into the nearest clustering center, thus forming m clusters.Step 3: recalculate the centroid (mean value) of each cluster, formingm new clustering centers.Step 4: repeat step 2 to step 4 until the cluster centers no longer change or the set number of iterations is reached.Output: them cluster centers in the final iteration as C=c1,c2,…,cm−1,cm.The selected set of cluster centers from Algorithm1 is also called the set of radial basis, which can be defined as C=c1,c2,…,cm−1,cm, where ci represents the selected cluster center. Define xi=xiembedding,xitimecode,xipca as the input sequence, and the Gaussian kernel function R is also regarded as the activation function of the network, and the definition is(15)Rxi−cj=exp−12σ2xi−cj2,where σ represents the hyperparameters.With formula (15), linear indivisibility in low dimension becomes linearly separable in high dimension, which is the core idea of kernel function. Thus, nonlinear mapping from input layer to hidden layer is finished. Then, with formula (16), the output target of RBF is determined.(16)yiweather&time=∑j=1hwjRxi−cj,where wj represents trainable weights from hidden layer to output layer.In part (b), to determine the value ofwj, different methods of parameter updating can be attempted, including BP algorithm and pseudoinverse matrix method (PIM). PIM is a network training algorithm similar to BP, but the calculation of PIM is more simple. Since the output of the network is linear to the adjustable parameters, using PIM to solve linear equations directly for the weights is more efficient and accurate than using BP algorithm.Pseudoinverse matrix is a generalized form of inverse matrix, aiming at the singular matrix or nonsquare matrix with no corresponding inverse matrix. UseW to represent the set of wj, Y to represent the output of the network, and X to represent the output matrix of the hidden layer processed by Gaussian kernel activation function. In training stage, we assumed that the input and the output of RBF satisfy the following formula:(17)XW=Y,where Wcan be calculated as(18)W=dotpinvX,Y,where dot is vector dot multiplication and pinvX is the pseudoinverse matrix of X. Algorithm 2 shows the calculation process of pinvX.Algorithm 2:Calculation of pinvX.
Input: matrixXProcedure:Step 1: singular value decomposition ofX:X=UDVT, where U and V are the orthogonal matrices.Step 2: construct the diagonal matrixS:Si,i=0,Di,i==01/Di,i,Di,i!=0Step 3: calculatepinvX:pinvX=VSUTOutput:pinvX


### 4.3. DDFM Using RBF

As shown in Figure7, DDFM is realized by another RBF with the same configuration of that in WPM and differs from its input dimension. Different from the former which used RBF for data fusion on feature level, DDFM is designed for the decision-level data fusion of the outputs of the former modules. RBF is a kind of neural network with simple structure, so there is no need to consider the hierarchical structure when modeling, and it can meet the requirement of data fusion both in feature level and decision level. The output of DDFM is also the final prediction of the combined model, and the definition is(19)y=yfusion=ffusionyflow,yweather&time.Figure 7
The structure of RBF in DDFM.


## 4.1. FPM Using SAE

FPM uses SAE to extract the temporal correlation of traffic flow, as shown in Figure5. The stacking of multiple hidden layers can improve the function fitting ability of the neural network for complicated issues. Weights of the former three hidden layers are trained by three different autoencoders (AEs) using backpropagation algorithm (BP algorithm). With these autoencoders, the input is reproduced and SAE learns multiple expressions of original data layer by layer. The input of the SAE is a sequence of flow data in continuous segments, which is denoted as(9)Xflow=xi+1flow,xi+2flow,…,xi+11flow,xi+12flow.Figure 5
SAE structure in FPM.Ignoring the calculation process of the model and focusing on the input and output, the output target of the model can be expressed as(10)yflow=fSAEXflow.To train SAE, firstly, an autoencoder encodes the inputXflow to a sparse representation as shown in the following formula:(11)Xhidden1=Whidden1Xflow,where Xhidden1 is the learned sparse representation of the original input data and Whidden1 is the weight matrix between the input layer and the first hidden layer.Then, with the other two autoencoders, the input is reconstructed again, as shown in formulas (12) and (13).(12)Xhidden2=Whidden2Xhidden1,(13)Xhidden3=Whidden3Xhidden2,where Whidden1, Whidden2, and Whidden3 are transferred to SAE as the weights of first three layers. The weights of the last layer of the SAE are trained separately in all with BP algorithm after weight transfer. In this way, after training, SAE can capture the time correlation of traffic flow data according to the former traffic flow which is close to the predicted value in time.


## 4.2. WPM Using RBF

As shown in Figure6, RBF is a three-layer neural network, consisting of an input layer, hidden layer, and output layer. Unlike general ANN, the transformation of RBF from input space to hidden layer space is nonlinear, while that from hidden layer space to output layer space is linear. The training of the RBF can be summarized with two stages: (a) determine the center of basis function of the hidden layer with unsupervised learning like clustering algorithm and (b) train weights between hidden layer and output layer with supervised learning.Figure 6
The structure of RBF in WPM.The purpose of WPM is to generate the flow prediction with three processed parameters includingxembedding, xtimecode, and xpca, so the function of RBF can be described as follows:(14)yweather&time=fRBFxembedding,xtimecode,xpca.In part (a), the K-means clustering algorithm is applied to find outm cluster centers of the input data as the radial basis of Gaussian kernel function. This procedure is summarized in Algorithm 1.Algorithm 1:K-means clustering.
Input: given the training set ofxembedding,xtimecode,xpca, the number of cluster centers mProcedure:Step 1:m points were selected randomly as clustering centers.Step 2: calculate the distance between every point and them clustering centers, and then the points are divided into the nearest clustering center, thus forming m clusters.Step 3: recalculate the centroid (mean value) of each cluster, formingm new clustering centers.Step 4: repeat step 2 to step 4 until the cluster centers no longer change or the set number of iterations is reached.Output: them cluster centers in the final iteration as C=c1,c2,…,cm−1,cm.The selected set of cluster centers from Algorithm1 is also called the set of radial basis, which can be defined as C=c1,c2,…,cm−1,cm, where ci represents the selected cluster center. Define xi=xiembedding,xitimecode,xipca as the input sequence, and the Gaussian kernel function R is also regarded as the activation function of the network, and the definition is(15)Rxi−cj=exp−12σ2xi−cj2,where σ represents the hyperparameters.With formula (15), linear indivisibility in low dimension becomes linearly separable in high dimension, which is the core idea of kernel function. Thus, nonlinear mapping from input layer to hidden layer is finished. Then, with formula (16), the output target of RBF is determined.(16)yiweather&time=∑j=1hwjRxi−cj,where wj represents trainable weights from hidden layer to output layer.In part (b), to determine the value ofwj, different methods of parameter updating can be attempted, including BP algorithm and pseudoinverse matrix method (PIM). PIM is a network training algorithm similar to BP, but the calculation of PIM is more simple. Since the output of the network is linear to the adjustable parameters, using PIM to solve linear equations directly for the weights is more efficient and accurate than using BP algorithm.Pseudoinverse matrix is a generalized form of inverse matrix, aiming at the singular matrix or nonsquare matrix with no corresponding inverse matrix. UseW to represent the set of wj, Y to represent the output of the network, and X to represent the output matrix of the hidden layer processed by Gaussian kernel activation function. In training stage, we assumed that the input and the output of RBF satisfy the following formula:(17)XW=Y,where Wcan be calculated as(18)W=dotpinvX,Y,where dot is vector dot multiplication and pinvX is the pseudoinverse matrix of X. Algorithm 2 shows the calculation process of pinvX.Algorithm 2:Calculation of pinvX.
Input: matrixXProcedure:Step 1: singular value decomposition ofX:X=UDVT, where U and V are the orthogonal matrices.Step 2: construct the diagonal matrixS:Si,i=0,Di,i==01/Di,i,Di,i!=0Step 3: calculatepinvX:pinvX=VSUTOutput:pinvX


## 4.3. DDFM Using RBF

As shown in Figure7, DDFM is realized by another RBF with the same configuration of that in WPM and differs from its input dimension. Different from the former which used RBF for data fusion on feature level, DDFM is designed for the decision-level data fusion of the outputs of the former modules. RBF is a kind of neural network with simple structure, so there is no need to consider the hierarchical structure when modeling, and it can meet the requirement of data fusion both in feature level and decision level. The output of DDFM is also the final prediction of the combined model, and the definition is(19)y=yfusion=ffusionyflow,yweather&time.Figure 7
The structure of RBF in DDFM.


## 5. Experimental Results

### 5.1. Evaluation Indexes

To test the performance of the proposed model, a variety of evaluation indexes were used in the experimental stage, including mean absolute percentage error (MAPE), symmetric mean absolute percentage error (SMAPE), mean absolute error (MAE), mean square error (MSE), and root mean square error (RMSE). These metrics mainly reflect the gap between the real values and the predicted values. The specific formulas of these metrics are as follows:(20)MAPE=100%n∑i=1nyipredict−yitrueyitrue,SMAPE=100%n∑i=1nyipredict−yitrueyipredict+yitrue/2,MAE=1n∑i=1nyipredict−yitrue,MSE=1n∑i=1nyipredict−yitrue2,RMSE=1n∑i=1nyipredict−yitrue2.


### 5.2. Configuration and Baselines

The proposed model and the baseline-model are implemented by Python-Keras, the data are normalized by MinMaxScaler, the iteration time is 600, the batch size is 256, the function optimizer selects RMSprop, and the loss function is MSE. According to the actual performance of each model based on the above standards, the relevant parameters are fine tuned. The used baselines include the following:(1)
HA: use historical averages as prediction value.(2)
LSTM: long short-term memory network for time series problem.(3)
GRU: a variant of LSTM with simpler structure.(4)
StackedLSTM: the stacking of LSTM layer.(5)
StackedGRU: the stacking of GRU layer.(6)
SAE: stacked autoencoders, as is described in Section4.(7)
Fusion with ANN (F-ANN): ANN for decision-level data fusion of the outputs of FPM and WPM.(8)
Fusion with RBF on feature and decision level (F-RBF-FDL): RBF for feature and decision-level data fusion of the outputs of FPM and the processed dataxembedding, xtimecode, and xpca.


### 5.3. Performance Comparison in WPM

The purpose of WPM is to capture the periodic evolution and weather disturbance. In data preprocessing stage, efforts have been done to meet the requirements of feature extraction of weather data, including PCC for feature selection and PCA for feature-level data fusion of the selected weather parameters. In addition, the scheme of WPM is selected from numerous experimental schemes. Some controlled trials with different scheme have been done. The comparison is shown in Table4. The specific contents of each scheme are as follows:(a)
RBF + PIM: use RBF without any process on the weather parameters. Use PIM to train the network.(b)
RBF + PCC + PIM: on the basis of scheme a, use PCC to reduce the amount of weather parameters. Use PIM to train the network.(c)
RBF + PCA + PIM: on the basis of scheme a, use PCA to fuse the weather parameters. Use PIM to train the network.(d)
RBF + PCC + PCA + PIM: the fusion of scheme a and scheme b. Use PIM to train the network.(e)
RBF + PCC + PCA + BP: similar to scheme d, but use BP instead of PIM in training.Table 4
Performance comparison in WPM.
ModelMAPE (%)MAEMSERMSE(a) RBF + PIM20.84421114.272642403.53466020.088172(b) RBF + PCC + PIM20.37593514.172910398.24786519.956149(c) RBF + PCA + PIM18.95560414.088218389.71763419.741267(d) RBF + PCC + PCA + PIM18.85725713.988252384.71668419.614196(e) RBF + PCC + PCA + BP21.44594717.124295517.26061222.743364From Table4, our selected scheme (RBF + PCC + PCA + PIM) achieves the best performance in all metrics. Control groups a–d confirm that the model with feature selection by PCC and feature-level data fusion by PCA can get more accurate prediction value. That is, our proposed method (PCC + PCA) for data process of weather parameters is significant for precise prediction by accurate extraction of periodicity and weather disturbance. Comparing experiment d with experiment e, we can conclude that using pseudoinverse matrix for solution (or weights) of linear equation in RBF can achieve higher accuracy in a simpler way and BP algorithm is probably not suitable for the training of RBF.


### 5.4. Performance Comparison between Proposed Model and Baselines in Single Step

The performance comparison between our proposed model and the baselines is shown in Figure8 and Table 5. The proposed model has the optimal performance in all the selected error indexes, with 10.378414% of MAPE, 10.059485% of SMAPE, 9.494741 of MAE, 151.153208 of MSE, and 12.294438 of RMSE. In Table 5, models g, h, and i get better performance in prediction for they all consider periodicity and weather disturbance in modeling. Experiment groups h and i confirm the effectiveness of decision-level data fusion. That is, with the combination modeling idea and selected data fusion scheme, the proposed model can effectively learn the temporal correlation in traffic flow, the time periodicity from historical data, and the disturbance of weather conditions so as to improve the accuracy and robustness. In Figure 8, we divided these methods into two categories: prediction with traffic data only (a–f) and prediction with multivariate data (g–i). The groups g–i which consider the weather conditions show higher accuracy compared with the groups a–f which do not.Figure 8
Performance comparison of different models. Prediction with traffic data only: (a) HA. (b) LSTM. (c) GRU. (d) StackedLSTM. (e) StackedGRU. (f) SAE. Prediction with both the traffic data and weather data: (g) F-ANN. (h) F-RBF-FDL. (i) Ours.
(a)(b)(c)(d)(e)(f)(g)(h)(i)Table 5
Performance comparison of the proposed one and baselines.
ModelMAPE (%)SMAPE (%)MAEMSERMSE(a) HA20.51102217.32034515.748870462.31432221.501496(b) LSTM11.34108410.6925839.998004166.77093012.913982(c) GRU11.89640610.96301410.053730167.20206812.930664(d) StackedLSTM10.57370210.6125389.695501155.60784712.474287(e) StackedGRU10.62816110.3169589.778009158.47903612.588846(f) SAE10.55825110.3977579.841813162.25418212.737903(g) F-ANN10.53703110.1343559.511839151.19047712.295954(h) F-RBF-FDL10.52175810.1344879.507794151.52827312.309682(i) Ours10.37841410.0594859.494741151.15320812.294438


### 5.5. Performance Comparison between Proposed Model and Baselines in Multiple Steps

In Figure9, with the increase of time step size, an upward trend of prediction error occurred in all models for the accumulated error from the last step, but the proposed model is still superior to baselines in most indicators. Compared with experiments in single step, the proposed model shows more obvious advantages of prediction accuracy in multiple steps. With the increase of prediction time span, the accuracy gap between models will be further widened, making the advantages of proposed model more obvious. From Table 6, we can see the indicators of the models in different steps. Models f and g achieve more precise prediction value when compared to the other models because they combine time periodicity with weather disturbance in modeling. Comparing f with g, g is slightly better than f in most indicators and most step sizes. In Step 05 and Step 06, f shows slightly better performance in MSE and RMSE than g because time periodicity in modeling is more important for the prediction accuracy in long-term prediction. That is, the proposed model is more stable and precise no matter in single step or in multiple step.Figure 9
The changing indicators of models in multiple steps.
(a)(b)(c)(d)Table 6
Performance comparison in multiple steps.
Prediction stepModelMAPE (%)SMAPE (%)MAEMSERMSEStep 01 (5 min)(a) LSTM11.3410.699.99166.7712.91(b) GRU11.8910.9610.05167.2012.93(c) StackedLSTM10.5710.619.69155.6012.47(d) StackedGRU10.6210.319.77158.4712.58(e) SAE10.5510.399.84162.2512.73(f) F-ANN10.5310.139.51151.1912.29(g) Ours10.3710.059.49151.1512.29Step 02 (10 min)(a) LSTM13.1912.0011.20215.4914.67(b) GRU13.5512.1811.20214.4014.64(c) StackedLSTM12.1511.7510.69195.1613.97(d) StackedGRU12.1411.3910.76197.5514.05(e) SAE12.1511.6710.99208.8314.45(f) F-ANN12.0511.1610.35182.0613.49(g) Ours11.5810.9410.24180.1913.42Step 03 (15 min)(a) LSTM15.0713.2112.34264.9016.27(b) GRU15.2913.3612.32263.1816.22(c) StackedLSTM13.9212.9411.64234.7715.32(d) StackedGRU13.7512.4111.70236.7815.38(e) SAE13.8212.9112.14258.0016.06(f) F-ANN13.2612.0311.14213.0914.59(g) Ours12.9211.8411.03211.2814.53Step 04 (20 min)(a) LSTM16.7714.3113.36314.9117.74(b) GRU16.8614.4513.33313.5817.70(c) StackedLSTM15.5114.0512.51273.8816.54(d) StackedGRU15.2113.3512.53274.6516.57(e) SAE15.3314.0913.20311.0117.63(f) F-ANN14.3712.8611.90246.1915.69(g) Ours14.1212.7111.79244.9915.65Step 05 (25 min)(a) LSTM18.3215.4114.38372.7519.30(b) GRU18.2915.5514.34372.2319.29(c) StackedLSTM16.7215.0513.32318.8817.85(d) StackedGRU16.4814.2813.36319.3717.87(e) SAE16.6615.2714.26373.0919.31(f) F-ANN15.3413.6712.65284.6716.87(g) Ours15.1813.5912.55285.8016.90Step 06 (30 min)(a) LSTM19.7516.5015.40434.1920.83(b) GRU19.6216.6615.36434.8920.85(c) StackedLSTM17.9316.1014.19370.3219.24(d) StackedGRU17.6415.2314.22369.0019.20(e) SAE17.8516.4615.33441.4421.01(f) F-ANN16.2014.5013.42326.7118.07(g) Ours16.1214.4713.34331.3318.20


## 5.1. Evaluation Indexes

To test the performance of the proposed model, a variety of evaluation indexes were used in the experimental stage, including mean absolute percentage error (MAPE), symmetric mean absolute percentage error (SMAPE), mean absolute error (MAE), mean square error (MSE), and root mean square error (RMSE). These metrics mainly reflect the gap between the real values and the predicted values. The specific formulas of these metrics are as follows:(20)MAPE=100%n∑i=1nyipredict−yitrueyitrue,SMAPE=100%n∑i=1nyipredict−yitrueyipredict+yitrue/2,MAE=1n∑i=1nyipredict−yitrue,MSE=1n∑i=1nyipredict−yitrue2,RMSE=1n∑i=1nyipredict−yitrue2.


## 5.2. Configuration and Baselines

The proposed model and the baseline-model are implemented by Python-Keras, the data are normalized by MinMaxScaler, the iteration time is 600, the batch size is 256, the function optimizer selects RMSprop, and the loss function is MSE. According to the actual performance of each model based on the above standards, the relevant parameters are fine tuned. The used baselines include the following:(1)
HA: use historical averages as prediction value.(2)
LSTM: long short-term memory network for time series problem.(3)
GRU: a variant of LSTM with simpler structure.(4)
StackedLSTM: the stacking of LSTM layer.(5)
StackedGRU: the stacking of GRU layer.(6)
SAE: stacked autoencoders, as is described in Section4.(7)
Fusion with ANN (F-ANN): ANN for decision-level data fusion of the outputs of FPM and WPM.(8)
Fusion with RBF on feature and decision level (F-RBF-FDL): RBF for feature and decision-level data fusion of the outputs of FPM and the processed dataxembedding, xtimecode, and xpca.


## 5.3. Performance Comparison in WPM

The purpose of WPM is to capture the periodic evolution and weather disturbance. In data preprocessing stage, efforts have been done to meet the requirements of feature extraction of weather data, including PCC for feature selection and PCA for feature-level data fusion of the selected weather parameters. In addition, the scheme of WPM is selected from numerous experimental schemes. Some controlled trials with different scheme have been done. The comparison is shown in Table4. The specific contents of each scheme are as follows:(a)
RBF + PIM: use RBF without any process on the weather parameters. Use PIM to train the network.(b)
RBF + PCC + PIM: on the basis of scheme a, use PCC to reduce the amount of weather parameters. Use PIM to train the network.(c)
RBF + PCA + PIM: on the basis of scheme a, use PCA to fuse the weather parameters. Use PIM to train the network.(d)
RBF + PCC + PCA + PIM: the fusion of scheme a and scheme b. Use PIM to train the network.(e)
RBF + PCC + PCA + BP: similar to scheme d, but use BP instead of PIM in training.Table 4
Performance comparison in WPM.
ModelMAPE (%)MAEMSERMSE(a) RBF + PIM20.84421114.272642403.53466020.088172(b) RBF + PCC + PIM20.37593514.172910398.24786519.956149(c) RBF + PCA + PIM18.95560414.088218389.71763419.741267(d) RBF + PCC + PCA + PIM18.85725713.988252384.71668419.614196(e) RBF + PCC + PCA + BP21.44594717.124295517.26061222.743364From Table4, our selected scheme (RBF + PCC + PCA + PIM) achieves the best performance in all metrics. Control groups a–d confirm that the model with feature selection by PCC and feature-level data fusion by PCA can get more accurate prediction value. That is, our proposed method (PCC + PCA) for data process of weather parameters is significant for precise prediction by accurate extraction of periodicity and weather disturbance. Comparing experiment d with experiment e, we can conclude that using pseudoinverse matrix for solution (or weights) of linear equation in RBF can achieve higher accuracy in a simpler way and BP algorithm is probably not suitable for the training of RBF.


## 5.4. Performance Comparison between Proposed Model and Baselines in Single Step

The performance comparison between our proposed model and the baselines is shown in Figure8 and Table 5. The proposed model has the optimal performance in all the selected error indexes, with 10.378414% of MAPE, 10.059485% of SMAPE, 9.494741 of MAE, 151.153208 of MSE, and 12.294438 of RMSE. In Table 5, models g, h, and i get better performance in prediction for they all consider periodicity and weather disturbance in modeling. Experiment groups h and i confirm the effectiveness of decision-level data fusion. That is, with the combination modeling idea and selected data fusion scheme, the proposed model can effectively learn the temporal correlation in traffic flow, the time periodicity from historical data, and the disturbance of weather conditions so as to improve the accuracy and robustness. In Figure 8, we divided these methods into two categories: prediction with traffic data only (a–f) and prediction with multivariate data (g–i). The groups g–i which consider the weather conditions show higher accuracy compared with the groups a–f which do not.Figure 8
Performance comparison of different models. Prediction with traffic data only: (a) HA. (b) LSTM. (c) GRU. (d) StackedLSTM. (e) StackedGRU. (f) SAE. Prediction with both the traffic data and weather data: (g) F-ANN. (h) F-RBF-FDL. (i) Ours.
(a)(b)(c)(d)(e)(f)(g)(h)(i)Table 5
Performance comparison of the proposed one and baselines.
ModelMAPE (%)SMAPE (%)MAEMSERMSE(a) HA20.51102217.32034515.748870462.31432221.501496(b) LSTM11.34108410.6925839.998004166.77093012.913982(c) GRU11.89640610.96301410.053730167.20206812.930664(d) StackedLSTM10.57370210.6125389.695501155.60784712.474287(e) StackedGRU10.62816110.3169589.778009158.47903612.588846(f) SAE10.55825110.3977579.841813162.25418212.737903(g) F-ANN10.53703110.1343559.511839151.19047712.295954(h) F-RBF-FDL10.52175810.1344879.507794151.52827312.309682(i) Ours10.37841410.0594859.494741151.15320812.294438


## 5.5. Performance Comparison between Proposed Model and Baselines in Multiple Steps

In Figure9, with the increase of time step size, an upward trend of prediction error occurred in all models for the accumulated error from the last step, but the proposed model is still superior to baselines in most indicators. Compared with experiments in single step, the proposed model shows more obvious advantages of prediction accuracy in multiple steps. With the increase of prediction time span, the accuracy gap between models will be further widened, making the advantages of proposed model more obvious. From Table 6, we can see the indicators of the models in different steps. Models f and g achieve more precise prediction value when compared to the other models because they combine time periodicity with weather disturbance in modeling. Comparing f with g, g is slightly better than f in most indicators and most step sizes. In Step 05 and Step 06, f shows slightly better performance in MSE and RMSE than g because time periodicity in modeling is more important for the prediction accuracy in long-term prediction. That is, the proposed model is more stable and precise no matter in single step or in multiple step.Figure 9
The changing indicators of models in multiple steps.
(a)(b)(c)(d)Table 6
Performance comparison in multiple steps.
Prediction stepModelMAPE (%)SMAPE (%)MAEMSERMSEStep 01 (5 min)(a) LSTM11.3410.699.99166.7712.91(b) GRU11.8910.9610.05167.2012.93(c) StackedLSTM10.5710.619.69155.6012.47(d) StackedGRU10.6210.319.77158.4712.58(e) SAE10.5510.399.84162.2512.73(f) F-ANN10.5310.139.51151.1912.29(g) Ours10.3710.059.49151.1512.29Step 02 (10 min)(a) LSTM13.1912.0011.20215.4914.67(b) GRU13.5512.1811.20214.4014.64(c) StackedLSTM12.1511.7510.69195.1613.97(d) StackedGRU12.1411.3910.76197.5514.05(e) SAE12.1511.6710.99208.8314.45(f) F-ANN12.0511.1610.35182.0613.49(g) Ours11.5810.9410.24180.1913.42Step 03 (15 min)(a) LSTM15.0713.2112.34264.9016.27(b) GRU15.2913.3612.32263.1816.22(c) StackedLSTM13.9212.9411.64234.7715.32(d) StackedGRU13.7512.4111.70236.7815.38(e) SAE13.8212.9112.14258.0016.06(f) F-ANN13.2612.0311.14213.0914.59(g) Ours12.9211.8411.03211.2814.53Step 04 (20 min)(a) LSTM16.7714.3113.36314.9117.74(b) GRU16.8614.4513.33313.5817.70(c) StackedLSTM15.5114.0512.51273.8816.54(d) StackedGRU15.2113.3512.53274.6516.57(e) SAE15.3314.0913.20311.0117.63(f) F-ANN14.3712.8611.90246.1915.69(g) Ours14.1212.7111.79244.9915.65Step 05 (25 min)(a) LSTM18.3215.4114.38372.7519.30(b) GRU18.2915.5514.34372.2319.29(c) StackedLSTM16.7215.0513.32318.8817.85(d) StackedGRU16.4814.2813.36319.3717.87(e) SAE16.6615.2714.26373.0919.31(f) F-ANN15.3413.6712.65284.6716.87(g) Ours15.1813.5912.55285.8016.90Step 06 (30 min)(a) LSTM19.7516.5015.40434.1920.83(b) GRU19.6216.6615.36434.8920.85(c) StackedLSTM17.9316.1014.19370.3219.24(d) StackedGRU17.6415.2314.22369.0019.20(e) SAE17.8516.4615.33441.4421.01(f) F-ANN16.2014.5013.42326.7118.07(g) Ours16.1214.4713.34331.3318.20


## 6. Conclusion

A combined framework of SAE and RBF is proposed for short-term traffic flow prediction based on traffic data and weather data. Before modeling for traffic prediction, lots of work for data processing are involved in our experiment. For precise numerical representation for time periodicity, HA is used to create time expression based on historical flow. In terms of weather data, one-hot coding and embedding component are used for numerical expression of weather type while PCC and PCA are applied to the feature-level data fusion of weather parameters. After data processing, we incorporate SAE and RBF to capture the features of traffic flow and weather conditions. The final prediction considers the temporal correlation, time periodicity, and weather disturbance, owning higher accuracy and robustness. Quantities of experiments have been done to test the performance of the proposed model from different aspects, confirming that the research of our short-term traffic flow prediction model driven by both the traffic data and weather data is valuable.



---

*Source: 6662959-2021-01-16.xml*