# Short-Term Traffic Flow Forecasting via Multi-Regime Modeling and Ensemble Learning
## 3 Methodology
### Forecasting Modeling via Ensemble Learning  
Given the historical traffic flow observations and the constructed features, the training data set and the test data set can be separately established. After that, the model to forecast the short-term traffic flow needs to be built. In this study, a forecasting model based on an ensemble learning strategy is developed using the obtained training set. Ensemble learning is a very popular and useful technique in machine learning [46], which can be utilized to improve the generalization performance of the forecasting models, and has achieved great success in various domains. The training and test procedure of the developed forecasting model is as follows:  
Step 1: Randomly draw an instance \(I_{h}(1\leq h\leq n)\) from the training data set \(D_{T}\) and add it to a new training data set \(D_{NT}\). Next, return the instance to \(D_{T}\). Repeat the sampling process \(n\) times and generate the final \(D_{NT}\). That is, \(D_{NT}=\{I_{1},I_{2},\ldots,I_{n}\}\).  
Step 2: Use \(D_{NT}\) to train an unpruned regression tree, denoted as \(T_{\alpha}(1\leq\alpha\leq K)\). The training procedure is as follows: for each node of the regression tree, \(\varphi\), randomly sample \(\beta\) features from the constructed features, and use them to compute the best split that has the maximum mean decrease in impurity. The impurity of the \(r(1\leq r\leq\beta)\)th sampled feature, \(x_{r}\), associated with node \(\varphi\) is defined as follows:  
\[\text{IMP}(\varphi)=\frac{\sum_{x_{r}\in\varphi}{(y_{r}(\varphi)-\overline{y} (\varphi))^{2}}}{n_{\varphi}}, \tag{6}\]  
where \(x_{r}\in\varphi\) is the value of \(x_{r}\) of the training samples, which falls into the range of the split at node \(\varphi\); \(y_{r}(\varphi)\) is the response value of the training samples associated with \(x_{r}\in\varphi\); \(\overline{y}(\varphi)\) is the mean of \(y_{r}(\varphi)\); and \(n_{\varphi}\) is the number of the training samples associated with \(x_{r}\in\varphi\).  
In the above training procedure, if \(\varphi\) is a leaf node, \(\overline{y}(\varphi)\) is set as the forecast of the node.  
Step 3: Repeat Step 1 and Step 2 \(K\) times to train a number of \(K\) regression trees.  
Step 4: Given the instance in the established test data set \(D_{E}\), denoted as \(I_{E}(I_{E}\in D_{E})\), the forecasts from the \(K\) regression trees are first computed. Assume the forecasts are \(f_{1}(I_{E})\), \(f_{2}(I_{E})\),..., \(f_{K}(I_{E})\), respectively. The final forecast of the instance is calculated as the median of the forecasts of the \(K\) regression trees, as follows:  
\[f(I_{E})=\text{median}(f_{1}(I_{E}),f_{2}(I_{E}),\ldots,f_{K}(I_{E})). \tag{7}\]  
Note that the median rule instead of the average rule is used to combine the outputs of the regression trees, in that the median rule can yield more robust forecasts in the cases where there are some outliers in the leaf nodes [47].  
\begin{table}
\begin{tabular}{c c} \hline
**Representative Features** & **Descriptions** \\ \hline state\_5 & Regime at time \(t-1\) \\ state\_10 & Regime at time \(t-2\) \\ state\_15 & Regime at time \(t-3\) \\ f\_5 & Flow rate at time \(t-1\) \\ f\_10 & Flow rate at time \(t-2\) \\ f\_15 & Flow rate at time \(t-3\) \\ s\_5 & Speed at time \(t-1\) \\ s\_10 & Speed at time \(t-2\) \\ s\_15 & Speed at time \(t-3\) \\ o\_5 & Occupancy at time \(t-1\) \\ o\_10 & Occupancy at time \(t-2\) \\ o\_15 & Occupancy at time \(t-3\) \\ \hline \end{tabular}
\end{table}
Table 1: Constructed features for forecasting modeling.