# Research on Ensemble Learning-Based Feature Selection Method for Time-Series Prediction
## 4 Experimental Results and Analysis of Feature Selection Based on Ensemble Learning
### Experimental Results and Analysis
The input gate determines which information will be written to the memory cell, the forget gate decides which information will be removed from the memory cell, and the output gate determines which information will be extracted from the memory cell. These gates govern the flow of information in and out, and the updating of the memory state within the cell. The primary computational processes of an LSTM network can be represented by the following equations:  
\[ \begin{split} f_{t}&=\sigma\Big{(}W_{f}\cdot[h_{t- 1},x_{t}]+b_{f}\Big{)}\\ i_{t}&=\sigma(W_{i}\cdot[h_{t-1},x_{t}]+b_{i})\\ \bar{C}_{t}&=\tanh(W_{\mathbb{C}}\cdot[h_{t-1},x_{ t}]+b_{\mathbb{C}})\\ C_{t}&=f_{t}\cdot C_{t-1}+i_{t}\cdot\bar{C}_{t}\\ o_{t}&=\sigma(W_{o}\cdot[h_{t-1},x_{t}]+b_{o})\\ h_{t}&=o_{t}\cdot\tanh(C_{t})\end{split}  \tag{10}\] \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Feature Selection Methods** & **Feature Subset of ICBC Stock** & **Feature Subset of GD Power** & **Feature Subset of China** \\  & **Data** & **Development Stock Data** & **Unicom Stock Data** \\ \hline \multirow{2}{*}{GBDT} & avg\_price, pe, float\_mv, low, & low, avg\_price, total\_mv, & high, total\_mv, avg\_price, \\  & high & float\_mv, high & float\_mv, low \\ \hline \multirow{2}{*}{RFE} & avg\_price, float\_mv, low, pe, & low, avg\_price, total\_mv, & high, total\_mv, avg\_price, \\  & pre\_close & float\_mv, high & float\_mv, low \\ \hline \multirow{2}{*}{XGBoost} & low, pe, avg\_price, float\_mv, & low, avg\_price, float\_mv, & high, float\_mv, avg\_price, \\  & open & open & open & total\_mv, low \\ \hline \multirow{2}{*}{Pearson} & float\_mv, avg\_price, open, & avg\_price, low, high, pre\_close, open & high, total\_mv, avg\_price, low, \\  & high & float\_mv, high & float\_mv, high & open \\ \hline \multirow{2}{*}{RF} & float\_mv, avg\_price, low, pe, & low, avg\_price, total\_mv, & high, float\_mv, avg\_price, \\  & high & float\_mv, high & total\_mv, low \\ \hline \multirow{2}{*}{Ensemble Learning} & avg\_price, float\_mv, low, pe, & low, avg\_price, total\_mv, & high, float\_mv, avg\_price, \\  & pre\_close & float\_mv, high & total\_mv, low \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The feature subsets obtained through various feature selection methods. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Feature Selection Methods** & **ICBC** & **GD Power** & **China Unicom** \\ \hline GBDT & 0.21 & 0.2 & 0.21 \\ RFE & 0.17 & 0.21 & 0.2 \\ XGBoost & 0.21 & 0.22 & 0.2 \\ Pearson & 0.2 & 0.16 & 0.18 \\ RF & 0.21 & 0.21 & 0.21 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Table of weights for feature selection methods. where \(f_{t}\) represents the output of the forget gate, \(i_{t}\) is the output of the input gate, \(\tilde{C}_{t}\) is the new candidate memory, \(C_{t}\) is the current state of the memory cell, \(o_{t}\) is the output of the output gate, \(h_{t}\) is the hidden state of the LSTM, and \(W_{f},W_{i},W_{C},W_{o}\) are weight matrices, while \(b_{f},b_{i},b_{C},b_{o}\) are bias vectors. The symbol \(\sigma\) denotes the sigmoid activation function. These equations describe the primary computational processes of the LSTM network, allowing the network to more effectively capture long-term dependencies when handling time-series data. For the evaluation of prediction performance, we utilized widely used metrics, namely, Mean Absolute Error (MAE) and Mean Squared Error (MSE). The model parameters are specified as outlined in Table 5.  
The parameter input_size represents the dimensionality of the input features, indicating the number of features input to the model at each time step. In the context of this predictive task, each time step comprised five features, making this parameter equal to 5. Hidden_size denotes the number of hidden units. In LSTM, these units capture patterns and relationships in time-series data. With 64 hidden units, the model exhibited a more complex learning capacity. Num_layers determines the depth of the network, i.e., the number of stacked LSTM layers.