# Hard Disk Failure Prediction Based on Blending Ensemble Learning
## 2 Ensemble Learning  
Ensemble learning, which is an important branch of machine learning, emerged early in 1988 [12]. Its basic idea is to make use of N weak learners with a weak learning effect as the base learners, integrate their results according to different rules, and treat the whole as a strong learner to make the final results more accurate. Bagging and boosting are commonly used ensemble learning strategies. The former was put forward by [PERSON] of Berkeley University as early as 1996 [13]. The strategy of training sample perturbation involves training multiple different base learners in parallel under the premise of multiple random samples put back, then finally taking the plurality or average of all results to obtain a final result according to the actual situation. Later, in 1998, [PERSON] [14] proposed the random subspace method (RSM), which uses the strategy of input attribute perturbation to train the base classifier by randomly selecting some features instead of all features in each training and obtains the final result by taking the average or the plurality of the results of each base classifier. Combining the above two random perturbation approaches, [PERSON] [15] proposed the random forest algorithm in 2001, which has been used as a representative algorithm \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Method** & **Precision** & **Recall** & **FAR** & **FDR** & **MCC** & **MAE** \\ \hline
**Mathematical Statistical**[3] & 40–60\% & - & 0.2–0.5\% & - & - & - \\
**Switchable state-space degradation model**[4] & - & - & 0.83\% & 97.44\% & - & - \\
**LSTM**[1] & - & - & 1.3\% & 85\% & - & - \\
**LSTM**[6] & - & - & - & - & 0.71 & - \\
**Bi-LSTM**[2] & - & - & - & - & - & 0.12 \\
**TCN**[7] & 91\% & - & 0.05\% & 89.1\% & - & - \\
**NTAM**[8] & 84.01\% & 76.43\% & - & - & - & - \\
**LSTM**[9] & 65\% & 89\% & - & - & 0.74 & - \\
**CNN-LSTM**[9] & 95\% & 95\% & - & - & 0.95 & - \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Performance of related work. of the bagging strategy until now. Correspondingly, the boosting method proposed by [PERSON] in 1990 [16] also follows the same strategy of training sample perturbation. The basic idea is to increase the weight of misclassified samples after each training of the base learner to train the next base learner until a strong learner with good results is obtained. Because the base learners need to be trained serially, there is a strong dependency between the base learners. Taking the representative algorithm AdaBoost as an example, its final result can be the weighted voting of each base learner. Other algorithms are GBDT and its improved versions such as XGBoost and LightGBM. In addition to this, there are strategies such as algorithmic parameter perturbation, output label perturbation, or mixing different perturbation methods to train different base classifiers. Several of these representative algorithms will be presented in Section 3. In addition to the two common ensemble learning strategies described above, there are two extended strategies, which are stacking [17] and blending, the technique used in this paper. The principles of the two ensemble learning strategies are basically the same. Stacking is divided into two layers; the first layer uses k-fold cross-validation of the training sets using M base learners to obtain k results on the verification sets and k results on the test sets. The k results are then grouped and spliced to obtain new training sets and test sets [18]. The obtained new training set and test set discard the original features of the data and replace them with the new features of M columns, that is, the prediction results on M base learners. Finally, the new dataset is trained and tested on the second layer using another learner to obtain the final results. The blending method abandoned k-fold cross-verification on the basis of stacking, which can simplify the algorithm process and greatly reduce the running time, while the training effects are not significantly different.