# Biogas Production Prediction Based on Feature Selection and Ensemble Learning
## 3 Results and Discussion
### Prediction Performance Analysis  
Theoretically, the stacking ensemble model should have a lower prediction error than a single model. This is because the stacking model combines the strengths of multiple primary learners and the combined strategy based on the learning method learns the data further on the meta-learner for better results. As a result, the stacking model improves the poor generalization performance of the data using a single model. At the same time, during the model training process, there is a situation where a single model falls into local minima. However, the stacking ensemble model can effectively reduce the risk of falling into local minima by combining the operating effects of multiple models. In summary, the stacking ensemble model improves the prediction accuracy and model generalization ability.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Meta-Learner** & **MSE** & **MAE** & **MAPE** \\ \hline
**XGBoost** & **74.489** & **3.658** & **0.220** \\ CatBoost & 80.054 & 3.699 & 0.223 \\ LightGBM & 86.994 & 4.709 & 0.238 \\ LSTM & 107.86 & 5.07 & 0.3022 \\ RF & 98.64 & 4.21 & 0.2520 \\ SVR & 118.03 & 5.19 & 0.3244 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Evaluation metrics for the different meta-learner stacking models.  
Figure 7: Correlation of the forecasting error for different models.  
To further assess the impact of the variability among primary learners on the overall prediction effect of the stacking model, the prediction errors of the stacking model for other combinations of primary learners are given in Table 3. The meta-learner of each model in Table 3 is XGBoost, and the combination of models with a high correlation of different Pearson errors is chosen. The average error correlation coefficient is the average of the error Pearson correlation coefficients between the meta-learner and the primary learner. Model 1 is the best combination of primary learners obtained from the previous analysis.  
From Table 3, different combinations of primary learners have a certain degree of influence on the overall prediction effect of the stacking ensemble model. The lower the average error correlation coefficient is, the smaller the prediction error of the obtained model. So, choosing a less correlated learner as the primary learner can make the prediction performance of the stacking ensemble learning model better. There are two reasons for this analysis: choosing the learner with a smaller error for integration can improve the overall prediction ability of the ensemble model; and different models are trained to process the data from different perspectives, so choosing the model with greater variability as the primary learner can give full play to the advantages of different models and improve the prediction effect of the stacking ensemble model. Figure 8 shows the histogram of the prediction error distribution obtained by the proposed method. After comparing Figure 8 with Figure 6, the prediction error of the stacking ensemble learning model is concentrated in the interval \([-4,4]\), and the overall prediction accuracy is higher than that of a single method.