# Construction of Ensemble Learning Model for Home Appliance Demand Forecasting
## 3 Experiment and Discussion
### Experimental Analysis  
In this study, LSTM, Random Forest, XGBoost, SVR, K-NN, and the LSTM-RF-XGBoost combination methods are used to forecast the demand for air conditioning products. Among them, LSTM can handle nonlinear features better, while also dealing with multivariate time series problems and improving the fitting accuracy. When dealing with complex nonlinear time series, introducing gating units successfully overcomes the problem of vanishing gradients and eliminates the step of differential smoothing. The Random Forest model can effectively simulate a variety of complicated non-linear regressions using two methods: data self-sampling and feature random sampling, resulting in a greater generalization effect while simultaneously avoiding overfitting. XGBoost calculates the minimal loss function using the second-order Taylor expansion to determine the splitting node and achieve the optimal result. According to the literature review [17; 18; 19], classical machine learning models such as Random Forest (RF) and XGBoost, along with the deep learning model LSTM, were selected and compared with the innovative LSTM-RF-XGBoost combination model proposed in this study. The LSTM-RF-XGBoost combination model aims to integrate both time series analysis elements and the fitting accuracy of deep learning neural networks. This approach better addresses the high demand and cyclicality issues of home appliances while improving prediction accuracy. In order to assess the accuracy of the model predictions, we compared the evaluation results of the models, as shown in Table 9. \begin{table}
\begin{tabular}{c c c} \hline
**Parameter** & **Meaning** & **Parameterization** \\ \hline objective & Loss function type & reg:squarederror \\ booster & Base learner type & gbtree \\ eval\_metric & Evaluation indicator & MSE \\ \hline \end{tabular}
\end{table} 
Table 6: Fixed parameters of XGBoost model. \begin{table}
\begin{tabular}{c c} \hline
**Parameter** & **Initialization Parameter** & **Optimized Parameter** \\ \hline learning\_rate & 0.3 & 0.12 \\ gamma & 0 & 0.7 \\ max\_depth & 6 & 5 \\ min\_child\_weight & 1 & 4 \\ subsample & 0.7 & 0.6 \\ colsample\_bytes & 1 & 0.58 \\ lambda & 1 & 2 \\ alpha & 0 & 5 \\ scale\_pos\_weight & 1 & 0.7 \\ \hline \end{tabular}
\end{table} 
Table 7: Bayesian parameter optimization diagram of XGBoost. \begin{table}
\begin{tabular}{c c} \hline
**Model** & **Parameter** \\ \hline SVR & Kernel is rbf, C is 0.5, epsilon is 0.05, gamma is auto \\ K-NN & Weight is distance, n\_neighbors is 3, algorithm is ball\_tree, leaf\_size is 20, p is 1 \\ \hline \end{tabular}
\end{table} 
Table 8: SVR and K-NN parameters. As shown in Table 9, the parameter values of LSTM, RF, and XGBoost were significantly better than those of SVR and K-NN, indicating that these three methods are more suitable for predicting the demand for air conditioning products. The results in the table show that the three single models exhibited relatively high \(MAE\) and \(RMSE\) metrics on the test set. This indicates that there was a certain problem of a poor generalization ability, which requires further adjustment of the model parameters. However, the models exhibited a high \(R^{2}\) on both the training and test sets, indicating that they fit the data better. In contrast, the blending model performed the best in prediction, and the evaluation indexes were better than the single-model evaluation indexes. The model not only performed optimally in terms of goodness of fit, with a result of 0.8682, but also had an MAE value of 0.0658, while the RMSE value was 0.1407, both of which are the lowest among the all models. This shows that the blending model had an advantage in terms of both model accuracy and stability. The combination of LSTM, RF, and XGBoost approaches leveraged the strengths of each individual algorithm. With its recurrent neural network architecture, LSTM effectively captures the long-term dependencies and temporal dynamics inherent in demand data.