# Construction of Ensemble Learning Model for Home Appliance Demand Forecasting
## 2 The Proposed Method
### Blending Ensemble Models  
For complex demand change scenarios, a single demand forecasting methodology model suffers from the problem of limited expressiveness in multiple dimensions. For this reason, this study proposes the multi-model fusion approach known as blending for optimization, which uses multiple sub-models to model, predict, and fuse data. Multi-model fusion can better cope with complex demand changes and improve accuracy in multiple dimensions compared to traditional single models or traditional demand forecasting methods [32]. Figure 1: LSTM network architecture. Figure 2: RF algorithm model architecture. In the previous three subsections, the LSTM, RF, and XGBoost models, respectively, were established. These three single prediction models produce good results for the demand forecasting problem and have differentiation, which is in line with the requirements of the first-layer diversity-based model. Given that home appliances have a large number of features and quantities, and feature interactions can reveal more complex relationships between variables, we add feature interactions to the first-layer model results. To improve the prediction accuracy, we use the mutual information method to identify the interacting features that contribute the most. For the selection of the second layer of the meta-learner, a multiple linear regression (MLR) algorithm is used, while regularization is introduced to avoid overfitting. The LSTM-RF-XGBoost algorithmic model architecture is illustrated in Figure 4.  
This blending framework consists of two layers of models. The first layer of the base learner includes three models consisting of LSTM, RF, and XGBoost, which are trained on the original training set. The base learners have different learning and prediction capabilities, and the outputs can be considered to be learning results under different feature dimensions. In the second layer, multivariate linear regression is used as a meta-learner, and the results of the first layer of base learners are used as inputs to output the final predicted values. The steps of the Blending Demand Prediction Fusion Model are as follows:  
Layer 1: In this study, the dataset is divided into two parts, a training set and a test set. The training set is further split into a primary training set and a primary validation set. The primary training set is used to train the LSTM, RF, and XGBoost models, and the resulting base models are then used to make predictions on the primary validation set and primary test set, respectively. Next, we apply feature interaction and filtering to the results from both the primary validation and test sets. The filtered interaction features from the primary validation set, along with its results, are combined to form the secondary training set, which is used to train the metamodel in the second layer. Finally, the results and filtered interaction features from the primary test set are fed into the trained metamodel for testing. The training set lasts 48 months, from January 2018 to December 2021. The primary training set lasts from January 2018 to December 2020. The primary validation set lasts from January 2021 to December 2021. The test set lasts from January 2022 to December 2022. Figure 3: XGBoost algorithm model architecture. We create the feature interactions using PolynomialFeatures with degree = 2, interaction_only = True, and include_bias = False. We choose 0.03 as the screening threshold for mutual information. Mutual information is exploited to quantify the information shared by two variables, and mutual information \(I(X;Y)\) between two variables \(X\) and \(Y\) can be defined as:  
\[I(X;Y)=\sum_{x\in X}\sum_{y\in Y}p(x,y)\log\frac{p(x,y)}{p(x)p(y)} \tag{1}\]  
Layer 2: The MLR is trained using the secondary training set. The process expression for MLR is shown in Equation (2).