# Electric Load Forecasting Based on Deep Ensemble Learning
## 2 Materials and Methods
### CNN-BiLSTM-RF Anomaly Detection Model
#### 2.2.3 The Random Forest Layer  
The random forest layer constitutes the final stage of the CNN-BiLSTM-RF model. Once the input sequence has traversed through the convolutional neural networks layer and the bidirectional long short-term memory network layer, it produces an output that models and extracts features from the series. Each output is treated as a sample, and each time step is considered a feature. These outputs then serve as inputs to the random forest layer. The random forest layer comprises multiple decision trees, each constructed by randomly selecting subsets of features and data [25; 26]. Each decision tree categorizes the input into distinct classes or labels. When making predictions, every decision tree within the random forest layer assesses the information and provides a forecast. This paper utilizes an enhanced version of random forest, with the forest's construction detailed in references [27; 28]:  
First, define a forest \(\theta_{0}\) that consists of \(B_{0}\) trees, define \(i\) as a node in the tree \(T\), and define \(F_{0}\) as the initial feature set. The label category of this node is defined as \(p(c)\), and the node is split using an entropy-based method; then, the entropy of the node is:  
\[E=\sum_{\forall c}p(c)\ln\frac{1}{p(c)} \tag{6}\]  
Randomly select features from the feature set \(F_{0}\), denoted as \(A\). If there is feature \(j\) in \(A\) and node \(i\) is split using feature \(j\), assign a local weight to feature \(j\):  
\[w^{T}(j)=\frac{\sum\limits_{i=1}^{N}Q(i,j)}{N} \tag{7}\]  
where \(N\) is the sum of all nodes in tree \(T\). The higher the value of \(w^{T}(j)\), the higher the quality of feature splitting the node. Next, calculate the weight of the tree based on the out-of-bag error (OOB error):  
\[\gamma^{T}=\frac{1/\delta^{T}}{\max(1/\delta^{T})} \tag{8}\]Here, \(\delta^{T}\) represents the out-of-bag error in tree \(T\). A high value of \(\gamma^{T}\) indicates that the classification error of tree \(T\) is smaller. For feature \(j\), its global weight is:  
\[w(j)=\frac{\sum\limits_{\forall T}w^{T}(j)\gamma^{T}}{\underset{j}{\max}\sum \limits_{\forall T}w^{T}(j)\gamma^{T}} \tag{9}\]  
Based on the size of the global weights, the features are reordered, with the top \(c_{n}\) number of features selected as important features and placed in the set \(\Gamma_{n}\). The remaining features are considered as unimportant features and put in \(\Gamma_{n}\)'. Using \(a_{n}\) and \(s_{n}\) to represent the mean and standard deviation of the global weights of the unimportant features in \(\Gamma_{n}\)', the features with weights less than \((a_{n}-2s_{n})\) are stored in a new set \(R_{n}\). The features that exist in \(R_{n}\) are directly discarded from the set \(\Gamma_{n}\)'. Let \(q\) be the minimum probability of a good split node or the probability of finding at least one important feature among all possible features. Using \(u\) and \(v\) to represent the number of important and unimportant features, respectively, the probability of finding an unimportant feature is:  
\[r=1-q=\frac{C(v,f)}{C(u+v,f)} \tag{10}\]  
where \(f\) is the number of features selected each time. If \(v<\)\(f\), then there exists \(q=1\), \(r=1-q=0\). Suppose it is defined as:  
\[q_{u}=\frac{\partial q}{\partial u},\ q_{v}=\frac{\partial q}{\partial v} \tag{11}\]  
Then, there is:  
\[q_{u}=-\frac{\partial r}{\partial u},\ q_{v}=-\frac{\partial r}{\partial v} \tag{12}\]  
\[\frac{\partial r}{\partial u}=\frac{\partial\frac{C(v,f)}{C(u+v,f)}}{\partial u }=\frac{\frac{\partial C(v,f)}{\partial u}\cdot C(u+v,f)-C(v,f)\cdot\frac{ \partial C(u+v,f)}{\partial u}}{\left(C(u+v,f)\right)^{2}} \tag{13}\]  
\[\frac{\partial r}{\partial v}=\frac{\partial\frac{C(v,f)}{C(u+v,f)}}{\partial v }=\frac{\frac{\partial C(v,f)}{\partial v}\cdot C(u+v,f)-C(v,f)\cdot\frac{ \partial C(u+v,f)}{\partial v}}{\left(C(u+v,f)\right)^{2}} \tag{14}\]  
In the differentiation process, the binomial coefficient's derivative is undefined at integer points due to its discrete nature.