# Improving Electric Energy Consumption Prediction Using CNN and Bi-LSTM
## 3 Materials and Methods
### The EECP-CBL Model  
Figure 2 introduces the overall architecture of the proposed model namely EECP-CBL for predicting electric energy consumption utilizing the combination of CNN and Bi-LSTM. The input variables from IHEPC dataset are extracted by two CNN layers in the first module and passed to the two Bi-LSTM layers in the second module. The Bi-LSTM layers are used for information analysis and time series prediction. Finally, the proposed method can generate predicted electrical energy consumption by the last module consisting of two fully connected layers. The results values predicted by the proposed model are evaluated by several performance metrics for time series forecasting such as MSE, RMSE, MAE and MAPE. In the first module, two one-dimensional CNN layers followed by two max pooling layers to reduce the computational complexity were used to analysis the input variables for feature extraction. At the early stage, MLP was often preferred as feature extractor where it is considered as a feedforward artificial neural network However, the model becomes inefficient and redundancy due to the face that  
Figure 1: Samples of four electrical energy consumption datasets. Figure 2: The overview architecture of EECP-CBL model. each perceptron is connected with every other perceptron. CNN, in the other hand, is a special type of MLP where every neuron does not need connected to other neurons. Those neurons only connected to a region of data (matrices or vector inputs) and are panned around entire data according to certain size and stride. The panning of filters in CNN essentially allows parameter sharing, weight sharing so that the filter looks for a specific pattern and is location invariant. Specifically, a number of neurons in a CNN layer is combined by weights and biases. These values have to be learned by the training process. In these models, several input variables are provided to each neuron. Next, the dot product operator is executed and followed by an optional nonlinearity function. In our application, convolution 1D layers, pooling 1D layers and fully connected layer are used. In details, CNN takes the time series data in one dimensional form wherein the data are arranged in the order of sequential time instants. Let the one-dimensional input vector be \(\mathbf{x}=\{x_{1},\,x_{2},\,\ldots,\,x_{n},\,ec\}\) where \(x_{n}\in R^{d}\) are the variables in the dataset and \(ec\in R\) denotes the energy consumption value. The convolution 1D constructs a feature map \(fm\) by applying the convolution operator on the input data with a filter \(w\in R^{\mathit{fl}}\) where \(f\) denotes the features inherent in the input data producing at its output, new set of features which is fed to input of the next block in line. A new feature map \(fm\) is obtained from a set of features \(f\) as the following equation. \[hh_{i}^{f/m}=tanh(w^{fm}x_{i+f-1}+b) \tag{1}\]  
The filter \(h\) in Equation (1) is utilized to each set of features \(f\) in the input data defined by \(\{x_{1:f},\,x_{2:f+1},\,\ldots,\,x_{n-f+1}\}\) and generate a feature map denoted by \(h\) = [\(h\)1, \(h\)2,..., \(h\)1\({}_{n-f+1}\)]. In addition, \(b\in R\) is a bias term and \(h\)\(\in R_{n-f+1}\). The output of convolutional layers is summed of weighted inputs which are comprised from multi linear transformations. Normally the linear transformation cannot capture the complex structures in the data, so that a non-linear activation layer is preferred to apply after convolutional layers to learning the data better in training step. In this study, we choose ReLU activation function that apply \(max(0,x)\) to each of the inputs. Then, the output of the convolutional layer is passed to the pooling layer which performs a down-sampling operator. In our model, the max-pooling layer is applied on each feature map \(\overset{\rightarrow}{h}\)\(=max\{h\}\). This process selects the most significant features with highest values. The output of the max-pooling layer is denoted as follows.