# Forecasting of Power Demands Using Deep Learning
## 2 Materials and Methods
### The Proposed Models  
We designed different deep learning models, namely convolution neural network (CNN) [29], recurrent neural network (RNN) [30], and a hybrid model that combines CNN with RNN. We aimed to find the best performing model according to the past and future data of our dataset. CNN model is a type of deep neural network that is utilized in various domains. It is also considered as a shift-invariant model as they have translation invariance characteristics and shared-weights architecture. CNN models were widely and successfully used in different areas such as image and \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Feature** & **Count** & **Mean** & **STD** & **Minimum Value (MW)** & **Maximum Value (MW)** \\ \hline Facility Capacity (MW) & 6352.0 & 83,447.03 & 20,918.82 & 53,800.0 & 126,887.0 \\ \hline Supply Capacity (MW) & 6352.0 & 72,243.58 & 13,245.45 & 44,937.0 & 101,923.0 \\ \hline Power Consumption (MW) & 6352.0 & 58,550.06 & 11,468.32 & 26,916.0 & 92,478.0 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The statistical overview of facility capacity, supply capacity, and power consumption. Figure 1: The characteristics of the features in the dataset within the period of the study. Figure 2: Example of data preparation. video classification, medical image analysis, natural language processing, bioinformatics, and time-series. In this work, we designed a simple two-layer CNN model as shown in Figure 3. Each layer consists of a 1-dimensional convolution layer with 32 filters and a filter size of 3, a non-linear activation function which is the rectified linear unit (ReLU), and a max-pooling layer with window size and stride of 2. The learned features from these two layers are then fed into a fully connected layer with a one-node for prediction. The convolution layer is a 1-d convolution expressed in Equation (1) where \(I\) is the input, \(k\), and \(o\) are the indices of kernels and the output position, respectively, \(W^{f}\) is the weight matrix of \(S\times N\) shape with \(S\) filters and \(N\) channels. \[\text{Conv}(I)_{ok}=\text{ReLU}\left(\sum_{s=0}^{S-1}\sum_{n=0}^{N-1}W^{f}_{sn }I_{o+s,n}\right) \tag{1}\]  
RNN is another typical deep learning model that is mainly used in natural language processing and speech recognition [31; 32]. It is used to understand the data's sequential behavior and predict the next likely outcomes [33]. In this paper, we have used a one bidirectional long short-term memory (LSTM) layer with 16 nodes followed by a dropout layer [34] with a dropout probability of 0.2, and a fully-connected layer with one node for prediction. Figure 4 shows the architecture of the RNN model. Bi-LSTM has been used in different areas such as phoneme classification [35], speech recognition [36], human action recognition [37], and machine translation [38]. Different gates are available in the LSTM cell. The input gate is used to decide which information should be stored for the next layer and update the current state. The forget gate is used to decide the information that should be removed according to the previous inputs. The output gate decides which part of the state value should be output. Thus, considering an input sequence \(\{\mathbf{x}\}_{t=1}^{T}\), the LSTM has cell states \(\{\mathbf{C}\}_{t=1}^{T}\), hidden states \(\{\mathbf{h}\}_{t=1}^{T}\) and outputs a sequence \(\{\mathbf{o}\}_{t=1}^{T}\). This can be expressed mathematically by Equation (2) where \(\mathbf{W}_{i}\), \(\mathbf{W}_{o}\), \(\mathbf{W}_{f}\), \(\mathbf{U}_{i}\), \(\mathbf{U}_{o}\), \(\mathbf{U}_{f}\) are the weight matrices and \(\mathbf{b}_{o}\), \(\mathbf{b}_{c}\), \(\mathbf{b}_{i}\), \(\mathbf{b}_{f}\) are the biases. Sigmoid and Tanh are the activation functions. The \(\odot\) is the element-wise multiplication. Figure 3: The architecture of the convultional neural network (CNN) model. \[