# A Hybrid DNN Multilayered LSTM Model for Energy Consumption Prediction
## 3 Proposed Model
### The DNN Multilayered LSTM Model
#### 3.2.2 Multilayered LSTM Model  
LSTM was specifically designed for modeling temporal sequences. LSTM is a modified RNN, but it stands out from other RNNs because of its long-range dependencies, which increase its accuracy. Using typical RNN algorithms eliminates the main problem of being unable to determine the significant connection between steps in data over 10. LSTM has special units, which are referred to as memory blocks, found in the recurrent hidden layer. The memory blocks have memory cells that store the temporal state of the network. These memory cells allow LSTM to have long-term memory capabilities that enable the updating of the previous hidden state. Memory cells provide feedback in every neuron. The input data to a memory cell usually contain the cell state, the previous step output, and the input data of the current step. This makes the output of the current cell dependent on the inputs and weights of the current step as well as the inputs of the previous step. This functionality is responsible for the LSTM's ability to understand temporal relationships with long-term sequences. It can create relationships even when the data contain 1000 steps. Each memory block has three gates that manage the cell state information for the LSTM. The forget gate is the first gate, and it is responsible for selecting and removing a part of the cell's memory. The input gate is the second gate, and it is responsible for controlling the input data flow into the cell's memory. The output gate is the last gate, and it is responsible for the output data flow from the cell's memory. The LSTM process is computed through a sequence of Equations (1)-(6) [38]. \[f_{t}=\sigma\Big{(}x_{t}W_{xf}+h_{(t-1)}W_{hf}+b_{f}\Big{)} \tag{1}\]  
\[i_{t}=\sigma\Big{(}x_{t}W_{xi}+h_{(t-1)}W_{hi}+b_{i}\Big{)} \tag{2}\]  
\[o_{t}=\sigma\Big{(}x_{t}W_{xo}+h_{(t-1)}W_{ho}+b_{o}\Big{)} \tag{3}\]  
\[\widetilde{c}_{t}=tanh\Big{(}x_{t}W_{xc}+h_{(t-1)}W_{hc}+b_{c}\Big{)} \tag{4}\]  
\[c_{t}=f_{t}\odot C_{t-1}+i_{t}\odot\widetilde{c}_{t} \tag{5}\]  
\[h_{t}=o_{t}\odot tanh(c_{t}) \tag{6}\]  
The current LSTM cell is fed by the output of the previous nodes (\(h_{t-1}\)), input \(x_{t}\), and the previous node's cell state \(c_{t-1}\). The cell states contain hidden long-term data. Information is passed through three different LSTM gates: input (\(i_{t}\)), forget (\(f_{t}\)), and output (\(o_{t}\)). Using the gates, the LSTM node can forget or keep cell states and compute the next outputs. In the mathematical equations, \(\sigma\) is the sigmoid function and \(\odot\) is element-wise multiplication. The bias and weight values are represented by \(W\) and \(b\), respectively. The values for the LSTM cells are shown in Figure 2, which has three cells. The middle cell is the current cell. * Equation (1), \(f_{t}=\sigma\Big{(}x_{t}W_{xf}+h_{(t-1)}W_{hf}+b_{f}\Big{)}\), determines the cell's information state. The forget gate is responsible for this action. It observes the previous nodes (\(h_{t-1}\)), input \(x_{t}\), and the previous nodes' cell state \(c_{t-1}\). It also produces binary outputs 1 and 0, with 1 directing the complete retention of information and 0 directing the information to be discarded. * Equation (2), \(i_{t}=\sigma\Big{(}x_{t}W_{xi}+h_{(t-1)}W_{hi}+b_{i}\Big{)}\), determines which information needs updating. The input gate is responsible for this action. * Equation (3), \(o_{t}=\sigma\Big{(}x_{t}W_{xo}+h_{(t-1)}W_{ho}+b_{o}\Big{)}\), is responsible for the output of the current node. The output gate is responsible for this action. The state of the outputgate is expressed by the values 0 and 1, with 1 indicating that the gate is open and 0 indicating that the gate is closed. * Equation (4), \(\widetilde{c_{t}}=tanh\Big{(}x_{t}W_{xc}+h_{(t-1)}W_{hc}+b_{c}\Big{)}\), calculates the memory gate, which is responsible for selecting the information that needs to be memorized from the temporary cell state. * Equation (5), \(c_{t}=f_{t}\odot C_{t-1}+i_{t}\odot\widetilde{c_{t}}\), determines the current memory cell's value.