# A Two-Stage Household Electricity Demand Estimation Approach Based on Edge Deep Sparse Coding
## 3 Proposed Methodology
### Stage 2: DUBPs-Based Electricity Demand Estimation Using LSTM  
This stage of the proposed method consisted of training and testing DUBP-based load forecasting. In the training part, the reconstructed household data for the past 30 min and two DUBPs extracted from the load profile using the edge deep sparse coding algorithm were used to train the LSTM network. In the testing part, the well-trained LSTM network was implemented to forecast household electricity demand in the subsequent 30 min. Household-level load demand estimation is critical for precise DR. However, accurate forecasting is difficult due to the variability and randomness of resident behavior. LSTM was selected because it can be flexible for the strong temporal correlation of household-level load data.  
An LSTM network refers to a specific architecture of recurrent neural network (RNN). It not only exhibits the recursive attribute of RNN, but also includes a unique memory cell with an additional forget gate, which solves the gradient vanishing or exploding problem in traditional RNN. LSTM networks have achieved considerable success and are currently popular in many sequential forecasting tasks.  
Both LSTM network and traditional RNN are composed of an input layer, an output layer, and a hidden layer. However, the hidden layer of an LSTM network is no longer a common neural unit, but a recurrent memory unit. Figure 5 presents a typical LSTM cell.  
Figure 4: The edge sparse coding architecture for load data.  
Let \(x=\{x_{1},x_{2},\cdots x_{T}\ \}\) denote a typical input sequence for an LSTM network, where \(x_{t}\in\mathbb{R}^{k}\) represents a \(k\)-dimensional vector for actual values at each time step \(t\). To establish temporal connections, LSTM defines and maintains an internal memory cell unit at each time step t as a collection of vectors in \(\mathbb{R}^{k}\) : an input gate \(i_{t}\), a forget gate \(f_{t}\), an output gate \(o_{t}\), a memory cell \(c_{t}\), and a hidden state \(h_{t}\).  
The memory cell state \(c_{int}\) interacts with the intermediate output \(h_{t-1}\), and the subsequent input \(x_{t}\) in the input gate, forget gate, and output gate is divided to determine which elements of the internal state vector should be updated, maintained, or erased. The signal of the input gate controls which elements to preserve in the internal state, whereas the forget gate controls which elements to forget from the previous state \(c_{t-1}\). The output gate determines the LSTM output \(h_{t}\) with the updated internal state.  
The LSTM transition equations are given as:  
\[i_{t}=\sigma(W_{xi}x_{t}+W_{hi}h_{t-1}+W_{ci}c_{t-1}+b_{i}) \tag{6}\]  
\[f_{t}=\sigma\Big{(}W_{xf}x_{t}+W_{hf}h_{t-1}+W_{cf}c_{t-1}+b_{f}\Big{)} \tag{7}\]  
\[c_{t}=f_{t}c_{t-1}+l_{t}tanh(W_{xc}x_{t}+W_{hc}h_{t-1}+b_{c}) \tag{8}\]  
\[o_{t}=\sigma(W_{x0}x_{t}+W_{ho}h_{t-1}+W_{co}c_{t}+b_{o}) \tag{9}\]  
\[h_{t}=o_{t}tanh(c_{t}) \tag{10}\]  
where \(W_{xi},\ W_{xf},\ W_{xc},\ W_{x0}\) are weight matrices for the subsequent input \(x_{t}\); \(W_{hi},\ W_{hf},\ W_{hc},\ W_{ho}\) are weight matrices for the hidden state \(h_{t}\); \(W_{ci},\ W_{cf},\ W_{co}\) are weight diagonal matrices from the input gate, forget gate and output gate to the previous state memory cell \(c_{t-1}\)[32]. The \(b\) terms denote bias vectors and \(\sigma\) is the activation function which will always be the sigmoid or tanh function.  
The proposed two-stage household electricity demand estimation approach based on edge deep sparse coding with appliance data is illustrated in Figure 6.  
Figure 5: The structure of memory cell in LSTM.