## 4 Methodology
### Forecasting Model--Multivariate Multilayered LSTM  
M-LSTM is an extension of the LSTM model, which is a type of recurrent neural network (RNN) architecture used for sequential data processing tasks such as TS forecasting. There are \(h\) sub-layers In the hidden layer. The M-LSTM model consists of multiple LSTM layers, with each layer having its own set of neurons that process the input variables independently. The output of each layer is then fed into the next layer, allowing the model to capture more complex and abstract relationships between the input variables. There are some additional layers, such as the dropout layer, normalization layer, etc., in the hidden part for training the model efficiently.  
As shown in Figure 6, the memory cell is responsible for storing information about the long-term dependencies and patterns in the input sequence, while the gates control the flow of information into and out of the cell. The gates are composed of sigmoid neural network layers and a point-wise multiplication operation, that allows the network to learn which information to keep or discard. In particular, each \(i\)th cell at the layer \(m\), denoted as \(C_{i}^{m}\), has three inputs, i.e., the hidden state \(h_{i-1}^{m}\), cell state \(c_{i-1}^{m}\) of the previous cell, and the hidden state \(h_{i}^{m-1}\) of the cell \(i\) at the previous layer \(m-1\). The cell \(C_{i}^{m}\) has two recurrent features, i.e., hidden state \(h_{i}^{m}\), and cell state \(c_{i}^{m}\). \(C_{i}^{m}\) is a mathematical function as Equation (1), that takes three inputs and returns two outputs. Both outputs leave the cell at time \(i\) and are fed into that same cell at time \(i+1\), and the input sequence \(\mathbf{x}_{i+1}\) is also fed into the cell. In the first layer (\(m=1\)), the hidden state \(h_{i}^{m-1}\) is the input \(\mathbf{x}_{i}\).  
\[(h_{i}^{m},c_{i}^{m})=C(h_{i-1}^{m},c_{i-1}^{m},h_{i}^{m-1}) \tag{1}\]  
Inside the \(i\)th cell, the previous hidden state \(h_{i-1}^{m}\) and input vector \(\mathbf{x}_{i}\) are fed into three gates. i.e., input gate (\(ig_{i}^{m}\)), forget gate (\(f_{S_{i}}^{m}\)), output gate (\(og_{i}^{m}\)). They are _sigmoid_ functions (\(\sigma\)), each of which produces a scalar value as described in Equations (2)-(4) respectively.  
\[ig_{i}^{m}(h_{i-1}^{m},h_{i}^{m-1})=\sigma(w_{ig,m-1}h_{i}^{m-1}+w_{ig,m}h_{i- 1}^{m}+b_{ig}) \tag{2}\]  
\[f_{Si}^{m}(h_{i-1}^{m},h_{i}^{m-1})=\sigma(w_{fg,m-1}h_{i}^{m-1}+w_{fg,  
scalar function, called the update function (denoted as \(ug\)) has a \(tanh\) activation function as described in Equation (5).  
\[u_{g_{i}}(h_{i-1}^{m},h_{i}^{m-1})=tanh(w_{ug,x}h_{i}^{m-1}+w_{ug,h}h_{i-1}^{m}+b_{ ug}) \tag{5}\]  
where \(w_{ig,x}\in\mathbb{R}\) and \(w_{ug,h}\in\mathbb{R}\) are further weighted to be learned. The returned cell state (\(c_{i}^{m}\)) and hidden state (\(h_{i}\)) are formulated in Equation (6), (7) respectively.  
\[c_{i}^{m}=fg_{i}^{m}\cdot c_{i-1}^{m}+ig_{i}^{m}\cdot ug_{i} \tag{6}\]  
\[h_{i}^{m}=og_{i}^{m}\cdot tanh(c_{i}^{m}) \tag{7}\]  
In energy forecasting, loss optimization is one of the important steps to improve the accuracy of the model [39]. One common technique for loss optimization is using the Adam optimizer, which is a stochastic gradient descent optimizer that uses moving averages of the parameters to adapt the learning rate. The Adam optimizer computes individual adaptive learning rates for different parameters from estimates of the first and second moments of the gradients. This makes it suitable for optimizing the loss function in models where there are a large number of parameters. By using the Adam optimizer, the model can efficiently learn and update the weights of the neurons in each time step, resulting in better prediction accuracy. In addition, the loss function formulation of the Adam optimizer used in the proposed model aims to strike a balance between accuracy and robustness, taking into account the unique characteristics of the data and the specific requirements of the forecasting task.