# A Comprehensive Hybrid Deep Learning Approach for Accurate Status Predicting of Hydropower Units
## 3 Methodology
### Dual-Path Memory Integration for Extracting Adjacent Temporal Features  
The Dual-Path Memory Integration (DPMI) module, a key component of our model, combines the strengths of REST-LSTM and GRU [41]. The REST-LSTM modifies the standard LSTM [42] by introducing a dense layer that transforms the input to match the LSTM's output dimensions. This transformed input is then added directly to the LSTM's hidden state, creating a residual connection that enhances the model's ability to learn and retain features effectively. Figure 2: The TCN layer in the hybrid deep learning model. The employed GRU architecture, illustrated in Figure 3, utilizes the reset gate \(g_{r}\) to control the amount of information from the previous time step \(h_{t-1}\) that is carried into the current GRU cycle. The calculation for \(g_{r}\) is shown in Equation (7):  
\[g_{r}=\sigma(W_{r}\cdot[h_{3b,t-1},h_{2,t}]+b_{r}) \tag{7}\]  
where \(\sigma\) represents the sigmoid activation function; \(W_{r}\) denotes the trainable weights; \(h_{3b,t-1},h_{2,t}\) denote the previous hidden state and current input, respectively; and \(b_{r}\) denotes the bias. In Figure 3, \(h_{3b,t-1},h_{2,t}\) are specifically referred to as \(H_{3b,t=1}\) and \(H_{2,t=2}\), respectively. The candidate hidden state \(\widehat{h_{t}}\) update is governed by Equation (8):  
\[\widehat{h_{t}}=tanh(W_{h}\cdot[g_{r}\circ h_{3b,t-1},h_{2,t}]+b_{h}) \tag{8}\]  
where \(tanh\) is the activation function, \(W_{h}\) is the learnable weight, and \(b_{h}\) is the bias. The update gate \(g_{z}\), which dictates the extent to which the hidden state should be updated with the new candidate state, are calculated as shown in Equation (9):  
\[g_{z}=\sigma(W_{z}\cdot[h_{3b,t-1},h_{2,t}]+b_{z}) \tag{9}\]  
where \(W_{z}\) is the learnable weight, and \(b_{z}\) is the bias. The final updated hidden state \(h_{t}\) is a blend of the old state and the candidate state, as determined by the update gate \(g_{z}\) (Equation (10)):  
\[h_{t}=(1-g_{z})\circ h_{3b,t-1}+g_{z}\circ\widehat{h_{t}} \tag{10}\]  
In traditional LSTM architectures, the hidden state is updated at each time step, effectively capturing and propagating information within short-term sequences. However, when dealing with rapid or frequent changes in the input data, the standard LSTM may struggle to retain and emphasize the most recent and relevant information, which can lead to a degradation in the processing of critical short-term dependencies. The REST-LSTM addresses this limitation by introducing a residual connection specifically designed to enhance short-term memory processing. This residual connection involves adding the transformed input directly to the hidden state, achieved through a dense layer that ensures dimensional compatibility. By reinforcing the influence of recent inputs on the current state, the model becomes better equipped to retain and utilize short-term features, particularly in scenarios where the input data exhibits rapid fluctuations. Additionally, this modification stabilizes the hidden state by providing a direct pathway for the most recent information, allowing the model to maintain a consistent flow of critical short-term data. This approach reduces the reliance on traditional gating mechanisms, enabling more robust and effective learning in tasks that require precise short-term temporal analysis. Figure 3: The GRU. The equations governing the computation of \(H_{l,t}\) and \(C_{l,t}\) are as follows (Equations (11) and (12)):  
\[H_{l,t}=ReLU[(W_{dense}\cdot X_{l,t}+d_{dense})+O_{l,t}\circ\tanh\text{h}(C_{l,t})] \tag{11}\]  
\[C_{l,t}=F_{l,t}\circ C_{l,t-1}+I_{l,t}\circ\tanh\text{h}(W_{c}\cdot[H_{l,t-1}, \ X_{l,t}]+b_{c}) \tag{12}\]  
where \(ReLU\) is the activation function, \(W_{dense}\) and \(d_{dense}\) represent the weights and biases of the fully connected layer, \(X_{l,t}\) is the input at the current time step, and \(O_{l,t}\) is the output of the output gate.