# An Interpretable Deep Learning Approach for Detecting Marine Heatwaves Patterns
## 2 Methods
### Additive Decomposition to Assess LSTM Decisions  
Decomposition methods can peek into the \"black box\" structure of the LSTM model to ascertain how hidden information is processed, yet they do not directly attribute the final output to the input features at each timestep nor provide insights into the internal signals. Given that the model output \(y\) in this study is derived from the hidden state of LSTM at the last time step (i.e., \(y=W_{d}h_{T}\)), we focus our attention on analyzing the signal origins of \(h_{T}\). Considering the update rule of the cell state (i.e., \(c_{t}=f_{t}\odot c_{t-1}+i_{t}\odot\bar{c}_{t}\)) and the transformation rule of the hidden state (i.e., \(h_{t}=o_{t}\odot tanh(c_{t})\)), the hidden state \(h_{t}\) can be approximated as the sum of the information obtained from the previous hidden state \(h_{t-1}\) and the new information obtained at the current time step [27], represented as:  
\[h_{t}=\frac{f_{t}\odot o_{t}}{o_{t-1}}\odot h_{t-1}+\bar{h}_{t} \tag{7}\]  
where, \(\frac{f_{t}\odot o_{t}}{o_{t-1}}\) represents the proportion of the information to be retained, which depends on the forget gate and the output gate. \(\bar{h}_{t}\) represents information acquired at timestep \(t\), but we do not need to know its exact form. Therefore, the hidden state at the last time step \(T\) can be iteratively traced back and decomposed into:  
\[h_{T}=\frac{f_{T}\odot o_{T}}{o_{T-1}}\odot h_{T-1}+\bar{h}_{T}=\frac{f_{T} \odot o_{T}}{o_{T-1}}\odot\left(\frac{f_{T-1}\odot o_{T-1}}{o_{T-2}}\odot h_{T -2}+\bar{h}_{T-1}\right)+\bar{h}_{T}=\cdots=\sum_{t=1}^{T}\left[\left(\prod_{k =t+1}^{T}\frac{f_{k}\odot o_{k}}{o_{k-1}}\right)\odot\bar{h}_{t}\right] \tag{8}\]  
In this way, \(h_{T}\) can be decomposed into the sum of the contributions from each timestep from \(1\) to \(T\). The contributed information at timestep \(t\) can be considered as the product of the initially obtained information from \(t-1\) to \(t\) (i.e., \(\bar{h}_{t}\)) and the retained proportion by the forget gates in subsequent cells from \(t\) to \(T\) (i.e., \(\prod_{k=t+1}^{T}\frac{f_{k}\odot o_{k}}{o_{k-1}}\)). Based on Formula (8), the decomposed form of \(h_{T}\) can further be reconstructed as:  
\[h_{T}=\sum_{t=1}^{T}\left[\left(\prod_{k=t+1}^{T}\frac{f_{k}\odot o_{k}}{o_{k- 1}}\right)\odot\left(h_{t}-\frac{f_{t}\odot o_{t}}{o_{t-1}}\odot h_{t-1}\right)\right] \tag{9}\]The main advantage of using Formula (9) over Formula (8) is that it is sufficient for analysis only knowing hidden state vectors \([h_{t}]\), forget gate vectors \([f_{t}]\), and output gate vectors \([o_{t}]\). Finally, the output of the model \(y\) can be decomposed as:  
\[y=W_{d}h_{T}=\sum_{t=1}^{T}[W\Big{(}h_{t}-\frac{f_{t}\odot o_{t}}{o_{t-1}}\odot h _{t-1}\Big{)}\odot\Big{(}\prod_{k=t+1}^{T}\frac{f_{t}\odot o_{k}}{o_{k-1}}\Big{)}] \tag{10}\]  
These formulas indicate that the final output of the model consists of information accumulated over \(T\) timesteps. The contribution at each timestep is part of the information obtained at that timestep, which is kept for the final timestep after \"forgetting\" by successive cells. The decomposition algorithm is efficient, requiring only an extraction of related vectors from the trained LSTM network and forward propagation operation.