## 4 Materials and Methods
### Introduction to the LSTM Algorithm
\begin{bmatrix}w_{hf,1,1}&w_{hf,1,2}&\ldots&w_{hf,1,64}\\ w_{hf,2,1}&w_{hf,2,2}&\ldots&w_{hf,2,64}\\ \vdots&\vdots&\ddots&\vdots\\ w_{hf,64,1}&w_{hf,64,2}&\ldots&w_{hf,64,64}\end{bmatrix} \cdot \begin{bmatrix}h_ {t-1,1}\\ h_{t-1,2}\\ \vdots\\ h_{t-1,64}\end{bmatrix} \in\mathrm{R}^{64} \tag{4}\]  
\[f_{t}[i]=\frac{1}{1+e^{-(w_{ij}z_{i}+w_{hf,j}y_{h-1}+b_{hf}[i])}}\text{for} \;i=1,2,\ldots,64 \tag{5}\]  
To enhance the robustness of the code, this paper selects the Huber Loss as the loss function. The Huber Loss is a loss function that is insensitive to outliers, combining the advantages of MSE (Mean Squared Error) and MAE (Mean Absolute Error). In regression problems, it not only maintains the smoothness of MSE but also reduces the influence of outliers. For the predicted value \(\hat{y}\) and the true value \(y\), the Huber Loss is defined as  
\[L_{\delta}\bigg{(}\hat{y},\hat{y}\bigg{)}= \begin{cases}\frac{1}{2}(\hat{y}- \hat{y})^{2},&|y-\hat{y}|\leq\delta\\ \delta(|y-\hat{y}|-\frac{1}{2}\delta\,,|y-\hat{y}|>\delta\end{cases}  \tag{6}\]  
where \(\delta\) is a hyperparameter controlling the threshold at which the loss function transitions from MSE to MAE. When the error is small, MSE is used to ensure gradient smoothness; when the error is large, MAE is used to mitigate the impact of outliers. This function is continuous and differentiable at \(\left|\hat{y}-\hat{y}\right|\), ensuring the stability of the optimization process. The backpropagation through time (BPTT) of LSTM requires computing the gradients of the loss with respect to each parameter. The core steps include the following:  
Gradient of the output layer:  
\[\frac{\partial L}{\partial h_{t}}=\frac{\partial L}{\partial\hat{y}_{t}}\cdot \frac{\hat{\partial y}_{t}}{\partial h_{t}}\]  
Gradient of the output gate:  
\[\frac{\partial L}{\partial o_{t}}=\frac{  
Gradient of the input gate:  
\[\frac{\partial L}{\partial t_{t}}=\frac{\partial L}{\partial C_{t}}\cdot\tanh(W_{C }[h_{t-1},x_{t}]+b_{C})\frac{\partial L}{\partial W_{i}}=\sum_{t}\frac{\partial L }{\partial t_{t}}\cdot i_{t}(1-i_{t})\cdot[h_{t-1},x_{t}]^{T}\]  
Gradient of the candidate cell state:  
\[\frac{\partial L}{\partial\widetilde{C}_{t}}=\frac{\partial L}{\partial C_{t} }\cdot i_{t}\frac{\partial L}{\partial W_{C}}=\sum_{t}\frac{\partial L}{ \partial\widetilde{C}_{t}}\cdot(1-\tanh^{2}(\widetilde{C}_{t}))\cdot[h_{t-1},x_{t}]^{T}\]  
The Adam optimizer used in this paper updates gradients in the following steps (taking the \((t+1)\ -th\) iteration as an example):  
1. Compute the current gradient: \(g_{t}=\
abla_{\theta_{t}}L(\theta_{t})\), which is the gradient of the loss function with respect to \(\theta_{t}\) parameter
2. Update the first moment \(m_{t+1}\) and second moment \(v_{t+1}\): Exponential moving average update: \[m_{t+1}=\beta_{1}\cdot m_{t}+(1-\beta_{1})\cdot g_{t}\] \[v_{t+1}=\beta_{2}\cdot v_{t}+(1-\beta_{2})\cdot g_{t}^{2}\]  
where \(\beta_{1}\) and \(\beta_{2}\) are hyperparameters (\(\beta_{1}=0.9,\ \beta_{2}=0.999\)), controlling the decay rate of historical information. 3. Bias correction (to address initialization bias): Since \(m_{0}=0\) and \(v_{0}=0\), \(m_{t}\) and \(v_{t}\) in the early iterations are biased toward zero. Corrections are applied as \[\dot{\vec{m}}_{t+1}=\frac{m_{t+1}}{1-\beta_{1}^{t+1}}v\] \[\dot{v}_{t+1}=\frac{v_{t+1}}{1-\beta_{2}^{t+1}}\]  
where \(\beta_{1}^{t+1}\) and \(\beta_{2}^{t+1}\) are bias correction factors that diminish as the iteration count t increases. 4. Update the parameter \(\theta_{t+1}\): Combine the first moment, second moment, and learning rate \(\eta\) to compute the parameter update: \[\theta_{t+1}=\theta_{t}-\eta\cdot\frac{\ddot{m}_{t+1}}{\sqrt{\dot{v}_{t+1}+ \epsilon}}\] where \(\epsilon\) is a small value \(\left(1\ \times\ 10^{-8}\right)\) to prevent division by zero.