## 2 Related Work  
As mentioned above, the causes of the gradient vanishing and exploding phenomenon in deep learning are explained by two categories: gradient cumulative multiplication and neuron inhibition. Specifically, in the case of recurrent neural network (RNN) models, due to their unique architecture and weight reuse, the model is more prone to encountering gradient issues during the training process. As early as the 1990s, [PERSON] and others argued that training neural network models over long time periods was difficult due to the phenomenon of gradient vanishing in models like RNNs [15]. [PERSON] used a model structure with only a single recurrent neuron as an example to illustrate that, as the model was trained, the partial derivatives of the model's objective function with respect to the model's recurrent layer weights would decrease and even vanish as the model's time period increased. Later, in 2012, [PERSON] and others discovered that under different weight initialization conditions, model training could also lead to situations where the partial derivatives of the objective function with respect to the weights increased and approached infinity as the time period progressed, a phenomenon termed gradient explosion [16]. With the rise of deep learning, numerous researchers found that gradient vanishing or explosion phenomena could also occur when training deep feedforward neural networks [17]. Compared to RNN-like models, the structure of feedforward neural network models is relatively simpler, and by adjusting hyperparameters reasonably, gradient problems can be easily suppressed and resolved. There are multiple existing solutions to gradient problems, broadly classified into three categories: model transformation, gradient clipping, and normalization. Model transformation methods include altering neuron functions, converting neuron functions to gate structures, and employing residual structures such as ResNet. Changing neuron functions involves replacing traditional Sigmoid [7] or Tanh [8] functions of Simple-RNN recurrent layers with novel functions like ReLU [9], SeLU [18], or ELU [19]. These novel neuron functions effectively expand the activation range of neuron functions compared to traditional sigmoid and tanh functions, preventing individual neurons from falling into inhibition zones and losing learning capability. Additionally, replacing neuron functions of recurrent layers with gate structures can significantly enhance the memory capacity of RNN models. Commonly optimized RNN models with gate structures include LSTM [20] and GRU [21]. These approaches address the gradient problem by transforming the neuron function into a threshold function. They are commonly employed in the processing of time-series signals and causal data due to their capacity to mitigate the occurrence of gradient vanishing or exploding during model training to a certain extent. Nevertheless, the necessity to maintain and update multiple threshold structures in LSTM models renders them more computationally expensive and difficult to parallelize effectively. Consequently, models with very long sequences remain susceptible to the gradient problem. Related studies [22] indicate that compared to gradient explosion, models like LSTM are better at suppressing gradient vanishing. These methods are empirical techniques that do not fundamentally solve gradient problems mechanistically, and effective mathematical explanations for gradient problems in RNN models are still lacking. Gradient clipping [23] is currently a common technique used in training neural networks, the core idea of which is to limit the gradient value before the gradient is updated. In the event that the magnitude of the gradient exceeds a pre-established threshold, it is scaled to fall within the specified range.