## 2 Related Work
Two principal methods of gradient clipping exist: gradient paradigm clipping, which preserves the direction of the gradient vector while reducing its length, and gradient value clipping, which may alter the direction of the gradient vector but ensures that the absolute value of the gradient remains within a reasonable range. Although gradient clipping plays an important role in preventing gradient explosion, it has two main limitations. Firstly, gradient clipping can only be used to prevent gradient explosion, and for another common problem, gradient vanishing, where the gradient values are too small for the model to effectively learn, gradient clipping does not provide a direct solution. Secondly, gradient clipping requires repeated experiments and manual adjustment of thresholds to achieve satisfactory results. This is because the appropriate clipping threshold often depends on the specific model structure and dataset without a universal standard, requiring researchers or engineers to determine the optimal clipping level through continuous experimentation. Normalization methods, represented by batch normalization (BN), are one type of these approaches. BN normalizes the values within a batch for the inputs of neuron functions, ensuring that the model can continue learning with less internal covariate shifting and input distribution during training, thereby accelerating the process. In addition, the batch normalization method can stabilize the activation values of the network throughout the training process, thereby reducing the gradient problem that arises due to changes in data distribution. However, the batch-normalization method is susceptible to alterations in the batch size setting. In instances where the batch size is relatively small, the estimation of the mean and variance of the samples within the batch may not be sufficiently accurate. Furthermore, the forced scaling of the input data may result in the destruction of the feature distribution of the original data. BN has shown good performance in conventional deep neural networks, but there is relatively less research on its application in RNN-type networks, and some conclusions are contradictory: [PERSON] found in their study [24] that horizontal BN had a detrimental effect, while vertical BN could accelerate parameter convergence. [PERSON]'s research also reached the conclusion that the horizontal BN has a detrimental effect and additionally found that if the depth of the vertical direction network is not deep enough, vertical BN would also have a detrimental effect [25]. However, [PERSON] overturned the aforementioned conclusion [26], proving that horizontal BN can simultaneously accelerate training convergence and improve model generalization. He believed that the reason why BN does not perform well in the horizontal direction in the first two works is likely due to the improper setting of the scale parameter of the BN. Subsequently, [PERSON] introduced BN into LSTM models [27], achieving human activity recognition on smartphones, thus validating the feasibility of BN in RNN-type models. These normalization methods are widely used in feedforward neural network models, showing good performance in handling image data, but there is a lack of research on their effectiveness in RNN models. In conclusion, while these three categories of methods are effective to some extent in mitigating the gradient explosion and vanishing issues in RNN-type models, there still exist limitations such as slow convergence speed, complex hyperparameter tuning, and a lack of effective mathematical explanations. Therefore, this paper proposes a novel weight-gradient normalization method tailored for RNN models.