## 3 Proposed Methodology
### Bert Model Structure
#### 3.2.4 Residual Networks  
The incorporation of a residual network [38] (ResNet) layer within the text classification model represents a deliberate effort to enhance both the performance and stability of the model architecture. This section will examine the specific functions and advantages of this layer in greater detail. ResNet is a deep learning model that employs residual blocks to construct deep neural networks. Each residual block contains a skip connection, which enables the direct transfer of input data to the output, effectively mitigating the gradient disappearance and gradient explosion issues commonly observed in deep neural networks. To prevent the downstream classification network from deteriorating and to enhance the model's stability, the residual structure is incorporated into the model to extract multiscale text segment features. The residual network formula is presented in Equation (1).  
\[y=F(x)+x \tag{1}\]  
\[F(x)=max(0,\ xW_{1}+b_{1})W_{2}+b_{2} \tag{2}\]  
where \(x\) denotes the input data and \(F(x)\) denotes the transform operation in the residual block. The transformation operation taken in this paper is shown in Equation (2). The output \(y\) can be obtained by adding \(x\) to the transformation operation \(F(x)\). This process can be seen as a correction to the input \(x\), thus making it easier for the model to learn the important information in the input \(x\). The specific functions and advantages of this layer are as follows:  
* Performance Enhancement: ResNet layers are renowned for their ability to facilitate the training of very deep neural networks. By incorporating residual connections, where the output of one layer is added to the input of another, ResNet layers alleviate the vanishing gradient problem encountered in deep networks. This enables more efficient gradient flow during training, facilitating the learning of intricate patterns and representations within the input text.
* Stability Improvement: The residual connections within the ResNet layer introduce skip connections that bypass certain layers in the network. This allows the model to retain information from earlier layers, mitigating the risk of information loss or degradation as the input propagates through successive layers. Consequently, ResNet layers promote greater stability and robustness in the model's predictions, reducing the likelihood of overfitting and improving generalization performance.
* Detailed Information Capture: By summing the outputs of different fully connected layers with the input vectors, the ResNet layer effectively integrates detailed information from multiple scales and abstraction levels. This enables the model to capture fine-grained textual features and semantic nuances present in the input text, enhancing its discriminative power and expressive capacity.
* Hierarchical Feature Fusion: The residual connections within the ResNet layer enable hierarchical feature fusion, allowing the model to combine information from different layers and abstraction levels. This facilitates the integration of both low-level and high-level features, enabling the model to capture complex relationships and interactions within the input text.