## 3 Materials and Method
### Proposed Method
#### 3.2.3 Skip Attention Mechanism  
In this study, we propose an improved attention mechanism--skip attention mechanism--based on the traditional Transformer structure. This mechanism is innovatively optimized to enhance the efficiency and accuracy of the model when processing large-scale datasets, as shown in Figure 7. By incorporating the skip mechanism, our model can effectively reduce unnecessary computations, particularly in handling complex tasks that require focused attention, significantly improving performance. Traditional Transformer models utilize self-attention mechanisms to process sequential data, calculating attention weights for every element against all others in the sequence to capture the intricate relationships within the data. While powerful, this mechanism encounters substantial computational resource consumption when dealing with very large datasets. For instance, for a sequence of length \(n\), the self-attention mechanism requires computing weights for \(n\times n\) pairs of elements, which becomes highly inefficient with large amounts of data. In contrast, the skip attention mechanism introduces skip connections, allowing the model to bypass certain layers during feature extraction and relationship modelling, and directly relay information from lower layers to higher ones. This not only reduces computational load but also helps maintain the benefits of deep networks while avoiding issues like gradient vanishing. When designing the skip attention mechanism, we focused on how to achieve efficient data processing through a streamlined network structure. The specific design parameters of the model are as follows:  
* **Network layers**: Our model is designed with three layers, each including a skip attention unit and a feed-forward network unit. This design ensures sufficient model depth to handle complex data relationships while maintaining computational efficiency. * **Network width and channel numbers**: The width of each skip attention unit is set to 64, meaning each attention head processes features of dimension 64. This width was determined through experimental balancing of model performance and computational efficiency. The number of channels is set to 256 to ensure the network can capture sufficient information. The core of the skip attention mechanism is the introduction of skip connections in the self-attention computation, which can be mathematically represented as  
\[\text{SkipAttention}(Q,K,V)=\text{LayerNorm}(X+\text{MultiHead}(Q,K,V)) \tag{6}\]  
Here, \(Q,K,V\) are the query, key, and value matrices, respectively, which are obtained from the outputs of the previous layer through linear transformations; \(\text{MultiHead}(\cdot)\) denotes the multi-head attention mechanism; \(X\) represents the input sequence; and \(\text{LayerNorm}(\cdot)\) is a layer normalization operation used to stabilize the network training process. By reducing  
Figure 7: Skip attention structure diagram: This figure illustrates the basic structure of the proposed skip attention mechanism, which introduces skip connections to optimize information flow and computational efficiency within the model. The structure includes multi-head attention and feedforward network modules, where skip connections enable direct information transfer between feature layers, effectively reducing redundant computations and allowing the model to more efficiently extract and utilize key information. computational steps, the skip attention mechanism can significantly improve the training and inference speed of the model on large-scale datasets. This is particularly important in secure computing tasks requiring real-time processing of vast amounts of data, such as real-time monitoring systems and online trading platforms. Additionally, this mechanism helps more effectively propagate gradients throughout the network by directly connecting input and output layers, thereby mitigating the common problem of gradient vanishing in deep networks.