## 3 Methods
### FFEAR-Swin Transformer
To address this issue, the second Swin Transformer block employs the shifted window-based multi-head self-attention (SW-MSA) module, which introduces shifted W-MSA. In this module, the windows are shifted, allowing for information exchange between adjacent windows and enhancing inter-window information interaction. The remaining steps are similar to those in the W-MSA-based Swin Transformer block, including Layer-Norm, MLP, and residual connections. It is important to note that W-MSA and SW-MSA are used in pairs, not individually, as each Swin Transformer block alternates between the two. The process can be expressed using Equations (3) and (4). \[\hat{x}^{l+1}=SWMSA\left(LN\left(x^{l}\right)\right)+x^{l} \tag{3}\]  
\[x^{l+1}=MLP(LN(\hat{x}^{l+1}))+\hat{x}^{l+1} \tag{4}\]  
In the equations, the \(SWMSA(\cdot)\) operation is similar to \(WMSA(\cdot)\) but incorporates a shifted window strategy to enable cross-window information exchange. The output \(x^{l+1}\) corresponds to the feature representation after the shifted self-attention module, while \(x^{l+1}\) represents the final output of the current block after applying Layer Normalization, the MLP module, and a residual connection--serving as the input to the subsequent block. As the network depth increases, the number of tokens is progressively reduced through the patch merging layers, with each layer reducing the token count to one-fourth of its previous value while doubling the number of channels. Each patch merging layer consolidates each 22 neighboring pixel block into a single patch, creating four feature maps from the original input. These four feature maps are then concatenated along the depth dimension, followed by a Layer Norm layer. Finally, a fully connected layer performs a linear transformation along the depth of the feature maps, increasing the depth from \(C\) to \(2C\). The first patch merging layer and the second Swin Transformer block together constitute Stage 2. This process is repeated to form Stage 3 and Stage 4. After passing through four stages, the original input image is transformed into tokens of size \(\frac{H}{32}\times\frac{W}{32}\times 8C\). Subsequently, these tokens are processed by MHAR, which is specifically designed to address the unique characteristics of pavement damage images. This self-attention mechanism enhances the extraction of features that might be overlooked due to the low damage ratio, thus making better use of the information contained in the images. The number of attention heads in this multi-head attention mechanism is consistent with the number of heads used in the previous stages, and the embedding dimensions match those of Stage 4. In MHAR, we also incorporate the FFE block to capture localized, subtle features from the input tokens. This approach significantly improves the model's ability to represent features. Afterward, multi-head self-attention is employed to compute attention across multiple heads. This process is described by Equations (5) and (6). \[x^{l+1}=Dropout(LN(MHA(\hat{x}^{l+1}))) \tag{5}\]  
\[x^{l+1}=ReLU(BN(PC(DC(x^{l+1})))+x^{l+1}) \tag{6}\]  
In these equations, \(Dropout(\cdot)\) serves as a regularization technique to prevent overfitting by randomly discarding a subset of features during training. \(LN(\cdot)\) denotes Layer Normalization, which stabilizes the training process. \(MHA(\cdot)\) refers to multi-head self-attention, which is used to capture global dependencies within the input features. \(DC(\cdot)\) represents depthwise convolution, which applies convolution independently to each input channel, thereby reducing computational cost. \(PC(\cdot)\) denotes Pointwise Convolution (\(1\times 1\) convolution), used for channel mixing or dimensionality adjustment. \(BN(\cdot)\) refers to Batch Normalization, which helps stabilize and accelerate gradient propagation. \(ReLU(\cdot)\) is a nonlinear activation function that enhances the model's representational capacity.