## 3 Methods
### FFEAR-Swin Transformer  
The FFEAR-Swin Transformer is an enhanced variant of the original Swin Transformer. After passing through four original Swin Transformer blocks, the FFEAR module is in  
Figure 1: Overall architecture of FFEAR-Swin Transformer. The part (**a**) is FFE. The part (**b**) is MHAR. Figure 3: Architecture of the MHAR. The part (c) is the multi-head self-attention mechanism. Figure 2: Architecture of FFE. troduced for feature-based attention re-filtering and embedding. Finally, the processed features are passed to the classification head layer. This approach maintains the original input dimensions, effectively avoiding additional model complexity while still achieving excellent classification results, including higher success rates and precision. As illustrated in Figure 1, when a pavement damage image is input into the model, it is first processed by the patch partition module. This module divides the input image of size \(HWC\) into non-overlapping patches of equal size, where \(H\) represents the height of the original image, \(W\) represents the width, and \(C\) denotes the number of channels (typically \(C=3\) for RGB images). If the input is a grayscale image or another type, it needs to be transformed into an RGB image with three channels before processing. After the patch partition module, an image of size \(H\times W\times 3\) is segmented into \(N\times(P\times P\times 3)\) patches, where each \(P\times P\times 3\) patch is treated as a token. Here, \(N\) denotes the number of tokens, \(P=4\) represents the patch size, and 3 is the number of channels. Specifically, each image of size \(H\times W\times 3\) is divided into \(N=\frac{H}{4}\times\frac{W}{4}\) tokens, with each token having a size of \(P\times P\times C=4\times 4\times 3=48\) dimensions. In the second step, a fully connected layer (linear embedding) projects these \(N\) tokens of 48 dimensions to an arbitrary dimension \(C\), resulting in a linear embedding of size \(N\times C\). This data is then fed into the first Swin Transformer block with self-attention. During the first input stage, the number of tokens remains unchanged at \(N=\frac{H}{4}\times\frac{W}{4}\). The first Swin Transformer block and the linear embedding layer are considered as Stage 1. When tokens are fed into a Swin Transformer block, they first pass through a Layer-Norm layer before proceeding to the window-based multi-head self-attention (W-MSA) module. This module divides the image into non-overlapping windows and performs self-attention calculations within each window independently, significantly reducing computational complexity and improving model efficiency. The process can be expressed using Equation (1). \[\hat{x}^{l}=WMSA\left(LN\left(x^{l-1}\right)\right)+x^{l-1} \tag{1}\]  
In the equation, \(x^{l-1}\) denotes the output feature from the previous \(l-1\) layer. \(LN(\cdot)\) represents Layer Normalization, which standardizes the input feature distribution. \(WMSA(\cdot)\) denotes the window-based multi-head self-attention mechanism, where self-attention is computed within non-overlapping local windows. The intermediate output \(hatx^{l}\) refers to the feature map after the attention module in the current layer. After the W-MSA stage, a residual connection layer is applied, followed by another Layer-Norm layer, then the tokens proceed through the MLP layer and undergo another residual connection. This completes the first Swin Transformer block.The process can be expressed using Equation (2). \[x^{l}=MLP(LN(\hat{x}^{l}))+\hat{x}^{l} \tag{2}\]  
In the equation, \(MLP(\cdot)\) denotes a multi-layer perceptron block consisting of fully connected layers with GELU activation and Dropout regularization. \(x^{l}\) is the final output of the current layer, serving as the input to the next layer. However, using the W-MSA module means that self-attention is computed only within each independent, non-overlapping window, potentially missing direct information exchange between adjacent windows.