# High-Precision Tea Plantation Mapping with Multi-Source Remote Sensing and Deep Learning
## 3 Methods
### Semantic Segmentation-Based Tea Classification Models
#### 3.3.3 Swin-UNet  
The conventional CNN-based segmentation methods inherently suffer from limitations in capturing global, long-term semantic information, as local convolutional operations struggle to learn such information effectively. Inspired by the significant success achieved in natural language processing by transformers, the Swin transformer (ST), a transformer-based visual model, has gradually been applied to image recognition tasks. Transforming images into sequences, transformers can weightedly process any position in the sequence through self-attention mechanisms, providing stronger capabilities in handling long-range dependencies compared to CNN's local convolutions (Figure 11). Hence, constructing a Swin-UNet, a symmetric encoder-decoder structure based on transformers with skip connections, enables the simultaneous utilization of multi-level encoders, skip connections for multi-scale fusion, and the transformer's ability to extract long sequence information [38]. The specific architecture of the Swin-Unet model used in this study is illustrated in Figure 12. Initially, to convert input images into sequence embeddings, patch partition divides the image into non-overlapping 4\(\times\)4 patches. Subsequently, linear embedding projects the feature dimension to an arbitrary dimension, followed by three consecutive downsampling steps. The downsampling process involves two ST blocks for representation learning and patch merging operations, reducing spatial resolution by half and doubling the feature dimension. Patch merging divides input patches into four parts, concatenates them along the feature dimension, and then reduces the feature dimension by half through a linear layer, replacing the pooling operation in traditional downsampling. After three consecutive downsampling steps, the image's spatial resolution becomes 1/8 of the linear embedding, and the feature quantity expands eightfold. For the bottom connection, two consecutive STs are employed in Swin-UNet to learn deep feature representations. Correspondingly, on the model's right side, four consecutive upsampling operations are performed, followed by linear projection to directly obtain classification results. The upsampling process includes two consecutive ST blocks and Patch expanding operations to double the spatial resolution and halve the feature dimension. Specifically, patch expanding doubles the feature maps of adjacent dimensions before upsampling through a linear transformation layer and then utilizes the rerange operation to double the resolution while reducing the feature dimension to 1/4. During the upsampling process, skip connections are employed to fuse multi-scale features with upsampled features, maintaining the same dimensionality through a linear layer. The ST block module is detailed as shown in Figure 11, comprising a LayerNorm (LN) layer, a multi-head self-attention module, residual connections, and a multi-layer perceptron (MLP) with a GELU activation function. The difference between the two consecutive ST blocks lies in the adoption of the window-based multi-head self-attention (W-MSA) module and the sliding window-based multi-head self-attention (SW-MSA) module in the two blocks. Following this mechanism, consecutive STs are defined in Equations (11)-(14). \[\tilde{z}^{l}=W-MSA\Big{(}LN\Big{(}z^{l-1}\Big{)}\Big{)}+z^{l-1} \tag{11}\]  
Figure 11: The diagram of the Swin transformer block module. The self-attention mechanism in transformers is a crucial step for capturing context information and modeling sequences. In W-MSA, patches are divided based on a fixed-size window, allowing attention weights to be computed only within the window. While this reduces computation and maintains local dependencies, it also constrains the attention mechanism's scope, directly affecting model performance due to the window size. The sliding window self-attention mechanism, SW-MSA, is an improvement over W-MSA.