# A Spatio-Temporal Encoding Neural Network for Semantic Segmentation of Satellite Image Time Series
## 3 Methodology  
### Related Work  
In this section, we have introduced some of the state-of-the-art works related to spatial encoding and temporal encoding, including models based on CNN and models based on Transformers. These models have provided valuable insights and references for us in extracting spatio-temporal features from SITS. Based on CNN spatial encoding. The conventional approach to semantic segmentation involves using pretrained image classification networks such as VGG [27] and ResNet [28] as backbone networks. These networks are characterized by multiple rounds of convolutional layers and down-sampling layers, which expedite the expansion of the receptive field and, consequently, the extraction of deep but low-resolution semantic information from images. While this semantic information is beneficial for image classification tasks, it is not suitable for dense pixel-level classification tasks. To adapt these networks for semantic segmentation, it is necessary to compensate for the loss of spatial resolution caused by down-sampling. Primarily in the decoding phase, this is achieved by integrating  
Figure 3: This image excellently illustrates the annual cyclic variations of crops in the SITS dataset. From the image, it is evident that even though different crops may exhibit similar spectral characteristics in single-temporal remote sensing images (as indicated by the red circles), their patterns of change throughout the annual cycle vary significantly. This cyclical variation not only reflects the unique growth habits of various crops but also provides us with rich information for understanding and predicting the dynamics of agricultural production. mid- to high-resolution feature maps generated at various stages of the encoder using skip connections, as seen in typical networks like FCN [29] and U-Net [9]. FCN, as the first network to perform end-to-end segmentation using fully convolutional layers, revolutionized research in semantic segmentation. The U-Net, originally designed for medical image segmentation, employs a symmetric network structure, comprising a contracting pathway to capture contextual information and an expanding pathway symmetric to it to support precise localization. By combining deep, low-resolution semantic information with shallow, high-resolution surface information through skip connections, U-Net is capable of performing dense and fine-grained segmentation tasks. Consequently, convolution has become the foundation for semantic segmentation tasks [30], and the encoder-decoder architecture has become a popular framework for semantic segmentation networks. The encoder is responsible for feature extraction, and the decoder projects the high-level semantic features learned by the encoder into high-resolution pixel space to achieve dense, fine-grained classification. For example, SegNet [27] utilizes the index of max-pooling from the encoding phase in its up-sampling process, reducing the number of parameters and computational workload compared to transposed convolution. However, down-sampling in classification networks results in irreversible spatial information loss. DeepLab V1 [31] introduced dilated convolutions to mitigate the reduction in spatial resolution and spatial insensitivity caused by down-sampling. DeepLab V2 [32] proposed the ASPP module for multi-scale targets, consisting of four parallel dilated convolutions modules with different dilation rates to extract multi-scale features. In addition to adjusting the dilation rates of the ASPP module, DeepLab V3 [33] added a global pooling layer to extract image-level features. DeepLab V3+ [34], building upon DeepLab V3's encoder, introduced depth-wise separable convolution to reduce the number of parameters and included a decoder module to refine object boundaries, significantly enhancing the network's segmentation performance.