# A Spatio-Temporal Encoding Neural Network for Semantic Segmentation of Satellite Image Time Series
## 1 Introduction
[15] used the Swin Transformer as an encoder to extract features, while the CNN was employed for multi-scale feature extraction and served as the decoder. [PERSON] et al. [16] introduced the dual-encoder structure, which utilizes the Transformer to cross-fuse global multi-scale semantic information and employs ASPP to extract context information with high-level semantics. GLFFNet [17] is another approach that combines two different network architectures, the Transformer and CNN, to extract global high-level interaction features and obtain low-level local features. [PERSON] et al. [18] embedded the Swin Transformer into the classical U-Net network, creating a dual-encoder structure wherein the extracted global contextual information compensates for the local nature of the convolution. As described in the aforementioned literature, local information, multi-scale features, and global context all play indispensable roles. However, the diverse phenological phenomena [19; 20; 21] of crops should not be overlooked. Their complex temporal patterns can be accurately represented through SITS [22]. Therefore, for the segmentation of agricultural land in remote sensing images, relying solely on spatial and spectral features is not sufficient. It is also crucial to consider the annual cyclic phenological characteristics of crops. Moreover, capturing the phenological characteristics of various land features as they change over time in SITS can effectively mitigate the challenges posed by the \"same material, different spectra; different material, same spectra\" issue encountered in single-temporal remote sensing imagery. This approach also enhances the utility of remote sensing imagery. In the early stages of land cover classification using SITS, the conventional approach involved extracting time, statistical, and spectral index information for each pixel from the imagery to create time series. These data were then combined with classical machine learn ing algorithms for classification [2; 19; 23; 24]. [PERSON] et al. [25] introduced a method based on ensemble encoders that successfully extracted statistical information about spectral distribution within the spatial extent of land parcels. They organized this statistical data into time series and applied a time attention encoder for classification. The aforementioned methods, utilizing feature engineering, made use of time information to a certain extent. However, with the continuous development of deep learning technology, an increasing number of studies have begun to explore the use of deep learning models to extract time features from SITS. U-TAE [1] represents the first end-to-end pixel-level segmentation framework for agricultural land within SITS. This framework includes an encoder for image time series, which employs convolutional blocks and a lightweight time attention encoder (L-TAE) to extract rich multi-scale spatio-temporal features. Additionally, they publicly released the first SITS dataset with semantic annotations, known as PASTIS [1]. While the use of L-TAE significantly reduces computational complexity, it fundamentally relies on learned weights for the fusion of feature map time series and does not consider or extract the time series features within SITS. In this paper, we introduce a novel end-to-end pixel-level semantic segmentation framework for SITS called the spatio-temporal encoding neural network (STENN). The framework is designed to reveal the representation of intrinsic temporal patterns of land features in images, addressing the limitations of previous methods in exploiting the potential information along the temporal dimension of images and, consequently, the underutilization of spatio-temporal information. The framework comprises two encoding modules: a dilated convolution-based spatial encoding module and a Transformer encoder-based temporal encoding module.