# A Spatio-Temporal Encoding Neural Network for Semantic Segmentation of Satellite Image Time Series
## 3 Methodology
### Related Work
While skip connections and up-sampling gradually restore deep semantic information to high-resolution space, the loss of spatial information in feature maps due to down-sampling is an irreversible process, and the lost high-resolution spatial information cannot be fully recovered. Image classification tasks involve making classification decisions by comprehensively considering the information across the entire image. Therefore, down-sampling can be utilized to expedite the enlargement of the receptive field, enabling the extraction of more abstract and robust semantic information without excessive concern for spatial resolution loss. However, for semantic segmentation tasks, which require precise classification for each pixel in an image, spatial precision is of utmost importance. As a result, we believe that improvements should not be limited to enhancing image classification networks but should focus on constructing networks tailored to semantic segmentation, a task highly sensitive to spatial information. Dilated convolutions were introduced to address image segmentation problems. Unlike the combination of convolutional and pooling layers, dilated convolutions have the advantage of rapidly increasing the receptive field while maintaining the feature map's size, thus avoiding the irreversible loss of spatial accuracy caused by down-sampling [33]. To mitigate gradient vanishing and enhance the utilization of feature maps, the output of each dilated convolution block is directly concatenated with the input of all subsequent dilated convolution blocks, without the need for up-sampling to ensure consistent feature map sizes [35]. Since the input of each dilated convolution block contains the original image, and the dilation rates increase progressively, the network can extract a rich set of multi-scale features to cope with the significant variation in object sizes within remote sensing imagery. Based on Transformer temporal encoding. Research into time series data classification initially focused on discovering specific patterns within the time series data, such as periodic and trending patterns, and performing classification by identifying these patterns. These methods have been superseded by more effective deep learning techniques that allow neural networks to automatically learn complex features within time series data,eliminating the need for manual feature engineering and selection. Models like LSTM [36] and Transformer [13] can capture dependencies and dynamic changes within time series data, enabling more accurate classification. Compared to recurrent neural network models, Transformers are more proficient at modeling long-range dependencies between elements in input sequences and support parallel processing of sequences, greatly enhancing model performance and efficiency. In particular, the Transformer encoder, through positional encoding and self-attention calculations, captures correlations between time series data. During self-attention calculations, the data are divided into multi-head, creating various subspaces. This enables the model to capture information from different perspectives, resulting in a more comprehensive understanding of the data. Finally, the information from these subspaces is integrated, and the encoded results are fed into a classifier for classification, as depicted in Figure 4.  
In this study, we primarily explore the use of dilated convolutions as a substitute for down-sampling operations, facilitating the rapid expansion of the receptive field. We achieve this by employing dense connections and varying dilation rates to obtain multi-scale features. Lastly, we employ the Transformer encoder to extract temporal features for each pixel, which are utilized for pixel-level classification.