# A Spatio-Temporal Encoding Neural Network for Semantic Segmentation of Satellite Image Time Series
## 4 Experiments and Analysis
### Experimental Detail  
The parameters for the six comparative algorithms were set according to the referenced papers. For our proposed STENN model, the parameter settings are as shown in the following table:  
In Table 2, the number of dilation rates determines the frequency of dilated convolutions. In our experiments, we employed the Adam optimizer with default parameters, set the batch size to two, and defined the learning rate as 0.001. Considering that the images in each sequence were captured at different time points, there may be variations in the sample distribution within each batch. To address this issue, we incorporated group normalization in the encoder, specifying four groups, instead of using batch normalization.  
To quantitatively and qualitatively validate the effectiveness of the STENN model, we selected U-TAE [1], ConvLSTM [37; 38], ConvGRU [39], U-ConvLSTM [40], U-BiConvLSTM [39], and 3D U-Net [40] as comparative algorithms.  
* The backbone network of U-TAE [1] is U-Net. At the lowest resolution, the attention-based time encoder generates a set of time attention masks for each pixel. After spatial interpolation, these attention masks fuse the feature map time sequences at all resolutions into a single feature map. In the coding branch, four sets of group normalization are utilized, while batch normalization is employed in the decoding branch. A temporal encoding approach employing an L-TAE with 16 heads and a key-query space dimensionality of four was chosen.
* ConvLSTM [37; 38] and ConvGRU [39] are recurrent neural networks, primarily replacing all linear layers in the model with convolutional layers. Their hidden sizes are set to 160 and 188, respectively.
* U-ConvLSTM [40] and U-BiConvLSTM [39] use U-Net as their backbone network, replacing L-TAE in the network with ConvLSTM [37] or bidirectional ConvLSTM. In comparison to the original method, batch normalization in the encoder is replaced with group normalization. Like the U-TAE, the Backbone utilizes the U-Net architecture, simply replacing the L-TAE with ConvLSTM or BiConvLSTM in their respective positions.  
\begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Layer Name** & **Kernel Size** & **Dilation Rate** & **Input Channel** &
\begin{tabular}{l} **Output** \\ **Channel** \\ \end{tabular} \\ \hline Conv Layer-1 & \(3\times 3\) & [1; 2; 3] & 10 & 16 \\ \hline Conv Layer-2 & \(3\times 3\) & [2; 4; 6] & 26 & 32 \\ \hline Conv Layer-3 & \(3\times 3\) & [2; 4; 6] & 58 & 32 \\ \hline Conv Layer-4 & \(3\times 3\) & [2; 4; 6] & 90 & 64 \\ \hline \multicolumn{5}{c}{Concat (dim = channel) \& Reshape} \\ \hline \multicolumn{5}{c}{Transformer encoder (input = 144, head = 8)} \\ \hline \multicolumn{5}{c}{Mean (dim = time) \& Reshape} \\ \hline Conv Layer-5 & \(3\times 3\) & [1, 1] & 144 & 20 \\ \hline \hline \end{tabular}
\end{table}
Table 2: STENN model configuration.  
* The encoding space of 3D U-Net [40] is three-dimensional, allowing it to simultaneously process spatial and spectral dimensions. Finally, it performs mean fusion in the temporal dimension. The network comprises five consecutive 3D convolutional blocks, conducting spatial down-sampling after the second and fourth blocks. Each convolutional block doubles the channel count of the processed feature maps, with the innermost feature map having a channel size set at 128. Leaky ReLU and 3D batch normalization are employed within the convolutional blocks of this architecture.