## 4 Experiments and Results
### Results  
The encoder-decoder structure model was compared with those of ConvLSTM and SA-ConvLSTM, by varying the number of layers from one to three for both the encoder and decoder.  
As shown in Figure 5, the comparison results indicate that the SA-ConvLSTM encoder-decoder model exhibits higher prediction accuracy than the standard ConvLSTM. This is attributed to the superior capability of SA-ConvLSTM in capturing both long- and short-term spatiotemporal relationships. Generally, deeper layers yield better result because additional layers enable the encoder to learn more complex features or representations of the input data. This enhances the capacity of the model, allowing it to manage intricate input patterns and variations. In the case of the decoder, more layers can effectively produce outputs that necessitate a nuanced understanding and generation in complex tasks. Deeper decoders can make better use of the context provided by the encoder, leading to a more accurate and consistent output.  
Figure 6 shows a comparison of the SSIM values between ConvLSTM and SA-ConvLSTM for different layer configurations. In the figure, the SA-ConvLSTM models generally have higher SSIM values than the ConvLSTM models. In particular, the SA-ConvLSTM 2-layer model has higher SSIM values than the ConvLSTM model, indicating that the SA-ConvLSTM model can predict the structure more accurately. This result indicates that the higher SSIM value of the SA-ConvLSTM model better captures structural features, such as the shape, edges, and texture of the predicted frames.  
Figure 7 shows a comparison of the MAE values between ConvLSTM and SA-ConvLSTM models with different layer configurations. In the figure, the SA-ConvLSTM model generally has lower MAE values than the ConvLSTM model. This indicates that the SA-ConvLSTM model provides more accurate predictions. A lower MAE value indicates that the model makes more accurate predictions at the pixel level, demonstrating the superiority of the SA-ConvLSTM model for tasks, such as dynamic motion estimation, where accurate predictions are important.  
Figure 5: Visual performance comparison by model.  
Figure 6: SSIM comparison ConvLSTM and SA-ConvLSTM with different layer configurations.  
Figure 8 compares MSE values of the ConvLSTM and SA-ConvLSTM models with different layer configurations. For each layer configuration, the SA-ConvLSTM model exhibits a lower MSE than the ConvLSTM model. This can be interpreted as a self-attention mechanism built into the SA-ConvLSTM model, which enables the learning of complex patterns and short- and long-term dependencies in spatiotemporal data. In other words, SA-ConvLSTM goes beyond simply remembering sequential information from past frames, and selectively focuses on past information that is important for predicting the current frame, resulting in more accurate predictions.  
However, as shown in Figure 9, the number of parameters in SA-ConvLSTM is significantly higher than in ConvLSTM, resulting in longer training times for the SA-ConvLSTM model than for the standard ConvLSTM model.  
Figure 8: MSE comparison of ConvLSTM and SA-ConvLSTM with different layer configurations.  
Figure 7: MAE comparison of ConvLSTM and SA-ConvLSTM with different layer configurations.  
Figure 9: Number of parameters in ConvLSTM and SA-ConvLSTM models with different layer configurations.  
Figures 6-9 present evaluation metrics for each model, providing insights into the benefits of using SA-ConvLSTM 3-layer for both the encoder and decoder compared to other architectures. The comparisons of the predictions are detailed in Table 2. The inference time was calculated as the mean time required to infer a sequence based on the KTH dataset, in milliseconds. 'ConvLSTM 1-layer' refers to ConvLSTM with one layer for the encoder and decoder, each.