# LTE: Lightweight Transformer Encoder for Orbit Prediction
## 4 Discussion  
In this research, for satellite data with repetitive patterns (e.g., K3, K3A, and K5), our model demonstrated faster execution times, required fewer parameters, and achieved reduced prediction errors. Therefore, if other researchers choose to apply our model, it appears best suited for relatively simple time series data with repetitive patterns. In such cases, they may find it helpful to begin their experiments with approximately 50 training epochs and a learning rate of 0.001. As for the selection of \(n_{\text{head}}\), it should be set as a divisor of the data dimension. When the dataset shows relatively simple repetitive patterns, we recommend using a smaller value, such as \(n_{\text{head}}=3\). However, defining the criteria for determining the simplicity of data remains challenging, and further exploration is required to establish guidelines on the optimal use of \(n_{\text{head}}\) based on data complexity. Investigating concrete criteria for setting \(n_{\text{head}}\) according to the complexity of data will be one of the topics of our future research.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & \(n_{\text{head}}\) & **Prediction Error (MSE)** \\ \hline LTE-h3 + PE + LN & 3 & 0.0000499 \\ LTE-h6 + PE + LN & 6 & 0.0000299 \\ \hline LTE-h3 + PE & 3 & 0.1211978 \\ LTE-h6 + PE & 6 & 0.0836058 \\ \hline LTE-h3 + LN & 3 & 0.0737088 \\ LTE-h6 + LN & 6 & 0.0712439 \\ \hline
**LTE-h3, ours** & ✓**3** & ✓**0.0000238** \\
**LTE-h6, ours** & **6** & **0.0000288** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Comparison of prediction performance based on the presence or absence of PE and LN, with different numbers of heads used (\(n_{\text{head}}\)). The best-performing model is indicated in bold with a checkmark, while the second-best-performing model is indicated in bold without a checkmark.  
Another direction for future research involves conducting additional experiments with more complex and diverse satellite datasets to validate the robustness and generalizability of the LTE model. To further extend our study's generalization capabilities, we plan to test the LTE model on more complex time series data, such as the Electricity Transformer Temperature (ETT) dataset [34] and the M4 dataset [35].  
Furthermore, our approach has focused on solving the orbit prediction problem solely from an orbit data-driven perspective, primarily using computer science methodologies. Future research could explore integrating this approach with traditional methods from fields such as astrophysics and statistics. By incorporating external factors that affect the space environment, such as solar activity or atmospheric conditions, as additional features, its prediction performance could be further improved. Moreover, the use of repetitive and periodic time series data collected from sensors on non-satellite objects, combined with methods such as transfer learning, could enhance the generalization ability of the model. Leveraging these techniques would enable the LTE model to adapt more effectively to diverse types of data and extend its applicability beyond satellite orbit prediction.