# LTE: Lightweight Transformer Encoder for Orbit Prediction
## 3 Results
### Prediction Performance  
Table 2 displays the prediction errors of the LTE and the baseline models. Excluding the row with column titles, rows 1 and 2 (SVR, XGBR) represent the machine learning-based models, rows 3 to 6 (LSTM, VAE, GRU, Bi-LSTM) represent the Conventional Deep Learning models, and from row 7 to the last row all eight models are Transformer-based models. Due to the characteristics of satellite orbits, even an error range of 0.001 can translate to actual orbital errors of tens to hundreds of kilometers. Therefore, our experiments present results with precision up to the seventh decimal place to ensure very accurate measurements of errors. The value in bold with a checkmark indicates the best-performing model and the value of the second-best-performing model is indicated in bold without a checkmark. Additionally, the value that is underlined is the best-performing model excluding our LTE model. \"Improvement\" refers to the percentage of accuracy improvement between the best-performing model (bold with a checkmark) and the best-performing model except ours (underlined). The prediction error is calculated using MSE, with lower values indicating a better performance. Also, \"h3\" means that \(n_{\text{head}}\) is set to 3, while \"h6\" in the model name indicates that \(n_{\text{head}}\) is set to 6. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multicolumn{5}{c}{**Prediction Error (MSE)**} \\ \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{\(n_{\text{head}}\)} & \multicolumn{3}{c}{**Prediction Error**} \\ \cline{3-5}  & & **K3** & **K3A** & **K5** \\ \hline SVR & - & 0.0033472 & 0.0015751 & 0.0010871 \\ XGBR & - & 0.0633434 & 0.0576222 & 0.0653850 \\ \hline LSTM & - & 0.0156621 & 0.0038404 & 0.0004618 \\ VAE & - & 0.0713109 & 0.0211886 & 0.0016111 \\ GRU & - & 0.0154912 & 0.0204662 & 0.0101774 \\ Bi-LSTM & - & 0.0086924 & 0.0041601 & 0.0003658 \\ \hline Transformer-h3 & 3 & 0.1071742 & 0.1492044 & 0.1120841 \\ Transformer-h3 (encoder only) & 3 & 0.0000488 & 0.0000214 & 0.0000010 \\ Transformer-h3 (decoder-only) & 3 & 0.0009327 & 0.0000934 & 0.0000011 \\
**LTE-h3, ours** & 3 & ✓ **0.0000241** & **0.0000107** & ✓ **0.0000007** \\ \hline Transformer-h6 & 6 & 0.0778209 & 0.1337293 & 0.1125202 \\ Transformer-h6 (encoder only) & 6 & 0.0000538 & 0.0000125 & 0.0000018 \\ Transformer-h6 (decoder only) & 6 & 0.0003631 & 0.0000558 & 0.0000030 \\
**LTE-h6, ours** & 6 & **0.0000292** & ✓ **0.0000072** & **0.0000008** \\ \hline Improvement & - & 50.61\% & 42.40\% & 30.00\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Orbit prediction performance of 14 different models. The prediction error is calculated using the MSE. The best-performing model is indicated in bold with a checkmark, the second-best-performing model is indicated in bold without a checkmark, and the best-performing model excluding our model is underlined. “Improvement” is the percentage of accuracy improvement between the best-performing model (bold with a checkmark) and the best-performing model except ours (underlined). “h3” in the model name indicates that \(n_{\text{head}}\) is set to 3, while “h6” in the model name indicates that \(n_{\text{head}}\) is set to 6. Performance on K3: The LTE model consistently performed well regardless of the \(n_{\text{head}}\) value, with the best performance achieved by the LTE-h3 model, yielding a prediction error of 0.0000241. The second-best-performing model is LTE-h6, with a prediction error of 0.0000292. Among a total of 12 models, excluding our models, Transformer-h3 (encoder only) was the best-performing model, with a prediction error of 0.0000488. As a result, LTE-h3 showed a 50.61% improvement compared to Transformer-h3 (encoder only). Performance on K3A: Similarly, the LTE model performs well across all \(n_{\text{head}}\) values. LTE-h6 had the lowest prediction error of 0.0000072, followed by LTE-h3 with 0.0000107. The best-performing model excluding our models was Transformer-h6 (encoder only), with a prediction error of 0.0000125.