# LTE: Lightweight Transformer Encoder for Orbit Prediction
###### Abstract  
As the focus of space exploration has recently shifted from national efforts to private enterprises, interest in the space industry has increased. With the rising number of satellite launches, the risk of collisions between satellites and between satellites and space debris has grown, which can lead not only to property damage but also casualties caused by the debris. To address this issue, various machine learning and deep learning-based methods have been researched to improve the accuracy of satellite orbit prediction and mitigate these risks. However, most studies have applied basic machine learning models to orbit prediction without considering the model size and execution time, even though satellite operations require lightweight models that offer both a strong prediction performance and rapid execution. In this study, we propose a time series forecasting framework, the Lightweight Transformer Encoder (LTE), for satellite orbit prediction. The LTE is a prediction model that modifies the encoder structure of the Transformer model to enhance the accuracy of satellite orbit prediction and reduce the computational resources used. To evaluate its performance, we conducted experiments using about 4.8 million data points collected every minute from January 2016 to December 2018 by the KOMPSAT-3, KOMPSAT-3A, and KOMPSAT-5 satellites, which are part of the Korea Multi-Purpose Satellite (KOMPSAT) series operated by the Korea Aerospace Research Institute (KARI). We compare the performance of our model against various baseline models in terms of prediction error, execution time, and the number of parameters used. Our LTE model demonstrates significant improvements: it reduces the orbit prediction error by 50.61% in the KOMPSAT-3 dataset, 42.40% in the KOMPSAT-3A dataset, and 30.00% in the KOMPSAT-5 dataset compared to the next-best-performing model. Additionally, in the KOMPSAT-3 dataset, it decreases the execution time by 36.86% (from 1731 to 1093 s) and lowers the number of parameters by 2.33% compared to the next-best-performing model. time series forecasting; orbit prediction; machine learning; transformer 202413390/electronics322437132243713224371322437132437  
collisions between satellites and with space debris, which could result in property damage and even casualties due to falling debris. As a result, accurate satellite orbit prediction has become crucial for preventing these risks. Conventionally, orbit prediction algorithms have relied on statistical and physical methods [6; 7; 8]. However, in recent years, machine learning and deep learning methods have been introduced to enhance prediction accuracy. Some notable studies have employed conventional machine learning architectures [9; 10; 11; 12; 13], but these studies were limited by their use of datasets obtained from simulations, which do not consider the irregular patterns that occur during actual satellite operations. For example, [PERSON] and [PERSON] [9] noted that the reason for the failure of physics-based prediction models is that they are not based on historical data of the space environment and resident space objects. Therefore, physics-based prediction models cannot learn the patterns of orbit prediction errors. For this reason, the authors proposed an approach using the support vector machine (SVM), a machine learning model, to learn the patterns of orbit prediction errors based on historical data. They demonstrated that the SVM shows a better prediction performance compared to physics-based prediction models, and the model's performance can be further improved by adding more training data. Also, [PERSON] [11] used the Two-Line Element (TLE) [14] set provided by The National Aeronautics and Space Administration (NASA) and the North American Aerospace Defense Command (NORAD) for the orbit prediction of Resident Space Object (RSO) satellites. They proposed a methodology that combines a machine learning model, specifically the Gradient-Boosting Regression Tree (GBRT), with a physical-based prediction model.