# A Data-Driven Battery Degradation Estimation Method for Low-Earth-Orbit (LEO) Satellites
## 2 Background on LEO Satellite Power System and Methods
### Transformer Network  
The transformer is a deep learning model originally designed for natural language processing (NLP) tasks but is now widely applied in various fields, including time-series data analysis, image processing, and speech recognition. Unlike traditional models, such as recurrent neural networks (RNNs) or convolutional neural networks (CNNs), the transformer effectively learns relationships in sequential data without relying on these architectures [14]. Figure 1: KOMPSAT-5 electrical power sub-system structure. The self-attention mechanism, which is at the heart of the transformer, calculates how each data point in a sequence relates to all other data points. This enables the model to consider the full context of sequential data, making it particularly effective for handling data with long-term dependencies. Additionally, the transformer overcomes its inability to learn positional order directly by incorporating positional encoding, which adds positional information to the input data. This enhancement addresses the limitations of its parallel processing structure, allowing the model to capture sequential relationships more effectively [15]. The transformer is a powerful tool for time-series data analysis. Time-series data occur continuously along the time axis, and thus, understanding past patterns and predicting future trends are critical. The transformer leverages the self-attention mechanism to learn relationships not only between the past and present but also for multistep predictions of future data [16]. The attention mechanism uses three components: query (\(Q\)), key (\(K\)), and value (\(V\)). First, the input data are transformed into these components using weight matrices. The similarity between data points is then calculated by taking the dot product of the query and key. A softmax function is applied to normalize these similarities into weights, which are subsequently multiplied by the value to generate the final output. This process can be mathematically expressed as follows:  
\[Attations\left(Q,K,V\right)=softmax\left(\frac{QK^{T}}{\sqrt{d_{k}}}\right)V\, \tag{1}\]  
where \(d_{k}\) denotes the dimension of the key vector and is used for scaling the dot product between the query and key to ensure numerical stability during computation. Instead of relying on a single attention mechanism, the transformer employs multi-head attention, which allows the model to learn relationships in the data from multiple perspectives simultaneously. Each attention head processes the data independently and captures different aspects of the input, improving the model's ability to understand complex patterns. The mathematical expression for multi-head attention is as follows:  
\[MultiHead(Q,K,V)=Concat\ (head_{1},head_{2},\ldots,head_{n})W^{\circ}\, \tag{2}\]  
where \(head_{i}\) is each attention head and \(W^{\circ}\) is a learnable weight matrix that projects the concatenated outputs of all heads into the desired dimension. Because of its lack of an inherent mechanism to learn positional information due to its fully parallel architecture, the transformer has to incorporate positional encoding to account for sequence order when processing input data. Positional encoding is designed to provide the model with information regarding the relative or absolute positions of each element in a sequence. The encoding is periodic and is computed using sine and cosine functions as follows:  
\[PE_{(pos,2l)}=\sin\left(\frac{pos}{10,000^{\frac{3}{d}}}\right)\,\qquad PE_{( pos,2i+1)}=\cos\left(\frac{pos}{10,000^{\frac{3}{d}}}\right)\, \tag{3}\]  
where \(pos\) represents the position in the sequence, \(i\) is the dimensional index of the embedding vector, and \(d\) is the embedding dimension. The transformer architecture consists of an encoder and decoder, with each component structured in layers. The encoder processes the input data using a combination of self-attention mechanisms and feedforward neural networks (FFNNs) to extract meaningful representations of the input.