# Fault Prediction for Rotating Mechanism of Satellite Based on SSA and Improved Informer
## 3 Methods
### CNN-Informer  
In the proposed model of this paper, the informer is an improved version based on the transformer framework [27]. This framework is specifically designed to address the prediction of long time series. In this paper, a Convolutional Neural Network (CNN) is introduced before the overall informer framework. The CNN primarily includes convolutional layers, pooling layers, and fully connected layers. Due to its characteristics of weight sharing and spatial pooling, the parameters and complexity of the network model are significantly reduced [28]. The architecture diagram of the CNN-Informer algorithm is proposed in Figure 3. Convolution operation is the core of CNNs. It utilizes sliding convolution filters, which move vertically and horizontally over the two-dimensional degraded data to refine them. The mathematical expressions for convolution and activation operations can be represented as follows:  
\[V_{ij}^{n}=\sum A^{n}\oplus xf_{ij}+B_{n} \tag{7}\]  
\[Y^{n}=\sigma(V^{n})\]  
where \(V_{ij}^{n}\) is the output of convolution operation, \(n\) represents the n-th feature map, \(i\) and \(j\) correspond to the number of steps of convolution filter in vertical and horizontal directions, which \(A^{n}\) are convolution kernel matrix, \(xf_{ij}\) is the filter matrix, \(B_{n}\) is the bias term and \(\sigma\) is the activation function. The Encoder captures the input sequence and maps it to a higher-dimensional vector, which is then fed into the Decoder to generate the output sequence. In this paper, the Encoder is used to learn long-term dependencies from the operational data records of circuit devices. Before the input sequence enters the Encoder, positional and temporal codes are incorporated into the input sequence, fully utilizing the positional and temporal information embedded within it. Sine and cosine functions are employed to derive positional encoding [27]. \[PE(t,2l)=\sin\left(t/10,000^{2l/m}\right) \tag{8}\]  
\[PE(t,2l+1)=\cos\left(t/10,000^{2l/m}\right)\]  
where \(t\) is the position of the word in the sequence, \(l\) is the dimension of the word embedding vector, \(m\) is the size of the word embedding projection, and \(PE\) is the positional encoding. Figure 3: CNN-Informer architecture diagram. The Probspare self-attention layer is designed to capture dependencies between features. It does not calculate the dot product of each vector in thw Q matrix and each vector in the K matrix. It calculates the sparsity measure of any Q vector according to the query sparsity measure Formula (9) and selects the top-u vectors which play a leading role in attention calculation, using these u vectors for calculations. The \(PE(t,2l+1)\) specific formula is as follows [29]:  
\[M(q_{i},K)=\ln L_{K}+\max_{j}\left\{\frac{q_{i}k_{j}^{T}}{\sqrt{d}}\right\}- \frac{1}{L_{K}}\sum_{j=1}^{L_{K}}\left(\frac{q_{i}k_{j}^{T}}{\sqrt{d}}\right) \tag{9}\]  
\[\text{Attention(Q,K,V)}=\text{softmax}\left(\frac{\mathbb{Q}K^{T}}{\sqrt{d_{h} }}\right)\text{V} \tag{10}\]  
where \(Q\) is a sparse matrix of the same size of \(q\), and it only contains the Top-\(u\) queries under the sparsity measurement \(M(q,K)\). The Decoder mainly includes masked multi-head probabilistic self-attention and multi-head attention. The former receives the following tag vectors [29]:  
\[X_{j+1}^{t}=\text{MaxPool}\left(\text{ELU}\left(\text{Conv1d}\left[X_{j}^{t} \right]_{AB}\right)\right) \tag{11}\]  
where \([\cdot]_{\text{AB}}\) represents the attention block. It contains the Multi-head ProbSparse self-attention and the essential operations, where Conv1d() performs a 1-D convolutional filters (kernel width = 3) on a time dimension with the ELU() activation function. Multiple Probspare self-attention is a stack of several Probsphere self-attention layers. Generally speaking, there are multiple multi-head Probspare self-attention layers in the Encoder.