# A Novel FECAM-iTransformer Algorithm for Assisting INS/GNSS Navigation System during GNSS Outages
## 2 Theory of INS/GNSS Integrated Navigation System
The design can avoid the noise problems introduced by the time-stamp dislocation or the distribution deviation of the sequence [25]. More importantly, this approach allows the model to make predictions with a more global perspective, enhancing its ability to represent and describe multivariate correlations. The multi-head self-attention mechanism primarily accomplishes this task. When each measurement sequence is treated as a token, the multi-head attention mechanism evaluates the similarity between measured values by comparing the query and key derived from the input measurements. It then extracts the essential features of the sequence through the value, revealing multiple correlations among all measured values, as shown in Figure 6.

iTransformer retains the encoder structure of the Transformer while discarding the decoder structure. Its encoder includes layer normalization, a feedforward layer, and a multi-head attention mechanism. First, the data are embedded to obtain a tensor where the entire time step serves as the token. This tensor is then fed into the encoder, and finally, the result is projected to the output through a linear transformation. The process can be simply represented by the following formula [25]:

Figure 5: The framework of iTransformer. Figure 6: Multivariate attention revealing multivariate correlations between the measurements. \[h_{n}^{0} = \text{Embedding}(X_{,n}),\] Figure 2:0 Figure 2:1

where \(X\) represents the time series within the sliding window and \(X_{,n}\) represents a token of \(X\), \(\hat{Y}_{,n}\) denotes a token of the predicted future series \(\hat{Y}\), \(H\) is the tensor after embedding, and \(h\) represents each token of the tensor \(H\), with the superscripts indicating the layer index. To enable the inverting operation, the designers of iTransformer have significantly modified each module within the encoder. Layer Normalization: Layer normalization standardizes each sample across all features of its current layer, ensuring that these features follow a Gaussian distribution. The normalized data are then adjusted using learnable scaling (scale) and shifting (shift) parameters, which restore the original distributional properties of the data. This process enhances the convergence and training stability of the deep network. The process can be represented by the following formula [25]:

Figure 2:2

where \(h_{n}\) is each token of \(H\), \(\text{mean}()\) represents the function of finding the mean, and \(\text{var}()\) represents the function of finding the variance. Feedforward Network: Due to changes in the way tokens are obtained, the feedforward neural network (FNN) in the iTransformer has been redesigned to represent each variate token rather than each time-step token as in the traditional Transformer model. This redesign avoids the limitations caused by the misalignment of time steps and the over-localization of the receptive field. Self-Attention Mechanism: The attention mechanism assigns different weights to different parts of the input data, allowing the model to focus more on important information when making predictions or inferences. In the classic Transformer model, the self-attention mechanism is used to simplify time-dependent modeling by enabling the model to process each element of a sequence while considering all other elements in the sequence, thereby capturing more comprehensive contextual information. In the iTransformer, the self-attention mechanism treats the entire sequence of each variable as an independent process to capture the correlations between multiple variables, expressed by the following formula [25]:

Figure 2:3

where \(Q\) represents the query, \(K\) represents the key, \(V\) represents the value, and \(\sqrt{d_{k}}\) represents a scaling factor. Both of them are important variables in the self-attention mechanism [25]. \(\text{softmax}()\) is the activation function. \(Attention(Q,K,V)\) are attention scores.