## 3 Construction and Training of the Multivariate Attention Neural Network
### Training of the Neural Network  
For the MANN described in the previous subsection, the neural network model is constructed in the Pytorch environment in this work. During the training process, the generated 100,000 samples are divided into two groups: a training dataset including 80% of the samples and a validation dataset including the remaining 20% of the samples. The MANN is trained for 1000 iterations, and in each iteration, the training dataset is randomly shuffled into ten batches (each batch contains 8000 samples). The mean square error (MSE) is used as the loss function of neural network optimization, and Adaptive Moment Estimation (Adam) is used to optimize the neural network. We set each multivariate attention layer in the MANN to contain four heads, each with a dimension of 64. We first considered two sizes of MANN: one with depth 2 and dim 16, and a second with depth 8 and dim 256. The training results for the two sizes of MANN are shown in Figure 5a,b. As can be seen from Figure 5a,b, there is no apparent overfitting phenomenon when the MANN is used. In addition, there is no obvious deviation between the MSE loss value of the training dataset and the validation dataset. Meanwhile, the minimum value of the MSE loss in the validation dataset of the first small-size MANN is about \(10^{-7}\), while that in the validation dataset of large-size MANN is about \(10^{-8}\), indicating that increasing the depth and dimension of the MANN can help to improve the prediction results of neural networks. Figure (c)c,d shows the training process of a traditional DNN to show the advantages of the MANN neural network. As shown in Figure (c)c, the DNN shows obvious overfitting in the training process, and the MSE loss in the training dataset is significantly smaller than the MSE loss in the validation dataset. At the same time, the MSE loss values of the DNN are at least one order of magnitude larger than those of the MANN. This shows that the developed MANN has advantages in prediction accuracy and generalization compared with the traditional DNNs. The training performances of MANNs and DNNs with different sizes are provided in Tables 1 and 2. It can be seen from Tables 1 and 2 that the MANN has significant advantages over the DNNs when the structure size is small. For example, with depth 2 and dim 16, the minimal MSE loss of the validation dataset of MANN is \(1.0263\times 10^{-7}\) while that of DNN is \(7.7148\times 10^{-5}\). The minimal MSE loss of the validation dataset of the MANN is less than the order of \(10^{-7}\), while that of the DNN is larger than \(10^{-6}\) in most cases. In the following simulation, the MANN with the smallest MSE loss (shown in bold in Table 1) is adopted as the neural network to predict the \(\mathbf{e}\). Figure 5: Time histories of the MSE losses of the MANN and DNN. ending epochs, the total flight time is about 911.7 days. The parameters for the simulations are listed in Table 3. We assume that a total of ten measurements can be obtained during the transfer process, i.e., the measurement interval is about 91 days (no measurement is available at the initial epoch). Assume that the position vectors can be measured. In general, the position vectors can be measured by observing planets or beacons. For example, by observing two different planets and obtaining the corresponding line-of-sight vectors, we can compute its own-position vector. Therefore, the measurement model is written as follows:  
\[\mathbf{z}_{k+1}=\mathbf{h}(\mathbf{x}_{k+1})+\mathbf{\varepsilon}_{k+1}=\mathbf{r}_{k+1}+\mathbf{ \varepsilon}_{k+1}. \tag{40}\]  
First, a single simulation was performed, with the initial position errors set to 10,000 km per axis and the initial velocity error set to 0.1 km/s per axis. The standard deviation of the measurements was set as 1000 km per axis. The initial covariance was set as  
\[\mathbf{P}_{0}= \begin{bmatrix}10^{8}I_{3}\text{ km}^{2}&0_{3\times 3}\\ 0_{3\times 3}&10^{-4}I_{3}\text{ km}^{2}/s^{2}\end{bmatrix} .