## 3 Intelligent Integrated Navigation Q-Learning FEKF Algorithm
### Design of the \(Q\)-Learning-Based FKF for Intelligent Integrated Navigation
Based on the simulated analysis mentioned above, it verifies that the proposed STD/XP-QLFEKF intelligent integrated navigation is very appropriate for the considered intelligent integrated navigation for the cruise phase of the SSBE. Table 5 below shows the comparison of the position state and velocity state estimation RMSE errors (\(3\sigma\)) for the above three integrated navigation algorithms. From Table 5 above, it can be seen that the position estimation RMSE error (\(3\sigma\)) accuracy is 443.560 m and the velocity state RMSE (\(3\sigma\)) precision is 0.630 m/s for the STD/XP-EKF. The PVES RMSE (\(3\sigma\)) precision of the STD/XP-FEKF are 351.803 m and 0.386 m/s, respectively. Based on the proposed STD/XP-QLFEKF algorithm in the paper,

Figure 8: Comparison of the velocity estimate RMSEs between STD/XP-QLFEKF and other integrated navigation algorithms. Figure 9: Comparison of the velocity estimate RMSEs based on three axes between STD/XP-QLFEKF and other integrated navigation algorithms. the PVSE RMSE error (3\(\sigma\)) accuracy of the cruise phase for the probe is 152.101 m and 0.243 m/s, respectively. By comparing the accuracy of the PVSE errors (3\(\sigma\)) between the STD/XP-QLFEKF and STD/XP-FEKF algorithms, the accuracy of the STD/XP-QLFEKF algorithm has been greatly improved. According to statistical analysis, the PVSE precision improvement capability of the STD/XP-QLFEKF algorithm is 55.84% and 37.04%, respectively. This is because the \(Q\)-learning algorithm can dynamically adjust the process and observation noise error covariance by periodically iterating according to the real-time in-orbit operating state after receiving different measurement state information. The cumulative reward is obtained through continuous iterations in each iteration cycle (\(ItCy\)). The \(Q\)-value of the corresponding state (\(\hat{Q}_{rk}^{(i)}\), \(\hat{Q}_{rk}^{(i)}\), \(R_{pk}^{(j)}\), and \(R_{dk}^{(j)}\)) position in the \(Q\)-table is calculated according to the cumulative reward value, and the appropriate information parameters are selected to the maximum extent through continuous trial-and-error to guarantee the performance superiority of the STD/XP-QLFEKF algorithm. Next, the range of the related parameters (including \(\alpha\), \(\gamma\), and \(\varepsilon\)) for the reinforcement Q-learning is set to evaluate the effects on the related learning parameters on the precision of the PVES error of the STD/XP-QLFEKF. The following sets the range of the \(Q\)-learning parameters involved above when the three learning parameters are changed in the range of \(\alpha\in[0.05\;0.14]\), \(\gamma\in[0.90\;0.99]\), and \(\varepsilon\in[0.05\;0.14]\), and the RMSE errors (3\(\sigma\)) for position state and velocity state estimation of the STD/XP-QLFEKF algorithm are shown in Figures 10-12, respectively. Figure 11: The PVSE RMSE errors (3\(\sigma\)) of the cruise phase as a function of the discount factor. Figure 10: The PVSE RMSE errors (3\(\sigma\)) of the cruise phase as a function of the learning rate. From Figures 10-12, it can be seen that the change of the \(Q\)-learning design parameters has relatively little influence on the PVES error precision for the STD/XP-QLFEKF algorithm. The simulation results show that the STD/XP-QLFEKF is insensitive to the changes in the navigation PVES errors within the range of the above \(\alpha\), \(\gamma\), and \(\varepsilon\), and the small changes of navigation accuracy estimation errors caused by the changes of the \(Q\)-learning parameters can be ignored in the process of intelligent navigation algorithm design. In addition, in the process of the STD/XP-QLFEKF algorithm reward mechanism design, we take the iteration period as a function of reward accumulation, and the setting of the iteration period has a certain influence on the calculation results of the reward accumulation. The following is the simulation and analysis of the PVES error corresponding different iteration cycles of the \(Q\)-learning of the QLFEKF algorithm.