## 3 Intelligent Integrated Navigation Q-Learning FEKF Algorithm
### Design of the \(Q\)-Learning-Based FKF for Intelligent Integrated Navigation
The influence of the different iteration cycles of reinforcement \(Q\)-learning on the accuracy of PVES errors in the probe cruise phase is shown in Figure 13.

The simulation results show that when the iteration cycle of the \(Q\)-learning in STD/XP-QLFEKF algorithm is 7200 s, the position state and velocity state estimation precision for the intelligent integrated navigation is higher. Therefore, the cumul

Figure 12: The PVSE RMSE errors (\(3\sigma\)) for the cruise phase as s function of the action selection probability. Figure 13: The influence of different iteration cycles of the reinforcement \(Q\)-learning on the precision of the PVSE errors in the probe’s cruise phase. of the \(Q\)-learning algorithm is carried out according to the iteration cycle (_ItCy_ = 2.0 h) in this research to ensure the high precision of the state error estimation of the intelligent integrated navigation system for the cruise phase of the SSBE. ## 5 Conclusions

Intelligent, high-accuracy integrated navigation for the cruise phase of the SSBE is deeply studied in this paper. The single navigation mode could not meet the high-precision and high-reliability requirements of the navigation system for a probe operating in long-endurance, highly dynamic, and extreme environments. In order to improve the navigation accuracy of the PVSE for the cruise phase of the probe, the QLFEKF based on the X-ray pulsar/solar and target planetary Doppler velocity measurement intelligent integrated navigation algorithm is proposed. The integrated navigation algorithm can compensate for the accumulated position estimation error of Doppler velocity measurements. The navigation system adopts the federation filter composed of the sub-filter and main filter to realize the information fusion of the various navigation modes and ensure the estimation performance of the intelligent integrated navigation system. At the same time, the cumulative reward mechanism of the reinforcement \(Q\)-learning algorithm is designed according to the sub-filter characteristics of the federated filter, and the iterative period of the \(Q\)-learning algorithm is used as an index to judge the cumulative reward. The process and measurement noise covariance matrices are carried out by tuning the parameters by traversing the \(Q\)-table. The simulation and analysis show that compare with the STD/XP-EKF and the STD/XP-FEKF, the STD/XP-QLFEKF algorithm has higher navigation accuracy, and its PVSE RMSE error (3\(\sigma\)) accuracy can reach 152.101 m and 0.243 m/s, respectively. The different values of the three learning parameters (\(\alpha\), \(\gamma\), and \(\varepsilon\)) for the QLFEKF algorithm are insensitive to the PVSE errors, and slight changes in the estimation errors of the navigation accuracy caused by changes in the \(Q\)-learning parameters can be ignored. Of course, it is necessary to consider the iterative period parameters related to the cumulative rewards of \(Q\)-learning in the QLFEKF algorithm. As a function of the reward accumulation, the iterative period has a certain influence on reward accumulation and thus affects the PVSE accuracy of the navigation system. It is important to choose the appropriate cumulative reward iteration cycle. Therefore, the QLEKF based on the intelligent X-ray pulsar/solar and target planetary Doppler velocity measurement integrated navigation system has high accuracy, which can fully fulfill the high accuracy demands for the cruise phase of the SSBE probe. In future work, the combination of reinforcement learning and traditional navigation methods will be further studied, and deep neural networks will be added to reinforcement learning to provide interpretability for intelligent integrated navigation methods and improve the flexibility of navigation system parameter setting. The fusion of multisource intelligent information of the integrated navigation measurement models would be simultaneously carried out to further improve the information fusion accuracy and efficiency of information obtained from various navigation measurements.