## 3 Intelligent Integrated Navigation Q-Learning FEKF Algorithm
### Design of the \(Q\)-Learning-Based FKF for Intelligent Integrated Navigation
It can be seen from the figure that the QLFEKF algorithm shows high accuracy position estimation performance with an error accuracy of \(10^{2}\) m. This shows that QLFEKF can adaptively choose the appropriate process states and observation (the X-ray pulsar detector measurement errors and spectrometer measurement errors) noise error covariance matrix by using the cumulative reward mechanism of the \(Q\)-learning through its interaction with the flight environment. The stability of the position for the three integrated navigation algorithms has corresponding fluctuations, which are caused by eliminating the accumulated position errors caused by Doppler velocity measurements. However, compared with the position state estimation of the STD/XP-EKF navigation algorithm and STD/XP-FEKF navigation algorithm, the stability of the STD/XP-QLFEKF position state estimation is better. This result shows \begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter Name** & & **Value** \\ \hline \multirow{2}{*}{Learning rate, discount factor, and greedy values} & \(\alpha\) & 0.1 \\  & \(\gamma\) & 0.9 \\  & \(\varepsilon\) & 0.1 \\ \hline \multirow{2}{*}{The upper edge and lower edge bounds of \(\hat{\textbf{Q}}_{rk}^{(i)},\hat{\textbf{Q}}_{ik}^{(i)}\), \(\hat{\textbf{R}}_{pk}^{(i)}\), \(\hat{\textbf{R}}_{pk}^{(j)}\), and \(\hat{\textbf{R}}_{dk}^{(j)}\)} & \(\begin{bmatrix}q_{kk}&\overline{q}_{nk}\\ \overline{q}_{kk}&\overline{q}_{kk}\\ \overline{r}_{pk}&\overline{r}_{pk}\\ \overline{r}_{dk}&\overline{r}_{dk}\\ \end{bmatrix}\)} & \(\begin{bmatrix}1/10^{2}&10^{2}\\ \overline{1}/101^{2}\\ \overline{1}/102&1/101\\ \end{bmatrix}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The design parameters of the QLFEKF algorithm. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Initial Condition** & **Parameters Name** & **Parameters Value** \\ \hline \multirow{3}{*}{XNAV Measurement} & Number of detectors & 3 \\  & Effective area of the detectors & 1 m\({}^{2}\) \\  & Measurement noise variance matrix & \(\hat{R}_{p0}=diag\big{(}[p_{1}^{2},p_{2}^{2},p_{3}^{2}]\big{)}\,\text{km}^{2}\) \\ \hline \multirow{3}{*}{
\begin{tabular}{c} Solar/target planetary Doppler measurement \\ \end{tabular} } & Number of the spectrometers & 2 \\  & Accuracy of the spectrometers & \(d_{1}=d_{2}=10^{-6}\) km/s \\ \cline{1-1}  & Velocity measurement navigation & \(\hat{R}_{d0}=diag\big{(}[d_{1}^{2},d_{2}^{2}]\big{)}\,(\text{km}/\text{s})^{2}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The corresponding parameters of the measurement models. that the \(Q\)-learning algorithm has a good convergence effect in the process of parameter tuning, which ensures that the probe can operate continuously and stably in the cruise phase with high precision. From Figures 8 and 9, it is seen that the velocity estimation precision of the STD/XP-QLFEKF algorithm is about \(10^{-1}\) m/s, which is much higher than that of the STD/XP-EKF algorithm and the STD/XP-FEKF algorithm. It can be indicated that the state error estimation precision of the FEKF is better than that of the EKF, which reflects the superiority of the federated filter in the fusion of various measurement information. The FEKF combined with the \(Q\)-learning algorithm can distribute the position, velocity, and state transition information from different measurement models according to the weighted proportions, and it ensures that the information fusion of various navigation measurement models can obtain high-accuracy position state and velocity state error estimations based on the time update and measurement update. In addition, we could see that the precision convergence of the intelligent navigation system based on X-ray pulsar/solar and target planetary Doppler velocity measurements is very stable. Figure 6: Comparison of the position estimate RMSEs between STD/XP-QLFEKF and other integrated navigation algorithms. Figure 7: Comparison of the position estimate RMSEs for three axes between STD/XP-QLFEKF and other integrated navigation algorithms.