## 5 Experimental Results
### Training Setup and Results  
In this study, we employed a high-performance desktop computing system to train our agent after collecting a substantial volume of epoch signals. The experimental setup utilized an AMD Ryzen 9 9950X 16-core processor (Advanced Micro Devices, Inc.; Sunnyvale, CA, USA) with 48 GB RAM and an NVIDIA GeForce RTX 5070 Ti GPU (NVIDIA Corporation; Santa Clara, CA, USA), providing the necessary computational power for efficient DRL implementation. This hardware configuration enabled effective processing of the extensive navigation epoch data while supporting the computationally intensive training of our DDQN-based positioning system. The complete specifications of training hyperparameters, including batch sizes, learning rates, and network architectures, are systematically documented in Table 2 to ensure experimental reproducibility and facilitate comparative research in the field of navigation-oriented DRL. During the training of the agent, we recorded the average cumulative reward obtained during its interaction with the environment (as shown in Figure 11) to evaluate the learning performance. The average reward at each timestep is obtained by averaging the immediate rewards that the agent receives through environmental interactions at that specific timestep across multiple episodes. The cumulative average reward at the current timestep is then calculated by summing these average rewards from the initial timestep up to the present timestep. In the early training phase (from the initial timestep to the 100 th timestep), the agent was in the exploration stage and predominantly made random decisions by outputting random weights. However, despite the large initial position estimation errors during the recursive initialization phase, the incorporation of epochs still led to a reduction in positioning errors, resulting in a rapid increase in the agent's average cumulative reward by nearly 1500. Subsequently, as the agent gradually shifted toward exploitation mode and began making reasonable decisions based on acquired knowledge, the average cumulative reward slowly increased from approximately 900 to nearly 1400. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter** & **Value** & **Description** \\ \hline \(M_{1}\) & 200 & The number of epochs used for initial positioning. \\ \(M_{2}\) & 600 & The number of epochs used for agent decision-making. \\ \(H_{1}\) & 256 & The number of neurons in the first hidden layer. \\ \(H_{2}\) & 256 & The number of neurons in the second hidden layer. \\ \(k_{1}\) & 5 & Reward coefficient associated with the absolute change in positioning error. \\ \(k_{2}\) & 1000 & Reward coefficient associated with the percentage change in positioning error. \\ \(\sigma\) & 0.01 & Minimum offset term. \\ \(\gamma\) & 0.8 & Discount factor, which quantifies the importance of future rewards. \\ \(N_{D}\) & 10,000 & Experience replay buffer capacity. \\ \(N_{B}\) & 64 & Number of samples per mini-batch. \\ \(\tau\) & 0.01 & Target network update coefficient. \\ \(\epsilon_{\min}\) & 0.01 & Minimum exploration rate. \\ \(\lambda\) & 0.99995 & The decay rate of the exploration rate. \\ \(\alpha\) & 0.0005 & The learning rate of the Adam optimizer. \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Environment parameters and hyper-parameters for training. Furthermore, we introduced both \"default\" and \"random\" strategies to benchmark the performance of the agent during the training phase (as illustrated in Figure 12). The \"default\" strategy assigns a fixed weight of 1 to all epochs (i.e., standard RLS), while the \"random\" strategy uniformly samples a weight from the discrete set \(\{0.1,0.2,\ldots,2.0\}\) at each iteration. Observations revealed that the 3D positioning errors of all three approaches exhibited a consistent downward trend throughout the training process, which aligns with the fundamental principles of epoch-based positioning. Figure 11: Agent’s per-timestep average cumulative reward during training.