## 4 The TRAJ-Q-GPSR Algorithm  
### Trajectory-Informed UAV Routing  
To address the inherent limitation of traditional GPSR protocols, where routing decisions rely solely on the current positions of nodes for greedy forwarding, this study introduces the integration of trajectory knowledge and a two-hop neighbor table. These enhancements enable UAV nodes to obtain a more comprehensive understanding of both spatial and temporal network dynamics. As a result, the likelihood of encountering routing voids is reduced, and both routing stability and path selection are significantly improved. The proposed approach first describes the acquisition of trajectory knowledge, followed by its mathematical modeling. Based on this foundation, a method is developed to construct a two-hop neighbor table that leverages the acquired trajectory information. We assume that the mission planner computes the trajectory for each UAV within a future time window \([t_{0},t_{f}]\). The trajectory \(\mathcal{T}_{i}\) of UAV \(U_{i}\) at time \(t\) is defined as a time-parameterized geometric state representation:  
\[\mathcal{T}_{i}(t)=(p_{i}(t),v_{i}(t),a_{i}(t)),\quad t\in[t_{0},t_{f}] \tag{16}\]  
where \(p_{i}(t),v_{i}(t)\) and \(a_{i}(t)\) denote the position, velocity, and acceleration of UAV \(U_{i}\) at time \(t\), respectively. To facilitate computation and storage, the trajectory is discretized into a finite time series:  
\[\mathcal{T}_{i}=\{(p_{i}(t_{k}),v_{i}(t_{k}),a_{i}(t_{k}))\ |\ t_{k}=t_{0}+k \cdot\Delta t,\ k=0,1,\ldots,K\} \tag{17}\]  
where \(\Delta t\) represents the sampling interval, and \(K\) is the number of planned steps, determined by the planning algorithm's prediction range. For the entire UAV swarm, the set of all predicted trajectories within the given time window is defined as  
\[\mathcal{T}=\bigcup_{i=1}^{N}\mathcal{T}_{i}. \tag{18}\]  
This dataset encapsulates the motion information of all UAV nodes within the prediction horizon, providing valuable spatiotemporal knowledge for routing decisions. In GPSR protocol, routing nodes can only sense their immediate neighbors at the current moment. Given the rapid changes in network topology, this often results in routing failures or frequent transitions into the perimeter forwarding mode. However, by sharing trajectory information, UAVs can proactively anticipate their future neighbors. The neighbor set \(\mathcal{N}_{i}(t_{k})\) of UAV \(U_{i}\) at a future time \(t_{k}\) is defined as the set of all nodes within its communication range \(R\):  
\[\mathcal{N}_{i}(t_{k})=\{U_{j}\ |\ \|p_{i}(t_{k})-p_{j}(t_{k})\|\leq R\}. \tag{19}\]  
By leveraging trajectory sharing, each UAV can obtain information about its neighbors' neighbors, thereby constructing a two-hop neighbor table:  
\[\mathcal{N}_{i}^{(2)}(t_{k})=\bigcup_{U_{j}\in\mathcal{N}_{i}(t_{k})}\mathcal{ N}_{j}(t_{k}). \tag{20}\]  
Here, \(\mathcal{N}_{i}^{(2)}(t_{k})\) represents the two-hop neighbor set of UAV \(U_{i}\) at a future time \(t_{k}\). Constructing a two-hop neighbor table at future time instances enables UAVs to make more informed routing decisions. This approach not only helps avoid local optima and reduces the likelihood of routing voids, but also supports the optimization of queuemanagement strategies. By incorporating estimated packet transmission times, the method ultimately contributes to more robust and efficient routing performance. The Traj-Q-GPSR theoretical framework of this paper is shown in Figure 3, which integrates trajectory knowledge with reinforcement learning to optimize routing decisions. The left part constructs a two-hop neighbor table using trajectory information. It evaluates node availability at the estimated transmission time based on a delay model, thereby determining whether to enqueue or drop the packet. The right part incorporates reinforcement learning to dynamically adjust routing weights. These adjustments are based on state features such as distance, queue length, and residual energy.