# Survey on Q-Learning-Based Position-Aware Routing Protocols in Flying Ad Hoc Networks
## 4 Q-Learning-Based Position-Aware Routing Protocols in FANETs
### Q-Learning-Based Geographic Routing Protocol (QGEO)  
[PERSON] et al. [39] proposed the Q-learning-based geographic routing protocol (QGEO) that outperforms position-based routing protocols in terms of PDR and network overhead in high-mobility scenarios of UAVs. Here, each UAV makes a routing decision in a distributed manner by utilizing the one-hop neighbor node position with the aid of GPS and RL. Each node broadcasts hello packets at a fixed hello-interval that includes mobility information (position, velocity, and direction), current Q-value, link condition (link capacity, interference, and delay), and location error. Unlike the position-based routing protocol, instead of only seeking progress in transmission distance toward the destination, it provides a concept for packet travel speed (PTS) to select the next relay node. The PTS jointly considers the distance progress toward the destination and the channel conditions. The channel condition is computed as the packet travel time (PTT), which consists of the MAC delay, transmission delay, link error, and localization error. The relay node that provides a higher PTS receives more rewards and is selected as the next forwarding node.  
PTT (\(PTT_{uij}\)) and PTS (\(PTS_{uij}\)) between source UAV (\(u_{i}\)) and relay UAV (\(u_{j}\)) are computed as follows:  
\[PTT_{uij}=\frac{\left(t_{uij}^{mac}+t_{uij}^{que}+t_{uij}^{tx}+t_{uij}^{p}\right) }{\left(1-E_{uij}\right)\times\left(1-E_{loc}\right)} \tag{2}\]  
and  
\[PTS_{uij}=\frac{d(u_{i},BS)-d\left(u_{j},BS\right)}{PTT_{uij}} \tag{3}\]  
where \(t_{uij}^{mac},\ t_{uij}^{que},t_{uij}^{tx}\), and \(t_{uij}^{p}\) represent the MAC, queuing, transmission, and propagation delays, respectively, for link \(uij\). \(E_{uij}\) and \(E_{loc}\) represent the link and localization errors, respectively, for link \(uij\). \(d(u_{i},BS)\) and \(d(u_{j},BS)\) represent the distance between the source UAV and destination BS and the distance between the selected relay UAV and destination BS, respectively. The parameter \(PTS_{uij}>0\) means that \(u_{j}\) indicates the progress of distance toward the destination BS, and a higher value of \(PTS_{uij}\) means that the link condition of \(u_{j}\) is good and offers a very small delay.  
If a local minimum incident occurs, the corresponding relay node receives a minimum reward as a penalty.  
_Advantages_: QGEO overcomes the limitations of position-based routing protocols by considering link conditions and location errors during the next-hop selection process via QL. QGEO can avoid the local minimum by assigning a minimum reward.  
_Limitations_: QGEO does not consider the mobility control of the UAV swarm and considers a fixed hello interval (4 Hz) during the simulation. The learning rate and discount factor are not adjusted adaptively with the mobility of the UAV swarm. Additionally, the RE of the UAV node is not considered while making the routing decision. QGEO does not consider the balance between exploration and exploitation strategies to explore better relay UAVs.