# Non-Terrestrial Networks with UAVs: A Projection on Flying Ad-Hoc Networks
## 6 Applications of Machine Learning in FANET
### Resource Allocation  
In order to achieve seamless service while maintaining costs and energy consumption constraints, FANETs require real-time, autonomous, and dynamic resource allocation techniques. This is even more crucial in FANETs in comparison to static terrestrial links since each UAV requires the selection of a flight trajectory, which ensures that the UAV is maintaining its coverage, energy consumption, and avoiding obstacles (Obstacles could be static, such as high-rise buildings in urban environments, or even dynamic, such as other UAVs or departing/landing airplanes that are at the same altitude). Other issues include the allocation of UAVs to certain geographical regions or the redirection of some UAVs to cover for others that might require charging. Traditionally, logistic regression and support vector machine (SVM) have been commonly applied to UAV resource allocation problems [149]. In recent years, neural network-based and deep RL-assisted methods have become more popular to cope with the complex and dynamic nature of FANETs [140]. In [150], a ML algorithm was proposed focusing on the integration of UAVs into terrestrial networks to find efficient resource allocation along with the optimized 3D locations of UAVs to maximize the sum logarithmic rate of the users. In [151], the authors suggested a model for optimizing the power transmission, trajectory design, and user association for a multi-UAV setting. In this setting, a central UAVs or a base station chooses an action for every time slot following the Q-learning method. In order to attain the required, the battery state, the state of the channel, and maximum transmission state were considered. Further, the researchers suggested that simple use of Q-learning is not advisable due to the high dimensionality problem that it presents. To further elaborate, due to the dynamic nature of UAVs, it is possible that thousands of states exist. Then if the values are stored in the Q-matrix table, there would be very large and insufficient samples to traverse every state of the UAV, often leading to the algorithm failing. As a result, the authors suggested combining Q-learning and Deep Neural Network (DNN). Instead of calculating the Q-value for each state, Deep Q-Network (DQN) can be used to estimate the value function for Q-learning. In [152], resource management was studied in UAV-enabled long-term evolution (LTE) settings over licensed and unlicensed bands. The major aim was to optimize users' QoS while fulfilling each user's latency requirement. Instead of using conventional echo state networks (ESNs) for this, the authors suggested using ESN with leaky integrator neurons. This was driven by the algorithm's capability to dynamically update the ESN's status for users and achieve the optimal user QoS. Through simulation, it was shown that this method outperformed Q-learning with LTE-U and Q-learning with LTE in terms of performance. Due to UAV privacy concerns and their computational limitations, distributed computing has recently gained attention. One such method for networked computing is federated learning (FL), which was developed by Google [153] in 2016. In FL, users do not exchange data, and models are locally trained. The base station conducts an aggregation function on the model parameters and generates a global model, which is then delivered back to the users, using the local models as input. In [154], the authors presented an asynchronous FL framework for resource management. Furthermore, an asynchronous advantage actor-critic (A3C) algorithm was implemented for device selection, UAV placement, and efficient resource management. The authors emphasized the usage of deep RL algorithms, such as DQN, which utilize experience replay for effective learning. However, leveraging experience replay necessitates having enough memory and processing capability to ensure more accurate learning.