# Machine Learning-Based Methods for Enhancement of UAV-NOMA and D2D Cooperative Networks
## 4 ANN-Based Placement Scheme Selection
### ANN Model Selection  
This section presents the evaluation results obtained from the ANNs methods for the testing set. The evaluation of the ANNs methods and, by extension, the choice of the ANN algorithm to solve the PSS classification problem is achieved based on the accuracy, precision, recall, and F1 score performance metrics. Specifically, accuracy, precision, recall, and F1 score are commonly used evaluation metrics for assessing the performance of ML models, particularly in classification tasks. These metrics are calculated based on the number of true positive (TP), true negative (TN), false positive (FP), and false negative (FN) predictions made by the model. Accuracy is the proportion of correct predictions made by the model out of all predictions made. In the context of sum-rate maximization, a high accuracy score would indicate that the ANN can predict the best PSS more often accurately, and it is calculated as follows:  
\[\text{Accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+ \text{FN}} \tag{34}\] \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameters** & **Values** \\ \hline Activation functions & ReLU and SoftMax \\ Training algorithm & Gradient Descent \\ Learning rate & 0.01 \\ Maximum number of epochs to train & 1000 \\ Loss function & cross-entropy \\ Minimum performance gradient & \(10^{-6}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Chosen hyperparameters values for ANNs models. Figure 4: Loss convergence progression versus epochs for the training, validation, and testing phase of all the introduced ANNs. Precision is the proportion of true positive predictions made by the model out of all positive predictions made. For example, in the context of sum-rate maximization, a high precision score would indicate that when the ANN predicts a PSS, it is more likely to be the best prediction that maximizes the system sum rate, and it can be expressed as follows:  
\[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{35}\]  
Recall (also known as sensitivity or true positive rate) is the proportion of true positive predictions made by the model out of all actual positive cases. In term of sum-rate maximization, a high recall score would indicate that the ANN is able to find more of the actual PSS solutions, and it is calculated as follows:  
\[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{36}\]  
F1 score is a harmonic mean of precision and recall. In the context of sum-rate maximization, a high F1 score would indicate that the ANN has a good balance of precision and recall, making fewer false PSS predictions while also identifying most of the relevant cases. It is calculated as:  
\[\text{F1}=2\times\frac{\text{Precision}\times\text{Recall}}{\text{Precision}+ \text{Recall}} \tag{37}\]  
Figures 5 and 6 present the evaluation results obtained from the ANN methods for the testing set. Accuracy, precision, recall, and the F1 score are used to evaluate the ANN's approaches. More specifically, the accuracy of each ANN model is depicted in Figure 5, while Figure 6 illustrates the mean precision, recall, and F1-score obtained from each ANN method. The classification accuracy in Figure 5 reveals that the best prediction is achieved through the ANN with two hidden layers ANN\({}_{8-32-31-2}\). Comparing the performance of the different ANN layouts, the prediction accuracy decreases until the neural network reaches two hidden layers in depth. Then, by extending the depth of the ANNs to more than two hidden layers, the accuracy is diminished. Specifically, the prediction accuracy increases from 92.5% for a single hidden layer (ANN\({}_{8-63-2}\)) to 95.32% for a two-layered (ANN\({}_{8-32-31-2}\)) and then decreases to 92.3% and 92.7% for a three (ANN\({}_{8-21-21-21-2}\)) and four-layered (ANN\({}_{8-16-16-16-15-2}\)) structure, respectively. As can be observed in Figure 6, the assessed ANN models exhibit exceptional performance with an F1-score greater than 91%, maintaining an average accuracy and average recall greater than 91%.