# Machine Learning-Based Solutions for Handover Decisions in Non-Terrestrial Networks
## 4 Experimental Results
### Performance Result of the Classification Algorithm  
Conditional handover is used to support mobility management in NTNs by allowing the system to make more informed decisions about when and how to perform a handover based on network conditions and the requirements of the mobile device. Since we know where each cluster centroid is, we can determine the distance between every UE and the cell center. The handover decisions are designed as a binary classification problem by assuming that UE close to the cell center (\(C3_{UE}\) and \(C2_{UE}\)) is not ready to handover; as a result, they belong to category 0; those who are close to the cell edge (\(C1_{UE}\)) are ready to handover, and they belong to category 1, as shown in Figure 8. One metric for measuring classification model performance is accuracy. Informally, accuracy is the percentage of predictions that our model correctly predicted. Formally, accuracy has the following definition:  
\[Accuracy=\frac{Number\ of\ correct\ predictions}{Total\ number\ of\ predictions} \tag{10}\]  
Accuracy can also be determined in terms of positives and negatives for binary classification, as seen below:  
\[Accuracy=\frac{TP+TN}{TP+TN+FP+FN} \tag{11}\]  
where TP = True Positives, TN = True Negatives, FP = False Positives, and FN = False Negatives. Building an efficient machine learning model requires evaluating a model, \begin{table}
\begin{tabular}{c c} \hline \hline
**Cell** & **Centroid [Latitude, Longitude]** \\ \hline First Cluster [Green] & 1.22293499, \(-\)114.66066589 \\ Second Cluster [Black] & 0.51339138, \(-\)116.18096421 \\ Third Cluster [Red] & \(-\)0.4651767, \(-\)117.6435267 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Centroid of each cluster. Figure 11: 49,000 UE clustered based on their location: Cell type, latitude, and longitude. and there are various evaluation measures in machine learning. Along with accuracy, the following measures have been used to check our model:  
\[Precision=\frac{TP}{TP+FP} \tag{12}\]  
Precision: The number of correct positive results divided by the number of positive results predicted by the classifier. \[Recall=\frac{TP}{TP+FN} \tag{13}\]  
\[F1\ Score=2(\frac{Precision\ \times\ Recall}{Precision+Recall}) \tag{14}\]  
A measure of the quality of the estimator is the Mean Square Error. It is always a positive number that gets smaller as the error gets closer to zero since it is derived from the square of the Euclidean distance. \[MSE=\frac{1}{N}\sum_{j=1}^{N}(y_{j}-\hat{y_{i}})^{2} \tag{15}\]  
During our experimentation, after collecting the data, we preprocessed them; we used preprocessed datasets to train the model using various machine learning algorithms. The machine learning model is trained offline, and on-device ML can be used to perform inference models directly on a device since our proposed architecture is for a regenerative onboard satellite payload. Once the model training was finished, we used evaluation metrics to test our model. The results presented in Table 3 describe the accuracy of different machine learning classifiers and the artificial neural network (ANN). Here, for the random forest, the value of the n-estimator was 30, and we found out that the max-depth equal to 9 was a good value for this specific generated dataset. Consequently, the random forest has a higher accuracy than the ANN, but the ANN has a lower MSE than the random forest. Most frequently, we use classification accuracy to evaluate the performance of our model, but it is not sufficient. F1-Score, AUC, recall, and precision were used to evaluate the performance of the model. Besides the logistic regression, naves, and the support vector machine, the ANN has a better recall and precision, as shown in Table 4.