# An Innovative Applied Control System of Helicopter Turboshaft Engines Based on Neuro-Fuzzy Networks
## 3 Results
### Neural Network Model Quality Evaluation  
Since the basis for the developed helicopter TE FTCS (Figure 4) is a neuro-fuzzy network (Figure 3), evaluating neuro-fuzzy network quality is reasonable. It allows for the determination of how effectively the neuro-fuzzy network addresses control tasks and makes decisions under uncertainty and variability. The evaluation includes analyzing its generalization ability, prediction accuracy, and resilience to external disturbances. Performance assessment for the developed neuro-fuzzy network (Figure 5) uses vital quality metrics such as accuracy, loss, precision, recall, F1-metric, and AUC-ROC [72; 73; 74; 75; 76; 77; 78]. These metrics comprehensively evaluate the neuro-fuzzy network's effectiveness in various aspects of its operation within the controlling helicopter TE context. Accuracy helps identify the proportion of correctly classified engine operating modes relative to the total number, providing insights into the model's overall performance. Loss measures how well the model aligns with actual data, indicating the need for further optimization in control strategies. Precision and recall offer insights into the accuracy and completeness of identifying critical engine states, which is especially important for preventing malfunctions. The F1-metric,  
Figure 11: A diagram of the cost function change during the researched interval from 0 to 320 s.  
representing the harmonic mean of Precision and Recall, helps balance these two indicators, which are critical in resource-limited conditions and high safety requirements. AUC-ROC provides information on the model's ability to distinguish between normal and emergency engine states at various thresholds, which is essential for assessing the reliability of the control system. This comprehensive metrics analysis allows for the neuro-fuzzy network's current performance evaluation and identifies areas for improvement, ultimately enhancing the developed helicopter TE FTCS's efficiency. These metrics are calculated according to the following expressions [73, 75, 77, 79, 80]:  
\[ \begin{split} Accuracy=\frac{1}{N}\cdot\sum\limits_{i=1}^{N}1(u_{ i}=\hat{u}_{i}),\;Loss=\frac{1}{N}\cdot\sum\limits_{i=1}^{N}{(u_{i}(t)-\hat{u}_{i}(t)) ^{2}},\\ Precision=\frac{TP}{TP+FP},\;Recall=\frac{TP}{TP+FN},\;F1- score=\frac{2\cdot Precision-Recall}{Precision+Recall},\\ AUC-ROC=\int\limits_{0}^{1}TPR\cdot\Big{(}FPR^{-1}(t)\Big{)}dt. \end{split}  \tag{44}\]  
In this scenario, \(u_{i}\) denotes the actual label for the \(i\)-th instance, whereas \(\hat{u}_{i}\) signifies the predicted label generated by the model for that particular instance. \(N\) represents the total number of examples in the dataset (applicable for either training or validation), and the indicator function \(\mathbf{1}(u_{i}=\hat{u}_{i})\) yields a value of 1 if the true and predicted labels match and 0 otherwise. In the helicopter TE control context, the metrics \(TP\) (True Positive), \(TN\) (True Negative), \(FP\) (False Positive), \(FN\) (False Negative), \(TPR\) (True Positive Rate), and \(FPR\) (False Positive Rate) play a key role in assessing the monitoring and diagnostic systems' effectiveness [51, 57, 81]. \(TP\) represents instances when the system successfully detects faults, while \(TN\) indicates incorrect identification. \(FP\) refers to erroneous signals about faults, and \(FN\) describes missed critical conditions. \(TPR=\frac{TP}{TP+FN}\), or sensitivity, reflects the proportion of correctly identified faults among all actual cases, which is vital for enhancing safety, while \(FPR=\frac{FP}{FP+TN}\) represents the level of false alarms [82, 83, 84, 85, 86]. Analyzing these metrics allows for an evaluation of system reliability, identifying shortcomings in diagnostic algorithms and implementing necessary adjustments to improve flight safety. Figures 12 and 13 illustrate the diagrams for accuracy and loss metrics.