# Intelligent Multi-Fault Diagnosis for a Simplified Aircraft Fuel System
## 5 Discussion  
This work addresses a more complex multi-fault diagnosis scenario compared to previous studies. It thoroughly investigates the capabilities of traditional machine learning methods, including the shallow neural network, in fault diagnosis. Additionally, this study uniquely examines the influence of algorithmic structural complexity on fault diagnosis outcomes. A more balanced and fair comparison was achieved by re-evaluating the performance of four algorithms under similar structural complexities. In their optimal configurations, the ANN demonstrated the best predictive performance, with an average accuracy of 90% on the validation dataset and strong F-1 scores across all 32 classes. The SVM achieved an accuracy of 84%, with the F-1 scores varying by class and reaching 100% accuracy in certain cases. Among the transparent algorithms, the DT had an overall accuracy of 83.8%, lower than that of both the SVM and the ANN, while the LR, as the simplest model, yielded the lowest accuracy at 79%. Predictive performance decreased as model complexity was reduced, yet the ANN maintained the highest accuracy (87.9%). Additionally, the opaque algorithms consistently outperformed the transparent algorithms in diagnostic accuracy. For the failure modes considered here, leaks in the AFS presented a greater challenge for ML algorithms than blockages. This difficulty is reflected in the confusion matrix, where a notable number of missed detections and false positives are associated with leaks. A possible reason for this discrepancy is that blockages produce more pronounced changes in pressure and flow rate than leaks, making them easier for ML algorithms to identify from the data. While black-box algorithms such as SVMs and ANNs exhibit strong diagnostic capabilities, they present interpretability challenges for domain users. To address this, this study applies XAI techniques to explain black-box models and leverages engineering insights into the fuel system to validate these explanations. The consistency between model outputs and engineering understanding serves as a measure of interpretability, where a higher degree of overlap indicates better interpretability. This approach is necessary because LIME and SHAP results are influenced by the scope of the input data, whereas engineering understanding--rooted in human experience and expertise--extends beyond the dataset. Despite requiring more computational resources and time, SHAP provided interpretability levels similar to those of LIME. Specific to each ML model, the ANN achieved better interpretability than the SVM (61% vs. 52%). However, as shown in Figure 15, consistency between engineering judgment and feature relevance from both LIME and SHAP varied widely, with differences of up to 50% across classes. For transparent models, the logistic regression (LR) parameters in Table 8 differ substantially from engineering insights across many classes. The LR's parameters represent global decision-making across the training dataset which contrasts with the local explanations provided by LIME and SHAP. While LRs offer intuitive global interpretability, their simplicity limits their interpretability (only 58%), restricting their ability to capture complex mappings fully. In contrast, the DT achieved higher interpretability (82%) and displayed greater consistency across fault classes, especially at high pump speeds. DTs' interpretability advantage stems from their rule-based approach, which divides the feature space into distinct regions associated with different classes. Class data are dispersed at high pump speeds, reducing inter-class ambiguity and simplifying classification and diagnostic tasks. These findings suggest a trade-off in performance outcomes depending on the emphasis on interpretability versus accuracy. The DT offered relatively consistent interpretability across fault classes but achieved moderate accuracy, whereas the ANN delivered superior diagnostic accuracy at the expense of interpretability.