# Intelligent Multi-Fault Diagnosis for a Simplified Aircraft Fuel System
## 3 Methodology
### Evaluating the Interpretability of the Selected ML Algorithms  
**Overview of the proposed validation process. ** Figure 7 illustrates an innovative structured process proposed to evaluate, quantify, and compare the interpretability of four ML algorithms by validating their explanation outputs against domain knowledge. Interpretability metrics were derived through two distinct methods designed for transparent and black-box algorithms. Domain knowledge, encompassing residuals and key features, serves as a benchmark for assessing the alignment between algorithmic outputs and engineering insights. **Method 1: interpretability assessment for transparent algorithms. ** Method 1 was applied to transparent ML algorithms, such as linear regressions (LRs) and decision trees (DTs), where model components inherently lend themselves to interpretation. Here, a residual table (Table 3) was employed to validate model interpretations. Interpretability stems from analysing feature coefficients for LRs, while it originates from diagnostic rules based on feature splits for DTs. The alignment between residual data and model outputs establishes the interpretability of these transparent algorithms. **Method 2: evaluating interpretability of black-box algorithms using XAI techniques. ** For black-box algorithms, interpretability was assessed using explainable artificial intelligence (XAI) techniques--specifically LIME and SHAP, both local and model-agnostic methods. These techniques facilitate an analysis of key features identified by XAI as significant to model predictions. A key-feature table (Table 4) provides an engineering basis to evaluate XAI outputs, with interpretability measured by the degree of overlap between the XAI-derived sensitive features and those identified as critical in engineering analysis. This approach has proven to be effective in applications such as unmanned aerial vehicle (UAV) elevators [19], gas turbines [39], nuclear power plants [40], cross-building energy systems [41], and rotating machines [42], where XAI techniques interpret the diagnostic logic of complex ML models. **Symptom vector formation. ** The process, shown in the left half of Figure 7, began with constructing symptom vectors based on pressure (p1 to p7) and flow rate (f1 to f3) features within the dataset. These features represent key indicators of system health, with each condition--such as normal operation, pump external leakage, or fuel oil heat exchanger (FOHE) blockage--having its own symptom vector (shown in Figure 8). A matrix was created for each of the five pump speeds, with rows representing classes and columns representing features. The mean values for each class and pump speed were then recorded in the matrix, providing a baseline for subsequent analysis. **Residual calculation and interpretation. ** Residuals from the symptom vectors produced a 31 \(\times\) 10 matrix for each pump speed, highlighting deviations between each faulty class and the normal condition. For example, Table 3 records the residuals for nozzle blockages (Class 2). Engineering knowledge interprets the resulting backpressure as an increase in downstream pressure (p3 to p7) and a reduction in flow rate (f1 to f3). Such correlations between engineering principles and residual data underpin the interpretability of diagnostic models. **Key feature identification for fault diagnosis. ** Analysing the residuals and leveraging engineering knowledge can identify the key features most indicative of each faulty class. These features show the greatest deviation from the normal condition and are instrumental in diagnosing specific faulty classes. Table 4 shows the key features for several faulty classes, highlighting the four features most significantly deviating from the normal condition. These  
conditions, including multiple faults. This dataset included 19,505 individual datasets. The objective of multiple-fault diagnosis is to develop an algorithm that accurately identifies all 32 classes.