# Fault Diagnosis Across Aircraft Systems Using Image Recognition and Transfer Learning
## 4 Results and Analysis
### Baseline Result from Non-TL Methods  
To understand how the conventional ML algorithm performs for the small sample problem in both ECS and APU datasets, a variety of common ML classifiers were selected, and a 1D-CNN model was constructed. The classifiers used included k-nearest neighbours (kNN), support vector machine (SVM), decision tree (DT), and random forest (RF) [27], which represent the most commonly used classifiers in existing IFD applications. The key parameters of the ML classifiers used in this work are listed in Table 2. The architecture of the 1D-CNN is also shown in Table 2. This was constructed using the Keras library [28]; the 1D-CNN model used contained three convolutional layers: a maximum pooling layer, a flattening layer and a dense layer. This 1D-CNN architecture was determined by a trail and error approach on the datasets used to generate the best accuracy and fastest convergence, and the parameters used in each layer are detailed in Table 2.  
The results are shown in Figure 11 and summarised in Table 3. Figure 11 plots the predictive accuracy for the entire ECS and APU datasets against the number of cases in the training set ranging from 1 to 50. From Figure 11, the ECS results from most methods rarely  
Figure 10: Image preparation process.  
show any meaningful change when 30 or more cases are in the training set, so analysing the scenario where 1 to 30 cases are in the training set would make a comparison between the methods more obvious. However, the results from most methods when dealing with the APU dataset showed a continuing trend when cases in the training set increased from 1 to 50, so the entire range will be considered.  
Hence, the average predictive accuracy for the testing set was calculated for each method when 1 to 30 cases were used in the ECS training set, and 1 to 50 cases were used in the APU training set, which is shown in Table 3. Although, in reality, a small sample problem would be one single scenario with a certain small number of cases in the training set (e.g., 20 cases in the ECS training set when designing the ECS diagnosis algorithm), taking an average in the way described above is useful to summarise a range of possible small sample problems and compare different methods. Judging by this metric, 1D-CNN outperformed all other ML methods. For the ECS case, the 1D-CNN average accuracy was 76.48%, which is 10.52% higher than the second-best method, the RF classifier. For the APU case, the average accuracy of the 1D-CNN was 56.95%, which is 7.67% higher than the second-best method, the RF classifier. In addition, it can be observed from Figure 11 that the 1D-CNN result in each of the ECS and APU scenarios is consistently the best one among all the methods used. Therefore, the result from the 1D-CNN will be used as the baseline non-TL result, representing the best performance of conventional ML methods.  
Figure 11: Result from conventional ML methods without TL for (**a**) the ECS case; (**b**) the APU case.  
\begin{table}
\begin{tabular}{c c c} \hline \hline \multirow{2}{*}{**Method** **(Non-TL)**} & \multicolumn{2}{c}{**Average Predictive Accuracy (\%) for Testing Set**} \\ \cline{2-3}  & **1–30 Cases in ECS** & **1–50 Cases in APU** \\  & **Training Set** & **Training Set** \\ \hline
1D-CNN & 76.48 & 56.92 \\ \hline kNN & 54.36 & 44.81 \\ \hline SVM & 63.16 & 48.96 \\ \hline DT & 64.21 & 48.24 \\ \hline RF & 65.96 & 49.25 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Summary of average accuracy from non-TL methods.