# Acoustic-Based Engine Fault Diagnosis Using WPT, PCA and Bayesian Optimization
## 3 Methodology
### Classification Algorithm Selection  
Classification algorithms are widely used in condition monitoring to differentiate between healthy and faulty signals. Here, six standard classifier algorithms were considered: decision tree, Naive Bayes, k-Nearest Neighbors (kNN), Support Vector Machine (SVM), Discriminant analysis and Ensemble classification. These classifiers are explained briefly as follow; readers can be directed to the references for further details. * Decision tree performs classification by creating a tree based on attributes of data. It uses hierarchical structures to find patterns in data for the purpose of constructing decision-making rules, to estimate the relationship between independent and dependent variables [38]. The hyperparameters for optimization here are minimum leaf size, maximum number of split and the split criteria, which further includes the Gini's diversity index, the twoing rule or the maximum deviance reduction. * Naive Bayes is a type of supervised learning algorithm based on the Bayes theorem, which classifies new data using the conditional probability to predict the outcome of an occurrence [39]. The hyperparameters used for optimization here are the distribution type, the window width and the kernel type. The distributions used include Gaussian, multivariate multinomial, multinomial and kernel. The kernel smoothing window width is given by \[f(x)=\frac{\sum_{j=1}^{n}K\Big{[}\frac{x-X_{j}}{h}\Big{]}}{nh}\] where \(K\) is the kernel function. * kNN algorithm is a supervised learning method which uses non-parametric method for classification. It classifies new data by calculating the distance of this data instance with the existing data instances, and then it selects the closest \(k\) instances for classification [40]. The hyperparameters used for optimizing the kNN include the distance measure, the number of neighbors and the distance weights. Figure 5: Feature space of: (**a**) an existing feature set (Mean of Coefficient 1 vs. Mean of Coefficient 2); and (**b**) the first two principle components (PC1 vs. PC2). SVM is a type of supervised learning algorithm, which achieves classification by creating a hyperplane with the maximum margin that separates two classes [41]. The hyperparameters used for optimizing the SVM are the Box constraint, the kernel function, the kernel scale and the polynomial order. Box constraint controls the penalty imposed on observations that lie outside the margin and helps prevent overfitting. The kernel functions used for optimization include Gaussian, linear and polynomial. * Discriminant analysis is an effective subspace technique which also uses Bayes theorem similar to Naive Bayes. The Bayesian rule divides the data space into disjoint regions that represent all the classes using the probability densities [42]. The hyperparameters for optimization here are Delta, Gamma and the type of discriminate. Delta is the amount of regularization to be applied when estimating the covariance matrix and Gamma is the linear coefficient threshold. The types of discriminates used include linear and quadratic. * Ensemble classifier is relatively new classification technique which has gained popularity in the recent years. It uses different classification techniques to improve the model accuracy [43]. The hyperparameters for optimization here are the classification method, number of learning cycles, learn rate, etc. There are several methods used for multiclassification problems such adaptive boosting, linear programming boosting and random under-sampling boosting. The standard model is obtained by running the data in each classifier using default parameters and finding the best fit model for each fault case. The best fit classifier and default parameters used by the standard model for each fault case are listed in Table 4.  
Then, the best fit algorithm along with its optimal hyperparameters are determined through a process using Bayesian optimization.