## 4 Rule-Based Reasoning
### Individual Classifiers of the Linear Model  
1. Decision Trees (DTs) Classifier  
First, the research team implemented a classifier using the DT, which can work with both numerical and categorical features and also provides interpretability for decision-making [29]. The algorithm starts at the root of the feature tree and splits the dataset on the tree node, which results in the largest information gain, IG, as shown in Equation (4) [28]. The objective function of a decision tree to optimize the training is to maximize the information gain at each node [29]. \[IG\big{(}D_{p},\ f\big{)}=I\big{(}D_{p}\big{)}-\sum_{j=1}^{m}\frac{N_{j}}{N_{p} }I\big{(}D_{j}\big{)} \tag{4}\]  
In Equation (4), \(f\) is the feature to perform the split, \(D_{p}\) and \(D_{j}\) are the datasets of the parent and \(j\)th child node, \(I\) is the measure of data applying the splitting criteria, \(N_{p}\) is the total number of samples at the parent node, and \(N_{j}\) is the number of samples in the \(j\)th child node. As stated by this equation, the information gain is the difference between the measures of splitting data of the parent node and the sum of the child node impurities. The splitting criteria we used in this study is Gini impurity (\(I_{G}\)), which is defined in Equation (5) [28]:  
\[I_{G}(t)=\sum_{i=1}^{c}p(i|t)(1-p(i|t))=1-\sum_{i=1}^{c}p\left(i|t\right)^{2} \tag{5}\]  
Here, \(p(i|t)\) is the proportion of the cases that belong to class \(i\) for a particular node \(t\). Table 8 and Figure 9 shows one of the prediction results of the DT classifier that we implemented and trained using training dataset #1 depicted in Figure 1, or the dataset without the feedback from rule-based reasoning. Among 98 testing cases, 81 cases were correctly classified, which made an overall precision of 0.85, recall of 0.83, and f1-score of 0.83. In order to avoid bias from one test, we repeated the above training and testing process 30 times, where different training and testing cases were randomly selected for each test. The average precision, recall, and f1-score of 30 tests are 0.88, 0.87, and 0.87, respectively. Figure 8: Error matrix used to evaluate the performance of the trained classifiers. 2. Support Vector Machine (SVM) Classifier  
SVM is also a common machine learning classifier that is widely used in practical and theoretically domains [29]. This study implemented a classifier using the SVM algorithm because they can generate accurate and robust results for classification problems, even when training data are nonlinearly separable [29; 33]. They can also be easily extended to solve nonlinear classification problems using a nonlinear \"kernel\" [32; 34]. The optimization objective of SVM classifiers is to maximize the distance between the separating decision boundary and the training samples that are closest to this boundary [29]. A penalty can also be applied for misclassification [28]. Using training dataset #1 and repeating 30 train-then-test cycles the same as stated previously, the SVM classifier implemented by this study used a linear kernel and obtained an average f1-score of 0.79 among 30 tests and an average f1-score of 0.74 when using a radius basis function kernel. 3. K-Nearest Neighbors (k-NN) Classifier  
Next, a classifier using the k-NN algorithm was implemented. k-NN was selected because it is a robust classifier that is often used as a benchmark for more complex classifiers, such as SVM and decision trees [35]. Besides, since both the number of features and size of our training dataset are not very large, this classifier does not suffer from the \"curse of dimensionality\" and low computation efficiency [28]. The average precision, recall, and f1-score of 30 tests for the k-NN classifier with three neighbors and a uniform weight are 0.79, 0.78, and 0.78, respectively, which is close to the results of the SVM classifier and lower than the DT classifier we implemented. This implies that the performances of both SVM and DT classifiers implemented by this study are considerably robust and trustworthy.