# Fault Detection in Wireless Sensor Networks through the Random Forest Classifier
## 6 Simulations and Results
### Results  
To evaluate the performance of SVM, RF, MLP, SGD, CNN, and PNN, four metrics were used for comparison. The first metric was DA [1; 7], and it is formulated as given in Equation (8):  
\[DA=\frac{\text{Number of faulty observations detected}}{\text{Total number of faulty observations}} \tag{8}\]The other metric used was TPR [7]. It is the measure of actual positives, which are correctly identified. It is defined as follows:  
\[TPR=\frac{TP}{TP+FN} \tag{9}\]  
In Equation (9), True Positive (TP) refers to the measurements that predict true positives, while False Negatives (FN) are those measurements that are incorrectly claimed as negative. The third metric used is Matthews Correlation Coefficient (MCC) [34], which is used to rank fault diagnostic schemes according to their DA. The range of MCC is between \(-1\) and one. The technique that scores \(-1\) is an incompatible algorithm; zero stands for similarity to random prediction; while one stands for an ideal technique. A closer value to \(+1\) states a very strong relationship between reality and the test. MCC is defined as follows:  
\[MCC=\frac{TP\times TN-FP\times FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}} \tag{10}\]  
In Equation (10), True Negative (TN) declares correctly-defined non-faulty nodes, while False Positive (FP) is defined as the number of faulty nodes incorrectly identified as faulty nodes. MCC was applied on all six classifiers that were implemented in this scenario to test the accuracy of binary classification and rank algorithms. The fourth metric used for statistical performance evaluation is the F1-score. The F1-score [35] is fundamentally the harmonic mean of precision and recall. It is used as a statistical measure to rate the performance of an individual classifier on the basis of FN and FP. Precision is defined as the accuracy of judgment, whereas recall is detecting all the instances in a sample that bear witness to a certain attribute, i.e., faulty or non-faulty. The F1-score is defined as below in Equation (11);  
\[F1-Score=\frac{2\times Precision\times Recall}{Precision+Recall} \tag{11}\]  
Table 3 shows the ranking of all the classifiers based on the MCC score. The values between \(-0.10\) and \(-0.50\) indicate weak classifiers; therefore, they have a poor relation with reality. In our scenario, RF and SVM were considered compatible because of the acquired MCC values. Since RF was closest to \(+1\) as compared to SVM, it was ranked as first, while SVM was graded as the second compatible algorithm, with a score of \(0.65\), among all the classifiers. Figure 3 shows the DA of SVM for all six fault types that were induced at different fault rates (\(10\%\), \(20\%\), \(30\%\), \(40\%\), and \(50\%\)). The best learning rates were obtained at \(\sigma=0.5\) and \(C=1\) after an extensive parameter tuning, where the kernel function of SVM has a parameter that significantly impacts the performance of the classifier, referred to as \(\sigma\), whereas \(C\) refers to the cost function that controls the influence of each support vector in SVM. The decrease of DA in gain and spike faults is because of the constant value \(\beta\) multiplied by the normal reading. The spike fault at \(0.3\) probability showed an usual decrease in DA because at \(30\%\), outliers were diligent at detecting, while at \(50\%\), the fault rate was increased and SVM did not perform well with the increased number of outliers in large datasets. Furthermore, the performance of SVM depends on the distance of support vectors from \begin{table}
\begin{tabular}{c c c} \hline \hline
**Technique** & **Matthews Correlation Coefficient (MCC)** & **Rank** \\ \hline SVM & \(0.65\) & \(2\) \\ RF & \(0.73\) & \(1\) \\ SGD & \(-0.32\) & \(6\) \\ MLP & \(-0.46\) & \(3\) \\ CNN & \(-0.43\) & \(4\) \\ PNN & \(-0.39\) & \(5\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: MCC of SVM, RFC, SGD, MLP, CNN, and PNN. the decision surface; therefore, at a 50% fault rate, the distance decreased from the margin, which illustrates the unusual behavior of the graph.