# Combining Supervised and Unsupervised Fuzzy Learning Algorithms for Robust Diabetes Diagnosis
## 3 Results
### Performance Evaluation  
We use random stratified sampled hold-out to setup the training set and the test set as 70% of the dataset is used in the training phase. The experiment is repeated 30 times and the dataset is randomly sampled at each hold-out cycle. The performance result reported here is the average of these 30 times hold-out experiments.  
The effect of the proposed method is measured with _sensitivity_, _specificity_, _accuracy_, and _F1 score_ in comparison with the backpropagation neural network (BPNN) with the unipolar sigmoid function and stand-alone FMM. All performance measures are based on the quantity of _TP_ (True positive), _TN_ (True Negative), _FP_ (False Positive), and _FN_ (False Negative) as shown in Table 2.  
The effect of combining fuzzy unsupervised learner and fuzzy supervised learner is summarized in Table 3. A simple fast supervised learner FMM is extremely good in _sensitivity_ but extremely poor in _specificity_ in this experiment. That means FMM alone produces too many _F_Ps. By combining FCM and FMM, the proposed method shows more stable and robust result with highest _F1 score_ and _accuracy_. Standard BPNN is not remarkable in all measures.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Attributes** & **Missing** \\ \hline Number of times pregnant & 0 \\ Plasma glucose concentration & 5 \\ Disastolic blood pressure & 35 \\ Triceps skin fold thickness & 221 \\
2-hour serum insulin & 375 \\ Body mass index & 11 \\ Diabetes pedigree function & 0 \\ Age & 0 \\ Class (0 or 1) & 0 \\ \hline \hline \end{tabular}
\end{table}
Table 1: PID dataset configuration.  
\begin{table}
\begin{tabular}{c c} \hline \hline _Accuracy_ & \(\frac{(TP+TN)}{(TP+TN+FP+FN)}\) \\ \hline _Sensitivity_ & \(\frac{TP}{TP+FN}\) \\ \hline _Specificity_ & \(\frac{TN}{TN+FP}\) \\ \hline _Precision_ & \(\frac{TP}{TP+FP}\) \\ \hline _F1 Score_ & \(2\Big{(}\frac{Precision+Sensitivity}{Precision+Sensitivity}\Big{)}\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance evaluation metrics.  
To verify the performance of three tested algorithms statistically, we use one-way ANOVA (Analysis of Variance) test and subsequent Tukey test to see if there exist any statistically significant group mean differences for each performance metric. We use open web site for statistical computation ([[http://vassarstats.net/](http://vassarstats.net/)]([http://vassarstats.net/](http://vassarstats.net/)) (accessed on 4 November 2022)) for this computation and the test results are summarized as shown in Tables 4-7 as follows: In Tables 4-7, HSD is the absolute [unsigned] difference between any two-sample means required for significance at the designated level (HSD [0.05] for the 0.05 level), M1 is the proposed FCM-FMM, M2 is Standalone FMM, and M3 is BPNN.  
The PID Dataset is used in many other machine learning algorithms and their performance is reported in the literature. However, [PERSON] et al.'s approach [23] applied outlier rejection preprocessing so that the number of actual instances learned is different and [PERSON] et al. approach [25] is designed to find optimum number of hidden layers in neural network model thus the purpose is different and only the _accuracy_ (75.7%) is comparable to our result.  
Two previous works reported the same measurement as ours and no specific preprocessing is treated. Thus, we report the comparison of our proposed method with other single supervised learning algorithms from the literature in Table 8.  
With the same experimental condition, our proposed method is the best in _sensitivity_ and the _accuracy_ and the second in _F1 score_. Thus, at least, this hierarchical combination of rather simple fuzzy unsupervised and supervised learner is competitive in this highly vague and complex domain. Naive Bayes [28] is the best for other two measures (_specificity_ and _F1 score_) among five compared algorithms in Table 8.