## 4 Research and Project Aimed at Developing Modules of a Classification and Expert System
### Final Classification After Treatment Module
#### 4.3.3 Achieved Results
* **Moderate scores: AdaBoost, K-nearest neighbors (KNN)**, and **Naive Bayes** showed lower scores (from around 0.88 to 0.87), suggesting moderate trade-offs in precision and recall. The highest F1 scores were achieved by ensemble models such as **LightGBM** and **random forest** (and several other models), confirming their robustness for applications requiring a balanced trade-off between precision and recall. The suboptimal performance of **KNN**, **Naive Bayes**, or **AdaBoost** suggests that it may require further optimization to improve their effectiveness in handling complex data. These observations reinforce the utility of ensemble methods in delivering better overall performance. The confusion matrices, presented in Figure 10, provide detailed insights into the classification performance of the evaluated machine learning models. By illustrating the distribution of true positives, false positives, true negatives, and false negatives across all seven outcome classes, the confusion matrices enable a deeper understanding of the models' strengths and weaknesses. Key observations were as follows:  
* **Random forest:** The **random forest** classifier demonstrated high accuracy across most classes, with minimal misclassifications. Notable challenges included occasional confusion with Class 5, likely due to overlapping clinical features in these categories. * **KNN: KNN** had slightly more difficulty with accurate predictions, showing misclassification in some classes. These results were consistent with its lower precision, recall, and F1 scores compared to the other models, indicating some limitations of the model in handling complex data distributions. Figure 9: F1-score results for the designed models. * **Bagging:** The **Bagging** classifier showed robust performance, with relatively balanced classification across all classes. Misclassifications were rare and mostly occurred between adjacent classes, reflecting its ability to handle minor ambiguities effectively. * **LightGBM:** LightGBM achieved the most accurate predictions, with the confusion matrix showing strong diagonal dominance, indicating excellent classification performance. Misclassifications were minimal. Misclassifications between specific classes (see Figure 10) are particularly concerning in clinical contexts, as these distinctions inform treatment strategies. The confusion observed in **KNN's** matrix highlights the importance of selecting models that prioritize precision and recall in critical clinical categories. The confusion matrix analysis confirmed that some ensemble models like **Bagging** and **LightGBM** outperformed less robust methods like **KNN**. This strengthens the conclusion that some ensemble techniques are better suited to classifying FSGS scores, offering greater accuracy and reliability in real-world clinical applications. This part provides an overview of the cross-validation results for the machine learning models designed for classifying FSGS outcomes. Cross-validation was performed using 5-fold splitting to ensure a robust estimation of model performance. Each model's mean accuracy score across the folds is presented in Table 7. The results highlight key insights into the models' generalization capabilities and reliability. Figure 10: Example confusion matrix for selected models. The cross-validation results revealed the following key insights:  
* **Top performers: LightGBM** and **random forest** demonstrated the highest mean accuracy (around 0.93), closely followed by **logistic regression** and **Bagging** (all with an accuracy of around 0.92). These models showed strong consistency with low standard deviation, indicating reliable performance across different folds. * **Ensemble methods: Bagging, random forest**, and **LightGBM** performed well, confirming the effectiveness of ensemble techniques in handling complex data. These models were not only accurate but also demonstrated robust performance, with small variability across folds.