# AI-Powered Stroke Diagnosis System: Methodological Framework and Implementation
## 5 Theoretical Evaluation
### Perfomance Metrics  
The comprehensive evaluation of our predictive models is fundamentally grounded in several pivotal and critical metrics that serve as essential indicators of their performance efficacy. While the overall accuracy metric provides a broad and generalized measure of the model performance across the entirety of the dataset, it is imperative to recognize that in the context of imbalanced datasets, such as the eICU Database, the metrics that specifically pertain to the minority class--namely stroke--hold extraordinary significance and cannot be overlooked. Stroke precision serves a vital role in quantifying the proportion of instances that were predicted to be stroke cases and that are, in fact, accurately confirmed as true stroke incidents, while stroke recall, often referred to as sensitivity, systematically assesses the fraction of actual stroke cases that have been correctly identified by the model. Furthermore, the F1-score, which is mathematically defined as the harmonic mean of precision and recall, provides an insightful and balanced measure of the model's performance.  
Moreover, the ROC curves and their associated AUC values effectively demonstrate the model's capacity to differentiate between classes across various decision thresholds, while the precision--recall curves, accompanied by Average Precision, provide further, detailed insights into the model's performance regarding the stroke class.  
Upon acquiring calibrated probabilities \(\hat{p}_{i}\in[0,1]\) for each model (XGBoost utilizing CalibratedClassifierCV, CatBoost, and ensembles through similar techniques), we systematically vary the decision threshold \(\tau\) from 0.00 to 1.00 in increments of 0.01 and calculate the F\({}_{2}\) score  
\[F_{2}=\frac{\left(1+2^{2}\right)*precision\ *recall}{2^{2}*precision+recall}, \tag{1}\]  
hence assigning double the importance to recollection compared to precision in the acknowledgment of therapeutic necessities. The best cutoff \(\tau^{\star}\) is the probability that maximizes F\({}_{2}\). Since \(\tau\) is dimensionless, reducing \(\tau\) enhances stroke recollection by categorizing additional borderline cases as positive. For instance, in our XGBoost validation set, decreasing \(\tau\) from the normal 0.50 to 0.32 enhanced recall from 47% to 71%, while maintaining precision above 55%. This grid-search method (executed by precision_recall_curve and our find_threshold_for_min_precision utility) guarantees that \"meticulous threshold calibration\" directly results in the increased sensitivity necessary for dependable clinical decision making.  
Table 2 serves as a succinct summary of the key quantitative performance metrics that have been derived from our comprehensive model analysis logs, encapsulating the essential findings of our evaluation process. Although the various boosting models demonstrate a robust level of overall accuracy, it is noteworthy that their moderate stroke recall highlights the inherent challenges and complexities that are posed by the presence of imbalanced clinical data, which can significantly affect the predictive performance. In stark contrast, the Convolutional Neural Network model exhibits an exceptional level of performance, not only in terms of the overall accuracy, but also in the critical task of stroke detection, as is compellingly evidenced by its high precision and recall values, which indicate its superior capability in identifying stroke cases accurately.  
For instance, the XGBoost model achieves an accuracy of 92.9% with stroke precision of 0.45, stroke recall of 0.47, and an F1-score of 0.46, accompanied by an approximate AUC of 0.85 and an average precision of around 0.43. In contrast, the CNN model achieves 96.0% accuracy, with stroke precision of 0.97, stroke recall of 0.91, and an F1-score of 0.94. These figures, extracted from our detailed analysis logs, highlight that although boosting models demonstrate robust overall accuracy, their moderate stroke recall underscores the challenges posed by imbalanced clinical data.