## 3 Results
### Feature Selection and Model Development
#### 3.3.2 The Performance of the Classification Model  
We trained four machine learning classifiers using 5-fold cross-validation with class weighting to evaluate different feature combinations. We compared three feature groupings: the basic covariates (years in education, MMSE, E-SDMT, P-SDMT scores), the handwriting dynamics (the median response interval, air length during clock-drawing, the CV in velocity during tree-drawing, and the air-paper ratio during trail-making), and the combined eight-feature set. Table 4 presents comprehensive performance metrics, with the SVM-RBF model achieving the highest overall performance. Key metrics include an AUC = 0.910 (95% CI: 0.853-0.967) for the overall discrimination ability; a balanced accuracy = 0.838 (95% CI: 0.753-0.915) accounting for the class imbalance; and an F1-score = 0.800 (95% CI: 0.679-0.895), balancing precision and recall. To understand the value of the handwriting features, we compared models trained on different feature subsets (Figure 8). The combined model (with basic covariates plus handwriting features) outperformed both the model using basic covariates alone and the model using only handwriting features. Adding handwriting features to the basic covariates improved the performance across all metrics, with notable increases in the AUC (from 0.884 [95% CI: 0.817-0.952] to 0.910 [95% CI: 0.853-0.967]), balanced accuracy (from 0.778 [95% CI: 0.684-0.867] to 0.838 [95% CI: 0.753-0.915]), and precision (from 0.714 [95% CI: 0.537-0.854] to 0.839 [95% CI: 0.663-0.945]). When analyzing each feature group independently, the traditional cognitive measures (years in education and MMSE and SDMT scores) provided a strong baseline perfor \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**Model** & **Accuracy** & **Balanced Acc.** & **Sensitivity** & **Specificity** & **Precision** & **F1-Score** & **AUC** \\ \hline SVM-RBF & 0.856 & 0.838 & 0.765 & 0.911 & 0.839 & 0.800 & 0.910 \\  (0.766-0.921) & (0.753-0.915) & (0.588-0.893) & (0.804-0.970) & (0.663-0.945) & (0.679-0.895) & (0.8853-0.967) \\ Logistic Regression & 0.811 & 0.790 & 0.706 & 0.875 & 0.774 & 0.738 & 0.871 \\ Random Forest & 0.789 & 0.767 & 0.676 & 0.857 & 0.742 & 0.708 & 0.885 \\ XGBost & 0.767 & 0.743 & 0.647 & 0.839 & 0.710 & 0.677 & 0.884 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance comparison of the classification model using the combined feature set (95% confidence intervals shown for the SVM-RBF model only to conserve space). Figure 8: A performance comparison across different feature groupings. The combination of the basic covariates and handwriting features achieves the highest AUC, demonstrating the complementary value of these feature types. mance (AUC = 0.884; 95% CI: 0.817-0.952). However, the handwriting dynamics features also demonstrated substantial discriminative ability on their own (AUC = 0.835; 95% CI: 0.756-0.914), suggesting that they captured important aspects of cognitive function that complemented the traditional assessments. Figure 9 shows the confusion matrix for the best-performing SVM-RBF model. The model correctly classified 26 of 34 cognitive impairment cases (76.5% sensitivity) and 51 of 56 cognitively normal cases (91.1% specificity), demonstrating a good performance across both classes despite the class imbalance in our dataset. To examine the potential impact of disease heterogeneity on the classification performance, we conducted disease-specific subgroup analyses for the MS (\(n=45\)) and NMOSD (\(n=34\)) patients. Figure 10 illustrates the model performance across different disease subgroups. Despite the uneven disease distribution between cognitive groups (MS: 60.7% CN patients vs. 32.4% CI patients, \(p=0.006\)), our approach maintained a robust performance in both subgroups. The SVM-RBF model achieved robust discrimination in MS patients (AUC = 0.914; balanced accuracy = 0.864), with strong sensitivity (84.6%) and specificity (88.2%).