## 6 Experiment Evaluation
### Evaluation Results
#### 6.2.3 Stroke Prediction Results  
We use BPNN, SVM, NB, RF and LR as the classification algorithm, and adopted 10-fold cross validation to verify the correctness of the experimental results. The original positive and negative samples were randomly partitioned into 10 equal-sized subsamples, respectively. Then we fitted our model to a data set consisting of 9 of the original 10 parts (including 18 positive and 144 negative samples), and the remaining one subsample was used as the testing dataset. The cross-validation process is repeated 10 times, with each of the 10 subsamples used exactly once as the testing dataset. The 10 results were assembled  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Feature** & **Description** \\ \hline SYST & Systolic blood pressure \\ DIAS & Diastolic blood pressure \\ HTNDerv & Hypertension \\ ParRpDiab & History of diabetes \\ AFIB & Atrial fibrillation or flutter \\ Lvh & Left ventricular hypertrophy \\ Chol & Cholesterol \\ \hline \hline \end{tabular}
\end{table}
Table 10: List of the clinical features.  
Figure 6: The box-plots of all features. The whiskers represent the smallest and largest observations, the edges of the box correspond to the lower and upper quartiles, the horizontal line indicates the median and the plus sign marks probable outliers.  
to produce a single estimation. As shown in Figure 7, the Precision-Recall (PR) curves indicate that the proposed model can keep a relatively high TPR and obtain a low FPR at the same time.  
Figure 7 shows that the combination of the baseline features and the proposed features is superior to the baseline features. Obviously, a good classification method should have both a high detection rate and low false alarm rate; therefore, the pair (Precision, Recall) is widely used to evaluate the performance of classification method. They are defined as:  
\[precision=\frac{TP}{TP\,+\,FP}. \tag{7}\]  
\[recall=\frac{TP}{TP\,+\,FN}. \tag{8}\]  
The models are further evaluated regarding four metrics, namely, the mean precision, recall, and F-Measure. As shown in Figure 8, the models perform better when using feature combinations than using baseline features. Taking the SVM-based model as an example, the precision, recall and PRC of the SVM-based model were improved by 27%, 55% and 46%, respectively.  
Specifically, in Figure 8, comparison between the TPR and the FPR for both the selected algorithms allows the observation that SVM (parameters: k = Polynomial and c = 0.8) and LR achieved better performance (higher TPR and lower FPR) than the other three models. Therefore, we built our basic models using SVM and LR, and then further optimized them using the pre-selection model (parameter: K = 7 in the pre-selection model). In particular, through testing different adaptive thresholds in algorithm 1, we found that the results of SVM were superior to LR, with an acceptance threshold equal to 0.2, depending on the empirical results. In terms of true positive rate and false positive rate, the performance of the optimized prediction model is shown in Figure 9.  
Accordingly, compared to the basic SVM, the SVM+pre-selection model achieves better performance, where the true positive rate was increased by 21%. Specifically, as shown in Figure 10, the optimized prediction model can predict 17 of the 20 Type II participants successfully, and only 10 of the 159 Type IV participants are labeled wrongly, while the  
Figure 7: Precision Recall (PR) curves of the proposed features and the baseline features. (**a**) Back Propagation (BP) neural network models PR curve, (**b**) Support Vector Machine (SVM) models PR curve, (**c**) Random Forest models PR curve, (**d**) Naïve Bayesian models PR curve, (**e**) Logistic Regression (LR) models PR curve.