## 2 Materials and Methods
### Experimental Setup and Performance Measure  
To construct the risk prediction models for digoxin treatment, this study adopted Weka 3.9.4 (www.cs.waikato.ac.nz/mL/weka), an open source machine learning software to investigate the performance of the selected techniques, including IBk (kNN in Weka), J48 (C4.5 in Weka), SMO (SVM in Weka), random forest (RF in Weka), multilayer perceptron (ANN in Weka), simple logistic (LGR in Weka). To optimize the prediction performance of the models, we used the CV parameter selection meta-learner module implemented in Weka to search the best parameter settings. Table 3 lists the tests of parameter tuning for each classification technique and the final best parameter settings found in this study.  
The classification performance of the prediction models was compared to find out the optimal classifier. We split the data into training and testing datasets in a proportion of 70:30 and applied 10-fold cross-validation method in the training dataset to search the best parameter settings. In addition, previous research indicated that the class imbalance problem can significantly affect learning performance [39]. To improve the classification performance, the resample module (spread subsample module in Weka) is used in this study to modify the distribution of instances of two classes in the training set. Finally, we used the best parameter setting to construct the binary classification models and reported the classification results of testing set.  
By using a confusion matrix (Table 4), the precision, recall and F-measure were adopted to analysis the performance of each prediction models. These three metrics were defined by the following Formulas:  
\[Precision=TP/(TP+FP), \tag{1}\]  
\[Recall=TP/(TP+FN), \tag{2}\]  
\[F\text{-}measure=2\text{. Precision. Recall/(precision+recall)} \tag{3}\]  
Precision is the ratio of correctly predicted high risk patients to the total predicted high risk patients. Recall is the ratio of correctly predicted high risk patients to all of the high risk patients  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Classes**} & \multicolumn{2}{c}{**Predicted Class**} \\ \cline{2-4}  & & **High Risk** & **Low Risk** \\ \hline \multirow{2}{*}{Actual Class} & High risk & TP & FN \\  & Low risk & FP & TN \\ \hline \hline \end{tabular}
\end{table}
Table 4: Confusion matrix.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Technique** & **WekA Module** & \multicolumn{2}{c}{**Parameters**} & **Test Range** & **Best Parameter Setting** \\ \hline kNN & IBk & Number of neighbors & 1-10 & 1 \\ C4.5 & J48 & Confidence factor & 0.1-0.3 & 0.3 \\  & & Minimum number of instances per leaf & 2-10 & 2 \\ SVM & SMO & Kernel & Polykernel & Polykernel \\ RF & Random forest & Number of trees & 10-100 & 60 \\  & & Number of attributes to be used in random selection & 2-10 & 10 \\  & & Number of hidden nodes & 3-10 & 9 \\ ANN & Multilayer perceptron & Learning rate & 0.1-0.5 & 0.3 \\  & & Momentum factor & 0-0.5 & 0 \\  & Maximum number of epochs & 300-1000 & 1000 \\ LGR & Simple logistic & None & N/A \\ \hline \hline \end{tabular}
\end{table}
Table 3: Parameter settings in Weka.  
in the dataset. F-measure is the harmonic mean of precision and recall. In addition, our study also included the area under the curve (AUC) as a performance indicator. To evaluate the performance of the models, we referred to the guidelines proposed by [PERSON] and [PERSON] [38]. The performance is defined as \"outstanding\" if AUC \(\geq\) 0.9, \"excellent\" if 0.9 \(>\) AUC \(\geq\) 0.8, \"acceptable\" if 0.8 \(>\) AUC \(\geq\) 0.7, \"fair to poor\" if 0.7 \(>\) AUC \(\geq\) 0.5.