# A Machine Learning Approach to Predict Customer Usage of a Home Workout Platform
## 4 Evaluation and Selection of Discriminant Model
### Receiver Operating Curve of a Discriminant Model  
Based on the analysis metrics previously described, the discriminant power of the four models was evaluated. Thus, we performed a receiver operating curve (ROC) analysis. ROC is a plot of the FP (real voice classification as positive) ratio on the x-axis and the TP (real positive classification) ratio on the y-axis.  
A receiver operating characteristic curve, or ROC curve, is a graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied. The method was originally developed for operators of military radar receivers starting in 1941, which led to its name.  
The ROC curve is created by plotting the true positive rate (TPR) against the false positive rate (FPR) at various threshold settings. The true-positive rate is also known as sensitivity, recall, or probability of detection [54] in machine learning. The false-positive rate is also known as probability of false alarm [48] and can be calculated as (1-Specificity).  
\begin{table}
\begin{tabular}{c c c} \hline
**Contents** & **Meaning** & **Numerical Value** \\ \hline Accuracy & Ratio of the exact actual value to the actual value & \(\frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+\text{FN}}\) \\  & (positive to determine the actual voice by negative) & \(\frac{\text{TP}}{\text{TP}+\text{FP}}\) \\ Precision & The percentage of positive values & \(\frac{\text{TP}}{\text{TP}+\text{FP}}\) \\  & positive values & \(\frac{\text{TP}}{\text{TP}+\text{FN}}\) \\ Recall & Percentage of positive determination of actual & \(2*\frac{\text{precision}+\text{recall}}{\text{precision}+\text{recall}}\) \\ F1-Score & Indicators evaluated by considering both factors using & \\  & the harmonic mean of precision and Recall & \(2*\frac{\text{precision}+\text{recall}}{\text{precision}+\text{recall}}\) \\ \hline \end{tabular}
\end{table}
Table 2: Numerical values of confusion matrix.  
It can also be thought of as a plot of the power as a function of the Type I Error of the decision rule. The ROC curve is thus the sensitivity or recall as a function of fall-out.  
Thus, when the curve is more convex upward, a higher TP ratio value and a lower FP ratio value denote better accuracy of the model. The lower part of this ROC curve is referred to as the area under the ROC curve (AUC) [55]. The value range of AUC is within 0.5 and 1. The ROC analyses are shown in Table 3. The AUC values for random forest reached the peak of 0.960, compared with 0.916 for logistic regression, 0.772 for SVM, and 0.734 for KNN. Furthermore, given the high precision, reproducibility, and F1 score of the random forest discrimination algorithm in the discrimination of the current work, the overall discrimination verification power of random forest was better than that of the other three models, as shown in Table 4.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \multicolumn{1}{c}{Random Forest (AUC: 0.960)} & \multicolumn{1}{c}{Logistic Regression (AUC: 0.916)} \\ \hline \hline \end{tabular}
\end{table}
Table 4: ROC Curve.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **AUC** & **F1** & **Precision** & **Recall** \\ \hline Random Forest & 0.960 & 0.928 & 0.930 & 0.930 \\ Logistic Regression & 0.916 & 0.859 & 0.861 & 0.864 \\ SVM & 0.772 & 0.832 & 0.850 & 0.846 \\ KNN & 0.734 & 0.770 & 0.773 & 0.786 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Predictability of evaluation model.