# Performance Predictions of Sci-Fi Films via Machine Learning
## 4 Results  
The Receiver Operator Characteristic (ROC) is a probability curve representing the true positive rate versus the false positive rate. The area under the curve (AUC) is the measure of the ability of the classifier to distinguish between classes (separability). The confusion matrix and the AUC-ROC curve were analyzed to compare the results, in addition to the accuracy, prediction speed, and training time. The confusion matrix is an N \(\times\) N matrix used to evaluate the classification model's performance. Where N is the number of classes, in our example, N is 4, where the actual classes are on the \(y\)-axis, and the predicted classes are on the \(x\)-axis. The confusion matrix extracted each class's true positive and false positive rates. The confusion matrix can generate the performance accuracy, precision, and recall measures [40]. A summary of the obtained results for different utilized KNN classifiers is provided in Table 7.  
Figure 7 shows the AUC-ROC curves for one of the weighted KNN methods. Each subgraph represents the relationship between the true positive rate and the positive rate for the viewer's opinion. Three main parameters are extracted from each graph: the ROC curve behavior, the AUC, and the accuracy of each classifier. The area under the receiver operating characteristic curves (AUC-ROCs) for one of the weighted KNN techniques is displayed in Figure 7; moving down the graph, it can be seen how the percentage of \"yes\" answers reflects the viewer's confidence. Each graph is analyzed for its ROC curve behavior, AUC, and classifier accuracy, three key metrics. An AUC of 0.99, a ROC point of 0.11, and a ROC that plateaus at a 1% true positive rate for a 0.18 percent false positive rate are all easily discernible in the graphic. However, the AUC for the below-average opinion is 0.99, the ROC point is (0.01, 0.94), and the ROC reaches one true positive rate at a false positive rate of 0.32, suggesting a more significant error is accomplished to guarantee the one true positive rate. Similarly, the flop opinion has an AUC of 1, the ROC point is (0.00, 0.93), and the ROC reaches a true positive rate of 1 at a false positive rate of 0 quickly. The ROC point for the hit opinion is (0.01, 0.6), and it achieves a true positive rate of 1 at a false positive rate of 0.2%, indicating that the rate of false positives is moderate. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Algorithm** & \begin{tabular}{c} **Number of** \\ **Neighbors** \\ \end{tabular} & **Distance Metric** & \
\begin{tabular}{c} **Distance** \\ **Weight** \\ \end{tabular} &
\begin{tabular}{c} **Standardize** \\ **Data** \\ \end{tabular} \\ \hline Fine KNN & 1 & Euclidean & Equal & True \\ \hline Medium KNN & 10 & Euclidean & Equal & True \\ \hline Weighted KNN & 10 & Euclidean & Squared Inverse & True \\ \hline \hline \end{tabular}
\end{table} 
Table 6: KNN algorithms. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**KNN Classifier** & **Accuracy (\%)** &
\begin{tabular}{c} **Prediction Speed** \\ **(obs/s)** \\ \end{tabular} & **Training Time** & \multicolumn{4}{c}{**Class True Positive Rate**} \\ \cline{4-7}  & & & **Average** & **Below Average** & **Flop** & **Hit** \\ \hline Fine KNN & 93.0 & \(\sim\)31,000 & 0.6328 & 96\% & 93\% & 100\% & 77\% \\ Medium KNN & 90.3 & \(\sim\)24,000 & 0.21746 & 98\% & 89\% & 73\% & 48\% \\ Weighted KNN & 92.9 & \(\sim\)24,000 & 0.21903 & 97\% & 94\% & 93\% & 60\% \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Summary of the results for different KNN algorithms. Table 7 summarizes the results obtained when different KNN algorithms are implemented. The Fine KNN algorithm provides the highest accuracy of 93%, but it has the lowest speed among the three classes. On the other hand, the Medium KNN classifier has the highest accuracy for the Average class, with a value of 98%, but the lowest for the Hit class, with a value of 48%.