# Applying Machine Learning Techniques to the Audit of Antimicrobial Prophylaxis
## 3 Results
All the algorithms presented with outstanding discrimination, i.e., AUC > 0.97. In this study, 544 instances were classified as surgical prophylaxis, 57 instances were classified as treatment of infections rather than surgical prophylaxis. The confusion matrix of decision tree algorithm for class-label attribute is shown in Table 6. Between-class imbalance was observed in this study, i.e., unequal number of observations in each class. Class C (5%) and E (9.5%) were much smaller subsets, as compared with that of Class A (42.6%), B (25.8%), and D (17.1%). There were also unequal sample sizes and minority instances within each labeled class, i.e., within-class imbalance. The sample sizes of minority instances were too small to be identified accurately by the machine learning tools, especially in Class E, as shown in Table 6.  
For the 544 instances of antimicrobial prophylaxis (Class A, B, C, and D), 75.6% (411/544) of antimicrobial choices (Class A and B) were classified as consistent with guidelines, 52.6% (286/544) of antimicrobial prescriptions (Class A and C) were discontinued within 24 h after surgery completion. Five specialties, orthopedics (22.5%), neurosurgery (13.8%), general surgery (14.0%), obstetrics/gyneecology (13.1%), and urology (9.7%) had the greatest number of instances. The dataset was further stratified and analyzed by specialty. The compliance with the recommendations of guideline for antimicrobial prophylaxis by the five major specialties is plotted in Figure 2. The compliance with recommended choice of prophylactic antimicrobial agents for neurosurgery, obstetrics and gynecology, general surgery, orthopedics, and  
Figure 1: Performance metrics of algorithms for whole dataset (601 instances). \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & \begin{tabular}{c} **Spread Subsampl-** \\ **Decision Tree** \\ \end{tabular} & \begin{tabular}{c} **Decision** \\ **Tree** \\ \end{tabular} & \begin{tabular}{c} **Bagging-Decision** \\ **Tree** \\ \end{tabular} & \begin{tabular}{c} **AdaBoost-** \\ **Decision Tree** \\ \end{tabular} & \begin{tabular}{c} **SMOTE-** \\ **Decision Tree** \\ \end{tabular} & \
\begin{tabular}{c} **Simple-** \\ **Logistic** \\ \end{tabular} &
\begin{tabular}{c} **Multilayer** \\ **Perceptron** \\ \end{tabular} \\ \hline Sensitivity & 0.895 & 0.932 & 0.940 & 0.945 & 0.960 & 0.958 & 0.967 \\ Specificity & 0.970 & 0.972 & 0.968 & 0.980 & 0.990 & 0.990 & 0.992 \\ Precision & 0.908 & 0.931 & 0.941 & 0.944 & 0.961 & 0.959 & 0.967 \\ AUC a & 0.978 & 0.985 & 0.991 & 0.983 & 0.985 & 0.987 & 0.992 \\ \hline \hline \end{tabular}  
* AUC: Area under receiver operating characteristics curve.  
\end{table} 
Table 5: Weighted average of performance metrics of algorithms for whole dataset. urology were 64.5%, 71.6%, 80.0%, 80.5%, and 81.0%, respectively. The compliance with duration of surgical prophylaxis for neurosurgery, urology, general surgery, orthopedics, and obstetrics and gynecology were 36.8%, 46.6%, 54.7%, 67.9%, and 77.0%, respectively. The AUC of six algorithms for five major surgical specialties are plotted in Figure 3. Multilayer perceptron was the best algorithms among the five specialties. Bagging with decision tree, decision tree with SMOTE could increase the performance of decision tree for the datasets of most specialties, except for urology dataset. SimpleLogistic worked better than decision tree with sampling methods for the datasets of obstetrics and gynecology and urology, but not for neurosurgery dataset. The hyperparameters and execution time of machine learning algorithms and manual review are shown in Table 7 (Whole dataset in Supplementary Materials). Auto-WEKA and multilayer perceptron took more execution time as compared with that of decision tree. The execution time of decision tree with SpreadSubsample and SMOTE is longer than that of decision tree alone. All the algorithms were more efficient in execution time than manual review.