## 4 Experiments
### ML on Buzz1  
To compare the performance of ConvNets with standard ML models, we trained the four standard ML models on the same Intel Core [EMAIL_ADDRESS] GHz \(\times\) 8 computer with 15.5 GiB of RAM and 64-bit Ubuntu 14.04 LTS with a 60-40 train/test split of the BUZZ1 train/test dataset and the \(k\)-fold cross validation. We did not use the \(k\)-fold cross validation with the DL methods on BUZZ1 (see Section 4.1) for practical reasons: it would take too long to train and test ConvNets on each separate fold on our hardware. In all four ML models, the optimal parameters were discovered through the grid search method [56] where, given a set of parameters, exhaustive search is used on all possible combinations of parameters to obtain the best set of parameters. We used three metrics to evaluate the performance of the ML models: classification accuracy, confusion matrix, and receiving operating characteristic (ROC) curve. The classification accuracy is the percentage of predications that are correct. A confusion matrix summarizes the performance of a model on a dataset in terms of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). An ROC curve is a graph that summarizes a classifier's performance over all possible thresholds. ROC curves are generated by plotting true positive rates on the \(y\)-axis against false positive rates on the \(x\)-axis. Table 5 shows the accuracy results of the ML method M1 (logistic regression) with two testing procedures (i.e., 60-40 train/test split and k-fold cross validation: 5466 training samples and 3644 testing samples) and five feature scaling techniques (i.e., S1, S2, S3, S4, and S5). In both test types, the scaling \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Training Samples: 6377, Testing Samples: 2733** & & & \\ \hline
**Model Name** & **Training Loss** & **Training Accuracy** & **Testing Loss** & **Testing Accuracy** & **Runtime per Epoch** \\ \hline RawConvNet & 0.00692 & 99.85\% & 0.00432 & 99.93\% & 462 s \\ ConvNet 1 & 0.55387 & 99.77\% & 0.57427 & 97.59\% & 545 s \\ ConvNet 2 & 0.67686 & 68.07\% & 0.59022 & 98.21\% & 532 s \\ ConvNet 3 & 0.68694 & 66.81\% & 0.59429 & 98.02\% & 610 s \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Contribution of Custom Layer. RawConvNet is compared with ConvNets 1, 2, and 3 after 100 epochs of training with a 70-30 train/test split of the BUZZ1 train/test dataset. The receptive field size \(n\) is set to 80 for all ConvNets. techniques S1, S2, and S3 resulted in an accuracy above 99%, whereas S4 and S5 caused the accuracy to drop to the low 90% range. Table 6 shows the confusion matrix that corresponds to Test 1 (row 1) in Table 5, i.e., train/test split with no feature scaling (S1). Since the confusion matrices for Tests 2 and 3 (rows 2 and 3 in Table 6, respectively) were almost identical to the confusion matrix in Table 6, we did not include them in the article to avoid clutter. Table 7 shows the confusion matrix that corresponds to Test 4 (row 4 in Table 5) with train/test split and L1 feature scaling (S4). Since the confusion matrix for Test 5 was almost identical to the confusion matrix for Test 4, we did not include it in the article. Regardless of the test type (train/test vs. k-fold), the S1, S2 and S3 feature scaling methods performed better than the S4 and S5 methods. Since the performance of S1 (no scaling) resulted in basically the same accuracy as S2 and S3, feature scaling did not seem to contribute much to improving the classification accuracy of logistic regression. Table 8 shows the accuracy results of the M2 classification method (KNN) with the same two testing procedures and the same five feature scaling methods. The feature scaling methods had almost no impact on the KNN classification accuracy in that the accuracy in all tests was above 99%. Table 9 gives the accuracy results for the M3 classification method (SVM OVR). The SVM OVR classification achieved the best performance when coupled with the S1, S2, and S3 feature scaling methods, and was