## 3 Results
### Association of Archer and RS Classification with Orcantral Communications  
As already described by other authors, we were able to find a significant association between OAC for the Archer groups B-D (Figure 6) as well as for the RS classes III/IV (Figure 7). We found these results for both the test and the validation set. Using multilayer perceptron models (MLP) and radial basis function neural networks (RBNN), the AUC was 0.59 (Figure 8) for the Archer classification and 0.638 (Figure 9) for the RS classification. Using the RBNN, we achieved an AUC of 0.599 for the Archer classification (Figure 10) and an AUC of 0.63 for the RS classification (Figure 11). \begin{table}
\begin{tabular}{c c c} \hline \hline
**Algorithm** & **Accuracy** & **AUC** \\ \hline Custom CNN & & \\ val & 0.708 & 0.716 \\ test & 0.745 & 0.853 \\ \hline BoVW & & \\ val & 0.792 & 0.694 \\ test & 0.726 & 0.715 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: _Cont._  
Figure 6: Illustration of Archer and OAC associations with CHAID decision trees for the test and validation dataset (80/20). Percent correct for the training set: 77.1%; percent correct for the test set: 77.4%. Figure 7: Illustration of RS and OAC associations with CHAID decision trees for the test and validation datasets (80/20). Percent correct for the training set: 76.3%; percent correct for the test set: 81.4%. Figure 8: Prediction of OAC using the Archer classification and MLP. The ROC curves show the predictive capability of the Archer class B–D in predicting OAC. Input layer: Archer classification (2 units). Hidden layer: 2 units; activation function: hyperbolic tangent. Output layer: dependent variable OAC (2 units); activation function: softmax; error function: cross-entropy. Number of units in the hidden layer was determined by the testing data criterion: the best number of hidden units is the one that yields the smallest error in the testing dataset. Train/test/val split: 80/10/10. Percent incorrect predictions on the validation set: 25.5%; percent incorrect predictions on the test set (hold out): 17.9%. AUC: 0.590. Figure 9: Prediction of OAC using the RS classification and MLP. The ROC curves show the predictive capability of the RS class III, IV in predicting OAC. Input layer: RS classification (2 units). Hidden layer: 2 units, activation function: hyperbolic tangent. Output layer: dependent variable OAC (2 units), activation function: softmax; error function cross-entropy. Number of units in the hidden layer was determined by the testing data criterion: the best number of hidden units is the one that yields the smallest error in the testing dataset. Train/test/val split: 80/10/10. Percent incorrect predictions on the validation set: 32.7%; percent incorrect predictions on the test set (hold out): 25.8%. AUC: 0.638. Figure 11: Prediction of OAC using the RS classification and RBNN. The ROC curves show the predictive capability of the RS class III, IV in predicting OAC. Input layer: RS classification (2 units). Hidden layer: 2 units, activation function: softmax. Output layer: dependent variable OAC (2 units); activation function: identity; error function: sum of squares. Number of units in the hidden layer was determined by the testing data criterion: the best number of hidden units is the one that yields the smallest error in the testing dataset. Train/test/val split: 80/10/10. Percent incorrect predictions on the validation set: 25.5%; percent incorrect predictions on the test set (hold out): 30.2%. AUC: 0.630. Figure 10: Prediction of OAC using the Archer classification and RBNN. The ROC curves show the predictive capability of the Archer class B-D in predicting OAC. Input layer: Archer classification (2 units). Hidden layer: 3 units; activation function: softmax. Output layer: dependent variable OAC (2 units); activation function: identity; error function: sum of squares. Number of units in the hidden layer was determined by the testing data criterion: the best number of hidden units is the one that yields the smallest error in the testing dataset.