## 4 Evaluation
### Classification Risk  
To assess the risks associated with triage, we consider how often items require a subsequent classification step. The underlying concept is that while the initial classification (layer 1) provides a preliminary decision, the subsequent classification stage (layer 2) can potentially alter that decision. This means that an observation initially deemed to be benign (a true negative) might later be identified as malicious (a false positive), thus altering the final classification outcome. Figures 3 and 4 present comparative results for various reward policies and selection strategies across both datasets and model types. Notably, the \(log(k)\) policy consistently demonstrates the lowest classification risk, irrespective of dataset, model type, or selection approach. Minimizing disturbances to the correct prediction space is generally desirable. Nonetheless, the degree of overlap between true and false prediction distributions can influence outcomes, as some datasets produce probability distributions that make it more challenging to avoid unnecessary reclassifications. To further quantify the impact of triage decisions on prediction quality, we examined the false positive rate (FPR)--the proportion of benign instances incorrectly flagged as malicious. Our findings indicate that contextual bandit configurations with aggressive triage reductions (e.g., high-confidence thresholds) may slightly increase the FPR. However, policies such as \(log(k)\), which restrict interference with correct predictions, effectively shift the focus toward the primary model's accuracy. Recognizing that probability distributions often overlap, practitioners seeking to optimize detection performance with modest cost reductions might consider \(log(d_{P,a,t}\times k)\). Although \(log(d_{P,a,t}\times k)\) endeavors to encapsulate as many negative observations as possible, the inevitable overlap between true and false  
Figure 3: Performance comparison of reward selection strategies in multi-layered detection frameworks based on true positives. prediction confidences can trigger avoidable second-stage classifications, inflating the cost for true predictions that would not have required re-evaluation (see Table 3). \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Dataset** & \begin{tabular}{c} **Detection** \\ **Model** \\ \end{tabular} & \begin{tabular}{c} **Reward** \\ **Description** \\ \end{tabular} & \begin{tabular}{c} **Selection** \\ **Strategy** \\ \end{tabular} & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline \multirow{8}{*}{Subset A} & \multirow{4}{*}{\
\begin{tabular}{c} Neural \\ Network \\ \end{tabular} } & \(log(k)\) & UCB & 0.891 & 0.881 & 0.902 & 0.891 \\  & & \(log(k)\) & Thompson & 0.892 & 0.882 & 0.903 & 0.892 \\  & & \(log(k)\) & UCBE & 0.89 & 0.88 & 0.901 & 0.89 \\  & & \(\rho^{d_{P,A}}\) & Thompson & 0.889 & 0.879 & 0.9 & 0.889 \\  & & 0.9 & Thompson & 0.885 & 0.875 & 0.896 & 0.885 \\ \cline{2-8}  & & \(log(k)\) & UCBE & 0.901 & 0.891 & 0.911 & 0.901 \\  & & \(log(k)\) & UCB & 0.902 & 0.892 & 0.912 & 0.902 \\  & & \(log(k)\) & Thompson & 0.9 & 0.89 & 0.91 & 0.9 \\  & & \(\rho^{d_{P,A}}\) & Thompson & 0.899 & 0.889 & 0.909 & 0.899 \\  & & 0.9 & Thompson & 0.895 & 0.885 & 0.905 & 0.895 \\ \hline \multirow{8}{*}{Subset B} & \multirow{4}{*}{
\begin{tabular}{c} Neural \\ Network \\ \end{tabular} } & \(log(k)\) & UCB & 0.971 & 0.961 & 0.981 & 0.971 \\  & & \(log(k)\) & UCBE & 0.972 & 0.962 & 0.982 & 0.972 \\  & & \(log(k)\) & Thompson & 0.971 & 0.961 & 0.981 & 0.971 \\  & & \(d_{P,A,t}^{d}\) & [PERSON] & 0.97 & 0.96 & 0.98 & 0.97 \\  & & \(\rho^{d_{P,A,t}}\) & [PERSON] & 0.969 & 0.959 & 0.979 & 0.969 \\ \cline{2-8}  & & \(log(k)\) & UCBE & 0.994 & 0.984 & 0.995 & 0.994 \\  & & \(log(k)\) & UCB & 0.993 & 0.983 & 0.994 & 0.993 \\  & & \(log(k)\) & [PERSON] & 0.993 & 0.983 & 0.994 & 0.993 \\  & & \(d_{P,A,t}^{d}\) & [PERSON] & 0.992 & 0.982 & 0.993 & 0.992 \\  & & \(\rho^{d_{P,A,t}}\) & [PERSON] & 0.991 & 0.981 & 0.992 & 0.991 \\ \hline \hline \end{tabular}
\end{table}