## 5 Discussion
Our results further suggest that a model's detection performance, shaped by the particular dataset, influences the degree of achievable cost mitigation. Realistically, not all models cleanly partition their predictions into confidently correct versus uncertain classifications. Some degree of overlap is inevitable, and not every dataset will be comprehensive enough to ensure that all incorrect classifications register as low-confidence and, thus, become targets for our decision optimization process. Consequently, ongoing improvements in the underlying detection models remain essential. By enhancing a model's baseline accuracy, our contextual bandit-based reinforcement learning approach can more effectively leverage confidence-based cost savings, focusing resource reductions primarily on the high-confidence prediction domain. In addition, our experiments confirm that a model's intrinsic strength has a direct bearing on how confidently it assigns probabilities to its predictions. The pairwise variations introduced by using two distinct datasets and two classifier types provided ample opportunity to observe differences in confidence levels. Ideally, the training process should encourage a model to assign lower confidence values to incorrect predictions than to correct ones, thereby facilitating the more precise implementation of cost-saving strategies. Our evaluation utilized a two-tier detection structure for identifying malware and phishing threats, mirroring the architectures employed by certain commercial cloud-based services (e.g., Microsoft Defender, Version: 1.429.356.0). In a similar fashion to our contextual bandit-based reinforcement learning approach, Microsoft Defender uses a cost-efficient local detection component supported by a more resource-intensive cloud-based system that is only invoked when the local module cannot produce a definitive decision. Although our contextual bandit-based reinforcement learning approach could theoretically be scaled to incorporate additional models, or even human analysts, such expansion increases the complexity of determining their optimal cost-performance ordering. Extensibility to Multi-Layered Architectures: Our contextual bandit-based reinforcement learning framework is intentionally modular, allowing it to be integrated into more complex, multi-layered detection environments. In such systems, each detection layer--ranging from signature-based filters to advanced behavioral analysis engines--can be associated with different computational costs and detection strengths. The reward function in our framework can be adapted to reflect these varying cost-accuracy trade-offs, enabling the system to dynamically adjust decision thresholds across layers. This makes it feasible to design pipelines where lightweight models filter the majority of traffic, while more resource-intensive models are invoked only when required. Such an approach mirrors real-world architectures, like those found in security operation centers (SOCs) and cloud-native security platforms, which often incorporate multiple detection stages before escalating alerts. Future work will explore how this dynamic allocation strategy can be extended to include additional layers, including human-in-the-loop verification stages. While our study focuses on contextual bandits, which are particularly suited for real-time, low-latency decision-making with limited feedback, we acknowledge the broader reinforcement learning landscape. Compared to hierarchical reinforcement learning methodsthat address large state-action spaces [59] and transfer reinforcement learning frameworks that leverage previously learned knowledge for improved performance on new tasks [60], contextual bandits offer faster convergence and lower computational requirements. These qualities make them especially effective for adaptive threshold tuning in time-sensitive, cost-constrained environments such as cyber threat detection.