## 1 Introduction  
In today's cyber arena, phishing has overtaken malware as the foremost hazard on the Web [1, 2]. Since the onset of the conflict in Ukraine, adversaries have increased their use of credential-harvesting phishing strategies to compromise targeted email accounts [3, 4]. Meanwhile, malicious code, whether in the form of binaries or scripts, remains a persistent and dangerous element. Traditionally, malware detection has relied on static and dynamic analyses of executables present in local storage. In response, cybercriminals have increasingly turned to fileless malware techniques [5, 6], a shift that makes detection more resource-intensive. Advanced methods, such as sandbox-based dynamic analysis, have consequently become more prevalent and costly. Furthermore, the rise of AI-generated phishing emails, polymorphic malware, and living-off-the-land attacks has complicated traditional detection practices [1, 2, 3]. These techniques are increasingly designed to evade signature-based detection, often leveraginglegitimate system tools or continuously mutating their code structure [4; 5]. Additionally, the rapid proliferation of Internet of Things (IoT) devices and edge computing has expanded the attack surface, creating new challenges for maintaining visibility and control across distributed environments [6; 7]. Protecting against these evolving threats continues to be an active focus of research. Notably, machine learning approaches have started employing richer, more expansive, and more intricate datasets and models [7; 8]. Unlike static detection methods such as denial lists, machine learning does not rely solely on current indicators of compromise (IoCs) to maintain effectiveness. Instead, it leverages its capacity to generalize from known samples to recognize never-before-seen threats. However, this adaptability comes at a price: machine learning-driven solutions are inherently computationally intensive. This complexity facilitates the uncovering of nuanced and highly sophisticated attack patterns. Simultaneously, cyber defenders face multiple difficulties stemming from the complex and layered data ecosystems embedded within their infrastructure. Certain segments of this environment are considerably more diverse than others--take, for example, the disparity between processing email attachments versus handling custom application logs. Consequently, detection solutions tailored to specific attack surfaces must accommodate the inherent intricacy of those environments. Although prior research, e.g., [9; 10], has explored the feasibility of compiling numerous attack strategies into robust datasets, only limited attention has been given to the practical costs associated with rolling out such elaborate detection frameworks. Moreover, as infrastructure sizes grow and additional devices are introduced, data movement, computational overheads, and storage demands escalate. All of these factors add time and expense before any model inference occurs, thus increasing latency and expenditure. While prior works have proposed \"multi-layered\" detection architectures as a way to manage these obstacles, the logic guiding progression through each layer has not been finely tuned [9; 11]. For instance, a recent study [12] sought to split detection from attribution steps to streamline processing and boost detection quality. Nevertheless, one might argue that the commonly employed default classifier threshold of 0.5, as observed in the literature (e.g., [13; 14]), is insufficient. In scenarios where model retraining is infrequent, adapting a flexible decision threshold could help sustain overall detection effectiveness. Against this backdrop, we introduce a contextual bandit-based reinforcement learning approach, a solution that brings flexible decision boundaries into multi-tier attack detection systems. Conceived as a framework for decision optimization, our approach implements a value estimation strategy that is intended to lower the operational costs linked to identifying malicious activities.