## 1 Introduction  
Interruptive alerts, in the form of suspended pop-up windows, have been widely used in clinical decision support (CDS) and computerized physician order entry (CPOE) systems to prevent medical errors during diagnosis and treatment [1, 2, 3]. In contrast to soft-stop and passive alerts, interruptive alerts require the physician to take one or more actions to close the alert dialog box. Although several studies have shown that interruptive alerts are an effective way to prevent dangerous prescriptions by ensuring that physicians are likely to see them, they can also lead to unintended consequences [4; 5; 6; 7], such as additional cognitive burden, delays in the delivery of appropriate therapy [8; 9; 10], and general alert fatigue.  
Alert fatigue, defined as \"the mental state that is the result of too many alerts consuming time and mental energy [11; 12]\", usually occurs when interruptive alert systems unnecessarily and frequently distract clinicians from their thought processes [13]. Consequently, both clinically important and unimportant alerts may be ignored. Causes of alert fatigue include, but are not limited to: (1) a large number of alerts [14]; (2) clinically irrelevant alerts or alerts of low clinical value [15; 16]; and (3) miscommunication or lack of communication of the meaning of the alert [17].  
Previous studies have primarily used alert override rates (total number of alerts that do not change the physician's prescribing behavior divided by the total number of alerts presented to the physician) to evaluate alert efficiency [18; 19; 20]. However, using alert override rates to evaluate alert efficiency still assumes that users read alerts to determine their relevance [21]. In this study, we assess \"alert dwell time\" instead, defined as the time elapsed from an interruptive alert being generated to it being closed or overridden, which more broadly quantifies the time required for clinicians to respond to interruptive alerts, whether they read them or not [22].  
To date, few studies have used alert dwell time to evaluate the efficiency of their alert system, with the majority focusing only on clinical alerts (e.g., prescribing and drug-drug interactions) [21; 22; 23]. However, APOE systems like ours generate both clinical and administrative (low or no clinical relevance) alerts [24; 25; 26; 27], with up to 80% being administrative as reported in our previous study [16]. Therefore, neglecting administrative alerts may limit the improvements that can be made to the alert system, especially in relation to alert fatigue.  
Here, we studied alert dwell times for both administrative and clinical alerts. To do so, we developed a window retrieval technique, which detects how much time the user stays in every program window while operating the computer. Using data collected by the alert log collector (ALC, a method used to collect alerts in a homegrown APOE system) [16], the specific alert dwell times for each alert could be identified. Additionally, we conducted an exploratory analysis of the impact of common alert characteristics (i.e., number of alerts and alert message length) on the alert dwell time, as well as potential differences between professional groups (physician, nurse, other).