VTaC: A  Benchmark Dataset of Ventricular Tachycardia Alarms from ICU Monitors
Authors: Li-wei H. Lehman
Benjamin E Moody
Harsh Deep
Feng Wu
Hasan Saeed
Lucas McCullum
Diane Perry
Tristan Struja
Qiao Li
Gari Clifford
Roger Mark
Venue: NeurIPS 2023 Datasets and Benchmarks Poster (2023)
Keywords: arrhythmia
false alarm
ICU
machine learning
deep learning
contrastive learning
physiological waveforms.
TL;DR: A benchmark database of over 5,000 waveform recordings from ventricular tachycardia alarms in intensive care unit patients.

Abstract: False arrhythmia alarms in intensive care units (ICUs) are a continuing problem despite considerable effort from industrial and academic algorithm developers. Of all life-threatening arrhythmias, ventricular tachycardia (VT) stands out as the most challenging arrhythmia to detect reliably. We introduce a new annotated VT alarm database, VTaC (Ventricular Tachycardia annotated alarms from ICUs)  consisting of  over 5,000 waveform recordings with VT alarms triggered by bedside monitors in the ICUs. Each VT alarm in the dataset has been labeled by at least two independent human expert annotators. The dataset encompasses data collected from ICUs in three major US hospitals and includes data from three leading bedside monitor manufacturers, providing a diverse and representative collection of alarm waveform data. Each waveform recording comprises at least two electrocardiogram (ECG) leads and one or more pulsatile waveforms, such as photoplethysmogram (PPG or PLETH) and arterial blood pressure (ABP) waveforms. We demonstrate the utility of this new benchmark dataset for the task of false arrhythmia alarm reduction, and present performance of multiple machine learning approaches, including conventional supervised machine learning, deep learning, contrastive learning and generative approaches for the task of VT false alarm reduction.

VTaC: A Benchmark Dataset of Ventricular Tachycardia Alarms from ICU Monitors
Li-wei H. Lehman
Benjamin Moody MIT
Harsh Deep MIT
Feng Wu MIT
MIT
Hasan Saeed MIT
Lucas McCullum
Diane Perry
Tristan Struja
MIT
MIT
MIT
Qiao Li
Gari Clifford
Roger G. Mark
Emory University
Emory University, Georgia Tech
MIT
Abstract
False arrhythmia alarms in intensive care units (ICUs) are continuing problem despite considerable effort from industrial and academic algorithm developers. Of all life-threatening arrhythmias, ventricular tachycardia (VT) stands out the most challenging arrhythmia to detect reliably. We introduce a new annotated VT alarm database, VTaCC (Ventricular Tachycardia annotated alarms from ICUs) consisting of over 5,000 waveform recordings with VT alarms triggered by bedside monitors in the ICUs. Each VT alarm in the dataset has been labeled by at least two independent human expert annotators. The dataset encompasses data collected from ICUs three major US hospitals and includes data from three leading bedside monitor manufacturers, providing diverse and representative collection of alarm waveform data. Each waveform recording comprises at least two electrocardiogram (ECG) leads and one more pulsatile waveforms, such as photoplethysmogrree (PPG or PLETH) and arterial blood pressure (ABP) waveforms. We demonstrate the utility of this new benchmark dataset for the task of false arrhythmia alarm reduction, and present performance multiple machine learning approaches, including conventional supervised machine learning, deep learning, contrastive learning and generative approaches for the task of VT false alarm reduction.
Introduction
Within intensive care units (ICUs), bedside monitors generate a substantial number of alarms, considerable proportion of which are false [Drew al. 2014). These false alarms not only burden clinicians with heightened cognitive strain but also have the potential to obscure genuine alarms, thereby endangering patient safety. Arrhythmia alarms, accounting for 45% of overall ICU alarms, in particular, pose a significant challenge, contributing to alarm fatigue that increases the risk of healthcare providers potentially overlooking true life-threatening events [Drw et al. 2014 Cvach 2012].
Accurate and reliable algorithms that can distinguish between true vs. false arrhythmia alarms can improve the overall effectiveness of monitoring systems ICUs. However, the development and evaluation of such algorithms rely on the availability of high-quality and representative datasets with ground-truth labels from human experts.
"Corresponding author: lilehman @ mit.edu.
37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.

---

We focus on the problem of detecting false alarms for one of the life-threatening arrhythmias, ventricular tachycardia (VT), defined as five or more ventricular beats with a heart rate higher than 100 bpm -eiittt [Clifford et al. 201 50 20T6]. Among all the life-threatening arrhythmia alarms, VT alarms are one of the most commonly occurring alarms [Drew et al. 2014 Aboukhalil et al.
2008 and false VT alarms have proven to be the most challenging to detect reliably Aboukhalil
ct al.
2008 Clifford et al. 2015a 2016 Zhou et al. 2022].
In this study, we present a new annotated VT database designed to address the challenges associated with false arrhythmia alarm reduction. The dataset consists of total of over 12,000 labeling decisions from six experts who reviewed over 5,000 waveform recordings with VT alarms triggered by patient bed-side monitors in an ICU setting. These records were sourced from multiple ICUs from three major hospitals in the United States, providing a diverse and representative collection of waveform data. Each waveform recording comprises electrocardiogram (ECG) leads and one or more pulsatile waveforms, such as photoplethysmogram (PPG or PLETH) and arterial blood pressure (ABP) waveforms.
One important aspect of our dataset is the inclusion of labeling decisions from multiple independent human annotators. This approach helps to ensure the robustness of the dataset by reducing the potential impact of individual subjectivity and variability among annotators. The availability of such a comprehensive and annotated dataset enables researchers and practitioners to evaluate and compare the performance of various machine learning approaches for false arrhythmia alarm reduction. We demonstrate the utility of our benchmark dataset by evaluating multiple machine learning approaches, including conventional supervised machine learning, deep learning, contrastive learning, and generative approaches.
By addressing the challenges of false arrhythmia alarms and providing a benchmark dataset, this research contributes the ongoing efforts to improve patient safety, reduce alarm fatigue, and enhance the efficiency of monitoring systems in ICUs. The introduced dataset VT&C, to the best of our knowledge, represents the largest open-access, multi-center VT alarm dataset, meticulously annotated by highly skilled clinical experts. It provides valuable resource for developing algorithms reduce false VT alarms, longstanding and unsolved challenge in ICUs. Data will be available on PhysioNet at potphysioneetor/oonnett//e
Background and Machine Learning Challenges
In this section, we outline challenges in developing machine learning approaches for false VT alarm reduction.
Artifact or noise. Erroneous triggers of VT alarms typically occur due noise and ECG-related artifacts, such as electrode movements, poor electrode contact, or other technical reasons. Ventricular arrhythmias characterized by distortion of beat morphology with a broader QRS complex. However, a challenge to be overcome is that noise and artifacts can exhibit morphologies similar to abnormal QRS complexes, almost indistinguishable from the periodic anomalies from a true VT episode [Clifford et al. 2015a].
Other rhythms with similar appearance. VT and other rhythms, such as atrial fibrillation (A-Fib) and aberrantly conducted supra ventricular tachycardia (SVT) may exhibit similar characteristics on an ECG, making it difficult to distinguish them [Littmann et al. 2019]. For example, both VT and SVT with aberrant conduction can have wide QRS complexes on the ECG. In some cases, patients may have multiple simultaneous rhythms, further complicating the interpretation. For instance, VT may coexist with A-Fib or SVT, making it challenging to identify and differentiate each rhythm component.
Individual differences and variable morphology. VT can have variable morphologies, which means it can appear differently in different patients even within the same patient. This variability can make it challenging to identify VT solely based on ECG characteristics immediately prior to the alarm onset. Inspection of the same individual patient's 's prior ECG characteristics at baseline may be necessary.
Long sequence with complex patterns and Missing Channels. In the false alarm reduction problem, one major challenge stems from the long time-series sequence and the sparse availability of labeled data [Clifford et al. 2015a Kiyassch et al. 202 Zhou et al. 2022 At sampling frequency of

---

250 Hz, a 5-minute segment of a single channel of ECG contains 75000 samples. The label of each record, on the other hand, just TRUE and FALSE, providing limited supervised information train a deep learning model. Additionally, missing channels pose a significant challenge for false alarm reduction.
Sparse availability of high-quality labeled data. While machine learning and deep learning have made significant advances in many domains, including image and voice analyses, the application of deep learning in physiological waveform analysis has had limited success, partly due to limited availability of high-quality labeled data. The best performing approaches for false arrhythmia alarm detection in the 2015 PhysioNet Challenge rely on a combination of expert -peert-define rule- based logic analysis [Plesinger al.) 2011] and simple machine-learning models, while, in comparison. conventional deep learning approaches generally under-performed in the 2015 PhysioNet Challenge (Clifford et al. 2015a
3 Related Works
3.1 Related Works in Annotated Arrhythmia Datasets
Several efforts have been made in the development and curation of annotated arrhythmia datasets, which serve as valuable resources for the development and evaluation of algorithms aimed at ar- rhythmia detection and classification. These datasets contribute to advancing the field of cardiac monitoring and have facilitated the progress of machine learning techniques in arrhythmia analysis. In this section, we highlight some notable related works in the domain of annotated arrhythmia datasets.
One seminal dataset widely used in arrhythmia research is the MIT-BIH Arrhythmia Database (Moody and Mark [. It comprises 48 ECG records (each slightly more than 30 minutes) obtained from long-term Holter recordings. These records include a mix of randomly selected and specifically chosen cases, showcasing broad spectrum of typical clinical waveforms, artifacts, and complex arrhythmias, both ventricular and supraventricular. The dataset has been instrumental in benchmarking different algorithms for arrhythmia classification and contributed to the evaluation of signal processing and machine learning techniques. The MIT-BIH Arrhythmia Database does not include annotations for arrhythmia alarms generated by commercial bed-side monitors.
The 2015 PhysioNet Challenge event [Clifford et al., 2015b 2016) focused on five types of life- threatening arrhythmias, including ventricular tachycardia, asystole, extreme bradycardia, extreme tachycardia, and ventricular fibrillationffluttr. The goal of the Challenge was to reduce the number of false alarms, while avoiding suppression of true alarms. The Challenge consists of two events: (1) real-time classification using only data up to the alarm onset; (2) retrospective analysis in which the contestants allowed use the 10-second data after the alarm onset for classification.
The 2017 PhysioNet/CinC Challenge [Clifford et al. 2017 is a collection of single-lead ECG recordings (each between 30 to 60 seconds) from wearable devices. The primary objective of this dataset is to facilitate the development of algorithms for the classification of cardiac rhythms into four main categories: normal sinus rhythm, atrial fibrillation (AF), an alternative rhythm, noisy recordings that cannot be classified.
Most recently, [Pelter et al. 2023 introduced the UCSF VT dataset comprising 18,455 annotated VT alarms from 858 ICU patients at UCSF. Their research differs from ours in several key aspects. The UCSF VT dataset was exclusively collected from a single hospital, utilizing patient monitors from a single commercial bed-side monitoring vendor. In contrast, VTaC comprises a multi-center collection, incorporating data from monitors manufactured by three major vendors of commercial monitoring systems. As a result, VTaCC provides a more diverse and comprehensive representation. which is crucial for the development of VT false alarm reduction algorithms. Furthermore, the UCSF VT dataset, as indicated by [Pelter et al., 2023), J, is not shared or made publicly available to the research community. Conversely, VTaCC is open- resource, freely accessible to the research community.
3.2 Machine Learning Approaches to False Arrhythmia Alarm Reduction
Multiple algorithms and machine learning approaches have been proposed for arrhythmia analyses and false alarm reduction. Conventional algorithms [Aboukhalil et al. 2008 Li and Clifford 2012)
3

---

and the best performing entries in the 2015 PhysioNet Challenge for false arrhythmia alarm reduction [F Plesinger et al. 2016 Kalidas and Tamil 2013 Clifford et al. 2013a| rely on signal processing and expert-defined rule-based logic to analyze physiological signals, or using manually engineered features as inputs for machine learning classifiers. Due to the sparse availability of labeled data, much of the recent machine learning approaches for false alarm reduction focus on techniques that
enable label-efficient learning, using e.g. multi-task learning [Schwab et al. 2018), supervised representation learning [Lehman et al. 2018). and self-supervised or contrastive learning [Kiasseh ctt al. 2021b Zhou et al.] [022] Lehman et al.] [2018] focus on VT false alarm detection, and
use FFT-transformatioo of individual ECG beats for scalable learning. Kiyasseh et al. propose family of supf-super pretraining mechanisms based on contrastive learning for physiological
signals (giyass(Kyyssech et al., 20221b. Zhou al. [2022] proposed a contrastive learning framework to
reduce five types of life-threatening arrhythmia alarms. More recently. WW et al. [2023] proposed
conditional generative modeling approach for the task of VT false alarm reduction, and showed
promising results from a diffusion contrastive learning model using an annotated arrhythmia dataset
(Aboukhalil et al. 2008 from the MIMIC II database (Saeed et al.) 2011
4 Data Collection and Annotation Methodology
In this section, we briefly outline our data collection methodology and the annotation process. For detailed description, please refer the Appendix.
4.1 Data
We extracted and compiled a total of 18,465 waveform VT alarm events, corresponding to 2,376 unique patient waveform records from bed-side monitors from three leading commercial vendors. These records were sourced from multiple ICUs from three major hospitals in the United States, providing diverse and representative collection of waveform data. Each waveform recording in our dataset consists of a 10-minute segment that encompasses the onset of the ventricular tachycardia (VT) alarm. This segment includes 5 minutes of waveform data preceding the alarm onset and 5 minutes following it. To maintain diversity, we randomly selected maximum of five alarm events from any particular patient waveform record, yielding a total of 5,742 events for annotation. This ensures that a reasonable number of events were sampled from different patient records, preventing an over-representation of events from any single patient record. Waveform records were de-identified to ensure the anonymization of all identifiable information, including patient names, dates, and medical record numbers. All signals were resampled to 250 Hz, and all signal labels were adjusted to match the nomenclature used in the PhysioNet Challenge 2015 database.
4.2 Annotation Process
Following the PhysioNet Challenge 2015, a VT episode is defined as five or more consecutive ventricular beats with heart rate higher than 100 beats pettsppe-mmnute (bpm) [Clifford et al. 2015a]. Each event was reviewed and labeled by at least two annotators independently. For our task, annotators were given the options of "True" for when they believe the alarm was correct, "False" for when they believe the alarm was incorrect, "Uncertain" for when they were unsure which annotation to assign, "Reject" for when the alarm was unreadable due to noise, artifacts, or other reasons. In order to reconcile conflicts between two annotator decisions, an adjudication process was implemented to resolve the conflicts. These disagreements were resolved either through direct one-on-one discussions between the annotators involved or by an adjudicator's 's vote to break the tie.
A total of 5.742 events were annotated by at least two independent annotators. Two independent annotators reached unanimous decisions on 4.534 (78.96%) events, whereas 21.04% (N=1,208) of the events received conflicting labeling decisions by two human annotators, Among the events with conflicting decisions, 816 (66.55%) were adjudicated. After removing 392 un-adjudicated events, total of 5,350 alarm events received final labeling decisions. See Table 5 in Appendix for the breakdown of the 5,742 events based on the annotation decisions.
Data Composition by Labeling Decisions Table T lists summary of alarms categorized by annotation decisions among the 5,350 alarms with final labeling decisions. A total of 102 (1.91% %) and 211 (3.94%) events received Uncertain and Reject decisions respectively. After excluding "Rejected"

---

Table Composition of the alarms categorized by labeling decisions.
Unanimous
Adjudicated
Total
Total Alarm Events
4,534
816
5,350
Uncertain
19
83
102
Reject
120
91
211
True (T)
1,222
219
1,441
False (F)
3,173
423
3,596
Final (T/F) Events Included
4,395
642
5,037
Percent True Alarms
27.80%
34.11%
28.61%
and "Uncertain" events, the final dataset used for modeling contains 5,037 events, among which 4,395 events had unanimous annotation decisions, and the final labeling decisions of the 642 adjudicated events were based on the final decisions from the adjudicators. Figure 2 in the Appendix A illustrates the VT&CC data collection and event sampling process.
Annotation Team The annotation team consists of six annotators, including highly-experienced board certified cardiac arrhythmia technician, and leading arrhythmia analysis expert physician who built the MIT-BIH Arrhythmia database. The team also includes three clinicians, and one biomedical signal processing engineer specializing in arrhythmia. We describe below expertise of three of the annotators who had contributed the most number of annotations. Annotator 1: Board certified cardiac arrhythmia technician with over 40 years of experience in interpreting and annotatin manotatingarrhythmia from ECG recordings. Annotator 2: Developed the MIT-BIH Arrhythmia database; physician in internal medicine with decades of experience real-time arrhythmia analyses and clinical interpretation of Holter recordings. Annotator 3: Endocrinologist and Internist with extensive experience in Emergency Medicine and cardiovascular intensive care. Additional details of the annotation team can be found in the Appendix.
Example True False VT Alarms in VTaC In Figure VT alarms that have been labeled as a true vs. a false alarm respectively. In each figure, waveform recordings from the final 10-seconds prior the VT alarm onset shown.
show waveform plots for two example
(a) Truc VT alarm
False VT alarm
Figure Example true vs. false VT alarms. Each plot shows data in the 10-second interval immediately prior to the VT alarm onset. The alarm onset is marked with a vertical red-ling at time 0. Figure (b) shows an example false VT alarm the event corresponds an episode of atrial fibrillation with rela-rd aberration instead of ventricular tachycardia.
4.3 Comparison with Other Annotated VT Databases
Table 2 we compare our newly developed VT alarm dataset with existing annotated VT databases on PhysioNet, including the training and hidden test sets from the PhysioNet Challenge 2015 [Clifford et al., [2015a], and an annotated arrhythmia alarm dataset from MIMIC [Saeed al.]
201 Goldberger et al. 2000) as reported in Aboukhalil al. 2008 We compare these datasets in

---

terms of annotated alarm events, the pereentage of true alarms, well as percentage of alarm events with ECG, ABP, and PLETH. We note that VTaC a federated database sourced from waveform recordings collected from multiple commercial bed-side monitors (as such, multi-vendor) across multiple ICUs three US hospitals. We also note that the current dataset provides fine-grained rre-adjudication labeling decisions from human experts, which are not included in prior datasets. As we randomly select VT alarm events from available recordings in constructing our dataset, the federated VT alarm dataset contains a false alarm rate that closely reflects the actual false alarm rate real-world ICU setting [Drww et al. 2004
Table Comparison of Annotated VT Databases on PhysioNet. Pre-Adj Decisions indicate whether labeling decisions before the adjudication processes are available.
Challenge' 15 Train
Challenge' 15 Test
MIMIC II
VTaC
N (# VT alarms) % True
341 26.67%
221
1,900 53.40%
5,037
20.36%
28.61%
Open Access*
Y
Y
Y
Multi-Vendor
Y
Y
N
Y
Pre-Adj. Decisions
N
N
Y
ECG (% events)
100%
100%
100%
100%
ABP (% events)
54%
57%
100%
36%
PLETH (% events)
83%
81%
0%
91%
5 Models & Evaluation
We demonstrate the utility of this new benchmark dataset for the task of false VT alarm reduction, and present performance of multiple machine learning approaches in both real-time and retrospective settings following PhysioNet Challenge 2015 [Clifford et al., (2015a]. , In the real-time setting, the goal is to reduce false alarms in "real-time" without using information after the alarm onset. In the retrospective setting, algorithms are allowed utilize data within brief time interval following the alarm, such as the 30-second interval specified in the PhysioNet Challenge 2015. However, in the context of this study, the retrospective setting incorporates data up seconds after the alarm.
5.1 Dataset
The final dataset used for modeling consists of 5,037 annotated VT alarms. The channels used for modeling included ECG, ABP, and PLETH (or PPG). We employed an 80-10-10 split for the train, validation, and test sets. The splits were performed at the patient record level rather than the individual alarm events level to ensure that events belonging to the same patient record are not distributed across the train, validation, and test sets. Table 3 displays the distribution true alarms and signal types across the train, validation, and test set.
For the real-time event, 10-second window of waveform data immediately prior to the alarm onset were used as input to all of our models, with the exception that the generative models used smaller window data due to computational constraints. The retrospective events used 15-second window of waveform data to include the additional 5-second window of data immediately after the alarm onset. For each VT event, the model was fed with total of four channels of waveform data, comprising two ECG leads, one ABP, and one PPG, all at frequency of 250 Hz.
Data Preprocessing For ECG, we perform the following filtering: 1) a high-pass filter with 1-Hz cutoff frequency to suppress residual baseline wander; 2) a second-order 30 Hz Butterworth low-pass filter to reduce high frequency noise; and 3) a notch filter to eliminate power line interference. For PPG signal, we utilize high-pass filter with stopband frequency of 0.3 Hz and passband frequency of 0.5 Hz, along with low-pass filter with a passband frequency of 8 Hz. We conduct z-normalization for all signals by utilizing the mean and standard deviation of each individual signal segment before feeding them into the model. cases where signals are missing, we impute them with value of O.
Hz and a stopband frequency

---

Table 3: Composition of the train, validation and test splits. Unanimous decisions had two annotators who were both in agreement.
Overall
Train
Validation
Test
Alarms (count)
5,037
4,060
495
482
Records (count)
2,260
1,808
226
226
True Alarms (count)
1,441
1,163
141
137
% True
28.61%
28.65%
28.90%
28.48%
ECG (% events)
100%
100%
100%
100%
ABP (% events)
35.7%
35.6%
36.5%
35.6%
PLETH (% events)
90.6%
90.42%
89.83%
92.32%
Adjudicated
642
542
46
54
Unanimous
4,395
3,518
449
438
5.2
Models
Models and Baselines. We compare the performance of the following baseline and models. 1) Rule-Based Method: For the rule-based approach, we used the implementation from Plesinger [2013). a winning entry in the PhysioNet 2015 challenge for false arrhythmia alarm reduction.
ct al.
Plesinger et al. 2013) test each channel in the record for regular heart activity using the QRS detection
and derived R-R information [Plesinger et al.) 2015). 2) MLP: We apply the multi-layer perceptron as feature extractor of the input waveform and then use dense layer classify. 3) SAE (supervised autoencoder): We use autoencoder to learn a lower dimensional representation from 10-second waveform signals, and simultaneously minimize the reconstruction loss from autoencoder and the cross entropy loss as described in [Lehman et al. 2018). One notable difference from [Lehman et al. 2018 is that we utilize the entire 10-second waveform segment as the model input, without heartbeat detection for learning beat-by-beat representations. 4) Transformers: We utilize the Transformer encoder (Vaswani et al. 2017] as the feature extractor of the input waveform. 5) CNN: We use light-weight I-D convolutional neural network as feature extractor (Zhou et al. [022]. 6) CNN+CL: We use a light-weight 1-D convolutional neural network as a feature extractor and augment it with contrastive learning [Zhou et al. 2022]. 7) FCN: We use fully-connected convolutional network as the feature extractor of the input waveform [Zhou al. 2022]. 8) FCN+CL: We use a fully- connected convolutional network as the feature extractor of the input waveform and augment it with contrastive learning [hhou et al. 2022]. 9) BeatGAN: BeatGAN [Zhou et al. 2019) is an GAN-based unsupervised anomaly detection algorithm for time series data. 10) TanoGAN: TanoGAN (Bashar and Nayak 2020 is a novel GaN-based unsupervised method for detecting anomalies when a small number of data points available. 11) Diffusion+CL: we use diffusion model reconstruct trajectories of physiological signal and classify alarms based on distances between the reconstructed and the actual trajectories [Wu et al. 2023).
5.3 Experiments
Training
For the supervised approaches, including their variants with contrastive learning, we conducted model training for a maximum of 500 epochs. For the real-time events, hyperparameters were chosen through a grid search, and the Adam optimizer [Kingma and Ba 2014) with a weight decay of 0.005 was employed to minimize both binary cross-entropy (BCE) loss and the discriminative constraint. Hyperparameter tuning was carried out, and a comprehensive description settings for all models can be found in the Appendix. For each machine learning model, we identified the hyperparameter setting with the best validation performance and proceeded to train the model with that setting using 10 different random seeds. We report the mean and standard deviation from the 5 random seeds with best validation performance. Model selection was based on the best Challenge Score on the validation set.
the hyperparameter
For retrospective events, we used the optimal real-time event, and proceeded to train the model with that setting using 10 different random seeds.
setting from the grid search of the
7

---

Table 4: Real-Time event performance from the VTaC database. Mean and standard deviation from seeds shown. Top- pop-performing models for each specific metric highlighted in bold. CL=Contrastive Learning.
Methods
TPR
TNR
PPV
F1
Score
AUC
Rule-based
0.942
0.629
0,502
0.655
67.32
MLP
0.597
0.691
00441
0.502
45.58=1.10
0.706=0.0s
SAF
0.848 ozz8
0.790=e0032
0.617 0.0s
0.713
68. 18.77
0.896=0000
Supervised
Transformer
0.837+0.099
0.707
0.535 +0.04
0.651 +0030
62.73+2278
0.852+0.00
CNN
0.928+aa.0
0.782
0.629
:0.09
0,750++001
16.17
0,936=0.009
FCN
0.920+00025
0.855 +0.018
0.717 +00004
0.805+o011
80.08
0.949
CNN+CL
0.930120016
0.676 0.823
0.783:10003
79.07=0.99
0,943
Supervised+CL
FCN+CL
0.931
0.81 10000
0.666:20048
0.775
=8.4
0,932
BeatGAN
0.597
0.597
0.373
0.455
41.02=271
0.597+a 028
Generative
TAnoGAN
0..04
0.421 100027
0,524:10008
47.61
0.657 20.87
Diffusion+CL
0.85310054
0.517:20040
0.412:1001
0.555
52.51
0.685
Evaluation The evaluation metrics for false alarm reduction are true positive rate (TPR), true negative rate (TNR), positive predictive value (PPV), F1-score, and area under the ROC (receiver operating characteristic) curve (AUC). The PhysioNet Challenge 2015 also provides an official scoring mechanism for evaluating [Clifford al. 2015a]. J. It is defined score (TP + TN)/ TN + FP + 5 * F), where 7P is true positives, FP is false positives, FN is false negatives, and TN is true negatives. The Challenge Score focuses more the TPR value, since mistakenly classifying a true alarm as false results in significantly more severe consequences. The PPV, F1, TPR, TNR and Challenge Score determined based on threshold value of predicted probability that maximizes the validation Challenge Score as cut-off value to determine true vs. false alarms in the test set.
Performance in Real-Time VT Alarm Classification
In this section, present the experimental results of eleven models for the real-time event as outlined in Table Results for the retrospective event are presented in the Appendix Table 8 The winning entry of the 2015 PhysioNet challenge using Rule-based approach serves reference point for benchmarking, achieving Challenge Score of 67.32 in the real-time event.
Our findings indicate that both FCN and CNN, along with their contrastive learning variants, demon- strated superior performance when compared to other models. Notably, FCN emerged as the top performer, achieving the highest Challenge Score (80.08+2.46) and AUC (0.949=0.006), surpassing other models. Contrastive learning enhanced the performance of 1-D CNN, as evidenced by a notable improvement in the Challenge Score from 76.17 to 79.07. However, this performance boost was not observed in the case of FCN.
Our study reveals distinctive findings when contrasted with Zhou et al. [2022] In their investigation, the application of FCN considerably smaller dataset from the 2015 PhysioNet Challenge resulted in significantly inferior performance compared to the lightweight 1-D CNN and CNN+CL. In contrast, our analysis, conducted on the VTaCC dataset, demonstrated that FCN outperformed 1-D CNN in terms of AUCs and Challenge Scores. This difference in outcomes could potentially be attributed to the common observation that increasing the size of the labeled datasct often leads to improved performance in more complex models.
Generative approaches that previously demonstrated superior performance [Wu et 2023 using the MIMIC II annotated arrhythmia dataset [Saeed et al. 2011 Aboukhalil et al.) 2008 faced challenges when applied to the VTaCC dataset. In particular, the diffusion model with contrastive learning [Wu] et al. 2023] outperformed other baselines when applied to the MIMIC dataset but under- rederrperformed with the current VTaCC dataset. The performance gap can be attributed to several potential factors. Firstly, the imbalanced label distribution in VTaC,, with relatively lower rate of true alarms at 28.6%, in comparison to over 50% true alarms in the MIMIC II dataset, presents a potential limitation. particularly for generative approaches proposed in [u et al. 2023], which depended on training

---

with abundant samples of true positives. Secondly, less than 40% of alarm records in VTaC included arterial blood pressure (ABP) waveforms, in contrast to the MIMIC annotated arrhythmia dataset, which had a higher percentage of ABP records due to its specific sampling criteria. Finally, the MIMIC II arrhythmia dataset is single-center, relatively homogeneous dataset collected from patient monitors of a single monitoring system, whereas VTaCC is a multi-center dataset.
Discussion
Previous algorithm development in false arrhythmia reduction has been hampered by the use of small, single-center or relatively homogeneous datasets. This limitation hinders their generalizability and real-world applicability. The introduction of this large-scale annotated VT alarm dataset provides a valuable opportunity address this challenge. The VTaC dataset presented in this study, encompasses data collected from ICUs in three major US hospitals and incorporates data from three major bedside monitor manufacturers. By encompassing data from diverse sources, this dataset enables the evaluation and refinement of algorithms in a broader context, across a more diverse monitoring devices and clinical settings. Furthermore, the random selection of VT alarm events from the available data for annotation ensures that the dataset's ' distribution and the proportion of true alarms closely resemble those observed in real ICU settings. This approach enhances the dataset's representativeness and allows for a more accurate evaluation of the models' performance in practical clinical scenarios.
Safety & Ethical Discussion We raise several safety and ethical considerations. Firstly, as the dataset contains waveform recordings from real patients, we acknowledge the importance of patient privacy and data protection. To ensure privacy, we have de-identified the data by removing any personally identifiable information. Secondly, the process of labeling the dataset involved clinical experts reviewing waveform recordings and making labeling decisions. We recognize the potential influence of individual subjectivity and variability among annotators. To mitigate this, we have sought ensure robustness by obtaining labeling decisions from multiple experts for each alarm event. This approach helps to capture diverse perspectives and minimize potential bias in the annotations. We have made efforts to curate a diverse and representative dataset, but there might still be underlying biases that could impact the performance of the machine learning models.
Limitations & Future Work The newly introduced annotated VT alarm database is specifically designed to tackle the challenges related to reducing false VT alarms. The VT alarm events are randomly sampled, and thus the selected VT events cannot be viewed of all VT alarm events for an individual patient. As such, the database is not designed for long-term forecasting of arrhythmia episode onset. Another limitation of our study is the absence of detailed clinical information accompanying the waveform recordings. We leave the collection of matched clinical data for future research endeavors. Finally, while the biases from expert annotators in this dataset is greatly reduced from having multiple annotators, we acknowledge that biases can still be present two or more annotators have the same bias.
comprehensive collection
In our analyses, we have opted to construct our machine learning models by utilizing data from up to 10 seconds before the alarm onset (except the contrastive learning based approaches, which sample data from prior to 10 seconds before the alarm onset). Additionally, this study involved the inclusion of a maximum of four channels of available waveforms per event to formulate our models. For future investigations, we aim to explore models capable of more effectively handling higher-dimensional multi-channel waveform data encompassing longer sequences. This will enable us to fully leverage data from earlier time points and identify patterns across multiple channels of physiological waveforms.
8 Conclusion
Ventricular tachycardia is the most challenging arrhythmia to detect reliably, and remains continuing problem in ICUS despite decades of effort from industrial and academic algorithm developers. We present new annotated VT database to address the challenges associated with reducing false VT alarms. We conducted comprehensive benchmarking of various machine learning models utilizing this annotated VT dataset. By providing this resource as open-access database, freely available to the research community, we aim to foster collaboration, facilitate benchmarking. and encourages the advancement of research efforts in the field of arrhythmia alarm analysis.

---

ACKNOWLEDGMENTS
This study was funded by NIH grant RO1EB030362. L. Lehman was in part funded by MIT-IBM Watson AI Lab. The authors would like to express their gratitude to Dr. Eric Gottlieb and Dr. Xavier Borrat Frigola for their contributions the annotations. Additionally, we extend our appreciation to Shaherul Haque and Anthony Baez at MIT for their technical assistance.
References
Barbara J Drew, Patricia Harris, Jessica K Zègre-Hemsey, Tina Mammone, Daniel Schindler, Rebeca Salas-Boni, Yong Bai, Adelita Tinoco, Quan Ding, and Xiao Hu. Insights into the problem of alarm fatigue with physiologic monitor devices: comprehensive observational study of consecutive intensive care unit patients. PloS one, (11) 10274, 2014.
Maria Cvach. Monitor alarm fatigue: an integrative review. Biomedical instrumentation & ttccnnoogy, 46(4):268-277 2012.
Gari Clifford, Ikaro Silva, Benjamin Moody, Qiao Li, Danesh Kella, Abdullah Shahin, Tristan Kooistra, Diane Perry, and Roger G Mark. The physionel/oopuuting in cardiology challenge 2015: reducing false arrhythmia alarms the icu. 2015 Computing in Cardiology Conference (CinC), pages 273-276. IEEE, 2015a.
GD Clifford, I Silva, B Moody, Qi, D Kella, A Chahin, Kooistra T, D Perry, and RG. Mark. False alarm reduction in critical care. Physiological Measurements, 37(8):E5-E23, 2016.
Anton Aboukhalil, Larry Nielsen, Mohammed Saeed, Roger G Mark, and Gari D Clifford. Reducing false alarm rates for critical arrhythmias using the arterial blood pressure waveform. Journal of biomedical informatics, 4 41344442--45, 2008
Yuerong Zhou, Guoshuai Zhao, Jun Li, Gan Sun, Xueming Qian, Benjamin Moody, Roger G Mark, and Li-wei H. Lehman. A contrastive learning approach for ICU false arrhythmia alarm reduction. Nature Scientific Reports, 2022.
Littmann, EG Olson, and MA Gibbs. Initial evaluation and management of wide-complex tachycardia: A simplified and practical approach. Am J Emerg Med, 37(7))1340-113 2019.
D. Kiyasseh, T. Zhu, and D. A. Clifton. Clocs: Contrastive learning of cardiac signals across space, time, and patients. In Proceedings of the 38th International Conference on Machine Learning, page 5606-5615, 202.a.
Filip Plesinger, Petr Klimes, Josef Halamek, and Pavel Jurak. False alarms in intensive care unit monitors: detection of life-threatening arrhythmias using elementary algebra, descriptive statistics and fuzzy logic. In 2015 Computing Cardiology Conference (CinC), pages 281-284. IEEE, 2015.
GB Moody and RG Mark. The impact of the MIT-BIH arrhythmia database. IEEE Eng in Med and Biol, 20-3):45-50, 2001.
Gari Clifford, Ikaro Silva, Benjamin Moody, Qiao Li, Danesh Kella, Abdullah Shahin, Tristan Kooistra, Diane Perry, and Roger Mark. The physionet/computing in cardiology physionet challenge 2015: Reducing false arrhythmia alarms in the icu. In Proceedings of Computing in Cardiology, 2015b.
Gari D Clifford, Chengyu Liu, Benjamin Moody, H Lehman Li-wei, Ikaro Silva, Qiao Li, AE John- son, and Roger G Mark. Af classification from short single lead ecg recording: The phys- ionet/computing in cardiology challenge 2017. In 2017 Computing in Cardiology (CinC), pages 1-4. IEEE, 2017.
MM Pelter, MG Carey, S Al-Zaiti, Zegre-Hemmeey C Sommargren, Isola, Prasad, Mortara, and F Badilini. An annotated ventricular tachycardia (vt) alarm database: Toward a uniform standard for optimizing automated identification in hospitalized patients. Ann Noninvasive Electrocardiol, 28(4), July 2023.
10

---

Qiao Li and Gari D Clifford. Signal quality and data fusion for false alarm reduction in the intensive care unit. Electrocardiol, 45(6):596 603, Nov-Dec 2012.
FF Plesinger, P Klimes, Halamek, and Jurak. Taming of the monitors: reducing false alarms in intensive care units. Physiological Measurements, 57:1313-1325. 2016.
Vignesh Kalidas and Lakshman S Tamil. Enhancing accuracy of arrhythmia classification by combining logical and machine learning techniques. In 2015 Computing in Cardiology Conference (CinC), pages 733-736. IEEE, 2015.
Patrick Schwab, Emanuela Keller, Carl Muroi, David J. Mack, Christian Strässle, and Walter Karlen. Not to cry wolf: Distantly supervised multitask learning in critical care. CoRR, abs/ 1802.05027, 2018. URL URL http://arxxx hwtpziii.rrr//bba. wwrgg/ba/1880 1atp:/8802.0000 05027
Eric Lehman, Rahul G Krishnan, Xiaopeng Zhao, Roger Mark, and Li-wei Lehman. Repre- sentation learning approaches to detect false arrhythmia alarms from ecg dynamics. In Machine learning for healthcare conference, pages 571-586. PMLR, 2018.
Dani Kiyasseh, Tingting Zhu, and David A Clifton. Clocs: Contrastive learning of cardiac signals across space, time, and patients. In International Conference Machine Learning, pages 5606- 5615. PMLR, 2021b.
Feng Wu, Guoshuai Zhao, Xueming Qian, and Li-wei Lehman. A diffusion model with contrastive learning for icu false arrhythmia alarm reduction. In Proceedings of IJCAI, 2023.
M Saeed, M Villarroel, AT Reisner, G Clifford, L Lehman, G Moody, Heldt, TH Kyaw, B Moody, and RG Mark. Multiparameter intelligent monitoring in intensive care ii (MIMIC-II): A public- access intensive care unit database. Crit Care Med, 39(5):952 960, 2011.
A. L. Goldberger, L. A. N. Amaral, L. Glass, J. M. Hausdorff, P. Ch. Ivanov, R. G. Mark, J. Mietus, G. B. Moody, C.-K. Peng, and H. E. Stanley. Physiobank, physiotoolkit, and physionet: Components of new research resource for complex physiologic signals. Cire, 101(23)ee15--2220 2000. [Circulation Electronic Pages;
Barbara J Drew, Robert M Califf, Marjorie Funk, Elizabeth S Kaufman, Mitchell W Krucoff, Michael M Laks, Peter W Macfarlane, Claire Sommargren, Steven Swiryn, and George F Van Hare. Practice standards for electrocardiograpiie monitoring hospital settings: an american heart association scientific statement from the councils on cardiovascular nursing, clinical cardiology, and cardiovascular disease in the young: endorsed by the international society of computerized electrocardiology and the american association of critical-care nurses. Circulation, 110(17): 2721-2746, 2004.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017
Bin Zhou, Shenghua Liu, Bryan Hooi, Xueqi Cheng, and Jing Ye. Beatgan: Anomalous rhythm detection using adversarially generated time series. In IJCAI, volume 2019, pages 4433 44399, 2019.
Md Abul Bashar and Richi Nayak. Tanogan: Time series anomaly detection with generative adversarial networks. In 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pages 1778-1785, 2020. doi: 10. 0009///S114883..000.900000
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 14Xiv:1412.694 2014.
Lucas McCullum, Hasan Saeed, Benjamin Moody, Gottlieb Eric Perry, Diane, Tom Pollard, Xavier Borrat Frigola, Qiao Li, Gari Clifford, and Li-wei Lehman. Physiotag: An open-source platform for collaborative annotation physiological waveforms. In Proceedings of Computing in Cardiology. 2023.
11

---

Appendix A Additional VT Alarm Dataset Description
This dataset is compilation of patient waveforms sourced from multiple institutions, and the process of collecting this data has received approval from the respective Institutional Review Boards (IRBs) of each participating institution. Requirement for individual patient consent was waived because the project did not impact clinical care and all protected health information was de-identified. It is important emphasize that the alarms and waveform records included in this dataset encompass data collected up until 2022 and do not overlap with those of the PhysioNet Challenge 2015.
Continuous multi- channel waveform recordings were collected from ICUs of three major US hospitals. The dataset consists of total of 20,158 continuous waveform recordings collected between 2013 and 2022, and represents data collected from three major bedside monitor manufacturers. The average duration of the patient waveform recording is 4.5 days. We randomly extracted recordings with least one VT alarm. The resulting dataset consists of total of 18,465 waveform alarm events, sourced from a randomly selected 2,376 unique patient waveform recordings (with at least one VT alarm). The annotated VT alarm events (5,742 alarms from 2,376 patient records) were randomly drawn from this collection, with the constraint that no more than a maximum of five VT alarms can be sampled from the same patient waveform records ensure diversity. After removing confirmed iiieet//nncertin events (by at least two independent annotators), 5,037 alarm events (from 2,260 unique patient records) remain for algorithm training and evaluation. Table ddisplaysss the number of events receiving unanimous vs conflicting decisions respectively. Unanimous category represents events with decisions from two independent annotators agreement. Figure illustrates the VTaCC data collection and annotation process.
Table 5: Annotated VT alarm events categorized by annotation agreement.
Conflicts (21%)
Unanimous (79%)
Total
Adjudicated
Un-Adjudicated
Alarm Events Labeling Decisions
4,534 4,534x2
816
392
5,742
816x3
392x2
12,300
Events with Final Decisions
4,534
816
5,350
20.158 continuous waveform records from major US hospitals
2,376 continuous waveform records with 18,465 VT alarms
Annolated 5,742 VT from 2,376 waveform records
5,350 VT alamms with unanimous adjudicated labeling decisions
Final Labeled Data
5,037 VT alarms from 2,260 waveform records
(Average duration record: 4.5 days)
Random selection from records with VT alarms
Random selection alarms per record
Exclude unadjudicnted alarm events
Exclude alarms with Reject Uncertaai decisions
Figure 2: VTaC data collection, event sampling and annotation pipeline.
ECG and pulsatile signals were recorded by the bedside monitor at various sampling frequencies ranging from 100 Hz to 250 Hz. ECG signals were recorded minimum of 240 Hz. ABP and PPG were all sampled at the same rate which would be either 100 125 240 Hz. All of the signals were resampled to 250 Hz for consistency.
Among the annotated 5,037 VT events, all events have at least two channels of ECG, and 36% and 91% of those events also had ABP and PPG respectively. The median number of ECG leads per waveform recording is 3. In 6 we show the summary statistics of the number of ECG leads per waveform recording in the dataset.
Appendix B Annotation Tool and Team
Annotation was performed using an open-source annotation platform, PhysioTag, which enables experts to collaboratively annotate physiological waveform records using standard web browser.
12

---

Table 6: Summary statisties for the number of ECG leads per waveform recording in the VTaC dataset.
ECG Leads
Average Median
3.4
25% percentile
75% percentile
4
Table 7: Real-Time event: Test set performance of the model that exhibited the best validation performance among the random seeds. Top-performing models for each specific metric highlighted in bold. CL=Contrastive Learning.
Methods
TPR
TNR
PPV
F1
Score
AUC
Rule-based
0.942
0.629
0.502
0.655
67.32
MLP
0.708
0.562
0.391
0.503
45.33
0.703
SAE
0.847
0.803
0.630
0.723
69.43
0.905
Supervised
Transformer
0.796
0.719
0.529
0.636
60.10
0.844
CNN
0.927
0.788
0.635
0.754
76.43
0.936
FCN
0.949
0.835
0.695
0.802
81.96
0.948
CNN+CL
0.912
0.835
0.687
0.784
77.92
0.946
Supervised CL
FCN+CL
0.912
0.829
0.680
0.779
77.55
0.922
BeatGAN
0.803
0.473
0.377
0.513
46.27
0.638
Generative
TAnoGAN
0.723
0.580
0.406
0.520
47.16
0.651
Diffusion+CL.
0.883
0.499
0.412
0.561
53.66
0.691
The software is freely available on PhysioNet (https://physiores) httppe///bpaaaaatttt..oo///rrttp
Events are assigned annotators dynamically. When new annotator joins the project, they are assigned 100 randomly-selecter events that have not yet been annotated. After they have finished reviewing the assigned events, they can assign themselves a new batch of events. For our VT annotation task, users were given the options of "True" for when they believe the alarm was correct, "False" for when they believe the alarm was incorrect, "Uncertain" for when they are unsure which annotation to assign, "Reject" for when the alarm is unreadable due to noise, artifacts, or other hindrance, and "Save for Later" for when the user would like to return annotate this event another time. For more detailed description of the annotation platform for this project, please see
McCullum et al. 2023].
The annotation team consists of six annotators, including a leading arrhythmia analysis expert physician who built the MIT-BIH Arrhythmia database, and highly-experieneed board certified cardiac arrhythmia technician. The remaining three clinical annotators had the following expertise: an endocrionologist and internist with extensive experience in Emergency Medicine and cardiovascular intensive care; an inpatient physician, board certified in nephrology and internal medicine, with substantial experience managing patients eritical care and telemetry settings; and an anesthesiologist working in ICU setting. The annotation team also included a senior biomedical research engineer with decades of experience in arrhythmia analyses.
Appendix C Additional Results on Model Performance
C.1 Real-time event
In this section, we present additional results for the real-time VT alarm classification task. Table 7 displays the test set performance the 10 random seeds. Model selection was based on the best Challenge Score on the validation set. In Figure we plot the ROC curves and their operating points for a subset of the models the real-time event.
model exhibited the best validation performance among
13

---

Thue Postive Rate
0.6
0.4
SAE
Transformer
0.2
CNN
CNN-CL
FCNOL
False
Figure 3: ROC curves and their operating points for subset of models in the real-time event. Operating point (marked as dots) for each model on the ROC curve was selected using the threshold that resulted in the best validation Challenge Score.
Table 8: Retrospective event performance from the VTaC database. Mcan and standard deviation from 5 seeds shown. hop-performing models for each specific metric highlighted bold. CL=Contrastive Learning.
Methods
TPR
TNR
PPV
Score
AUC
Rule-based
0.964
0.571
0.471
0.633
65.54
MLP
0.585
0.711#aa16;
0,445
0.505
46.02+225
0.695 0.01
Supervised
SAE
0.836 0.466
0.761 +-0os
0,585
0:445
0.687+taz2
66.01 +2337
0.878=0.0119
Transformer
0.677 0.802
0.500=0.031
0.613
58,20++207
0.81
=0,029
CNN
0.923
0.803
0.651
0.763
7..94
0.942
FCN
0.923
0.786
0.632.00025
0.750110016
1.88
0.925
CNN+CL
0.920=aaaz
0.837
0.692
0.78914004
78.85
0.946=0.007
Supervised+CL
FCN+CL.
0.921
0.83640011
0.691 =0.11
0.789
78.95
0.933
BeatGan
0.7811:..0 0.497:..114
0.385 0.023
0,513
1:0013
46.22:11.77
0.639
Generative
TAmoGAN
=..49:0.05)
0.056
0.397 0.014
0,518
10:001
46.97:1049
0.648 0.000
Diffusion++1.
1.
0.503
0.462_
0.518
1:00
49.70=242
0.569
C.2 Performance in Retrospective Event
In the context the retrospective event, machine learning models were assessed for their ability to reduce false arrhythmia alarms using not only the data preceding the alarm but also five seconds of data after the alarm. For retrospective events, we used the optimal hyper-paramette grid search of the real-time event, and trained the model with that setting using 10 different random seeds. Table displays the performance averaged over the models from the 5 random seeds with the best validation performance.
setting from the
The winning entry the 2015 PhysioNet challenge using Rule-based approach achieved Challenge Score of 65.54 in the retrospective event. The CNN and FCN models and their contrastive learning variants remained consistently strong performers in retrospective event. comparing the performance of the real-time and retrospective events, FCN with contrastive learning had noticeable improvement the challenge scores. Other models did not benefit significantly from the additional five seconds of post-alarm data input the model.
14

---

Appendix D Machine Learning Model Setting
D.1 Supervised Models
displyysssss the range of hyperparameter settings employed during the grid search as part of the hyperparameter tuning process. To overcome the problem of imbalanced classes, a positive class weight was added to the positive samples in the BCE loss function.
The Multilayer Perceptron (MLP) consists of three hidden layers with node sizes of 1024, 256, and 64, respectively. The Supervised Autoencoder (SAE) architecture utilizes an encoder and decoder, both composed of four convolutional layers. A four layer MLP operates on the 32 dim vector to produce model predictions. Learning rate, dropout probability and positive class weight were chosen from hyperparam tuning via grid search based on best validation set performance.
The Convolutional Neural Network (CNN) architecture employed in this study consists of four 1 I-dimensional convolutional feature extractors. Each extractor is composed of two convolutional layers with window sizes of 50, 100, 200, and 400, respectively. Dropout regularization is applied to mitigate overfitting. The resulting feature maps are then flattened and passed through a hidden layer with 128 nodes before being used to make the model's final prediction. The FCN used 3 convolution layers followed by an adaptive pooling layer. Between each convolution, batch normalization was used. Lastly, we used adaptive max pooling followed by dense layer which is used to make the model's 's final prediction.
The Transformer model consists of 1 single-layer Transformer encoders. The dimensionality of the representations is set to 128, and number of attention head is set to 8. The input time steps first mapped to a 128-dimensional feature space using a linear layer. The processed features are then passed through the Transformer layers and activated using the GELU activation function. The Transformer model use dropout with 0.1. The models are trained using BCE loss as the loss function and please refer to Table 9 for specific hyperparameter settings.
Code is available at https: // http//gttubb...oo/...e Hoahh/ooaooaaaaaaaaaaaaaaaaaaaa Code for the rule-based approach PPlesinger et al. 2015 from the Challenge 2015 PhysioNet website
D.2
Conditional Generative Models
In addition to conventional classification models, we also employed conditional generative models [Wu et al. 2023] Zhou al. 2019 Basharr and Nayak 2020) to classify the VT alarms as true or false. The generative models generate waveform trajectories conditioned on patient's 's observed waveform data. Specifically, conditioned on an observed waveform segment from a patient, our generative approach predicts what the subsequent trajectory of the patient's 's waveform would look like if it were a true alarm. Thus, in the case of false alarm, the predicted waveform (simulating a true alarm) will differ significantly from the observed ground truth. The distance between the generated trajectory and the original waveform thus calculated to determine the type of alarm. Since false alarms can be caused by various factors, while true alarms have specific pathological features, we choose to generate trajectories for true alarms to calculate the anomaly scores. If the candidate observed sample is from a genuine arrhythmia alarm, the distance between the generated waveforms and the observed samples will be small; on the other hand, if the alarm is false, the discrepancy between the two will be large. Unlike the classification models, we need to generate physiological waveforms using time intervals that correspond to the true event as much as possible. For BeatGAN [Zhou et al.) 2019 and TanoGAN [Bashar and Nayak 2020). We used their Github repository's settings and modified the input channels to fit the number of channels in our dataset, and we changed the threshold setting from fixed threshold to threshold search approach using validation data. In the BeatGAN and TanoGAN models, the learning rate is set to 0.0001, the weight decay is set to 0.0005, and the batch size is 128.
The diffusion-bart based generative model consists of a network of 36 residual layers with 256 residual and skip channels. The diffusion embedding layer have three level of diffusion embedding of 128, 256, and 256 dimensions. Each layer are connected by a swish activation function. Then, we leverage
hhups://www, http::/wwwww..oooooo./oooooeoeta 1tt./////0.///rree//
e hhtps://githuubb cww/hi- hhtps://giinnh..oom/bbbboggggggo "ntps://githuu.coom///o http://hhbb..oom///mmaa/s////aaa
com/hi-bingg//ReaGGGA
15

---

two Transformer encoders for extracting the noise input and conditional input. Each encoder contains one encoder layer and the "dmodel" of each encoder layer 512 and "nhead" and feed forward layer between each encoder layer have 512 dimensions. The number of negative samples is 32 and the temperature parameter of total loss lamda 0.5. the inference stage, we used 200-time steps on a linear schedule for diffusion configuration from beta of 0.0001 to 0.02. We utilize an Adam as the optimizer with the learning rate of Ic-4. We randomly mask the half part of data for train and mask the first half of data for test. Because of the noose (O, 1) where / = 1, we scaled down the ABP channel by factor of 10 to solve the problem of sample value out range. We use the MSE Loss to calculate the distance between the generated results and the original waveforms. The threshold used for the test set is the one corresponding to the optimal score obtained on the validation.
Table 9: Model Hyperparameters and their Search Range. Parameter settings with the best validation
Score underlined.
Hyperparameters
Search Range
MLP
Batch Size
32, 64, 128 0.0001, 0.001 0.0, 0.1, 0.3 3.54, 4
Learning Rate
Dropout Probability
Positive Class Weight
SAE
Batch
32, 64, 128 0.0001, 0.001 0.5, 1.0, 1.5 0.0, 0.1. 0.3 3.54, 4
Learning Rate
Reconstruction Loss Weight
Dropout Probability
Positive Class Weight
CNN
Batch Size
32, 64, 128 0.0001, 0.001 0.0, 0.1, 0.3 3,54. 4
Learning Rate
Dropout Probability
Positive Class Weight
CNN CL
Batch Size
32, 64, 128 0.0001, 0.001
Learning Rate
Dropout Probability
0.1. 0.3, 0,5 0.5, 1.0. 1.5 3.54.
Contrastive Loss Weight
Positive Class Weight
FCN
Batch Size
32, 64, 128 0.0001, 0.001 0.0, 0.1, 0.3 3.54, 4
Learning Rate
Dropout Probability
Positive Class Weight
FCN + CL
Batch Size
32, 64, 128 0.0001, 0.001 0.1. 0.3. 0.5 0.5, 1.0, 155 3.54, 4
Learning Rate
Dropout Probability
Contrastive Loss Weight
Positive Class Weight
Transformer
Batch Size
32, 64, 128 0.0001, 0.001
Learning Rate
Dropout Probability
0.0. 0.1. 0.3 3.54.
Positive Class Weight
16

---

D.3 Hardware and Software
In our experiments, we used NVIDIA Tesla V100 GPUs with 32 GB of VRAM and IBM Power9 CPUs, running on Red Hat Enterprise Linux 8.3.1. We used Python version 3.8.18, Pytorch version 1.9.0 using a CUDA version 12.2 backend and WFDB version 1.1.1.
17


Reviews:
[{"review_id": "w1DhZaNSSD", "reviewer": "Reviewer_pRjp", "summary": "", "questions": "", "limitations": "It will be useful to denote any details of the bedside monitor software that was used to raise the alarms, and discuss any implication of bedside monitor device alarm software version updates on this dataset. \n", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "Data collection methodology is strong, with a larger number of patients and two hospital sites represented in the dataset. The labelling strategy of having multiple annotators for most of the labelled waveforms in the dataset is a strong attribute of this dataset, and will be useful to assess the maximum performance one can aspire to reaching. The total number of labelled waveforms in this dataset is larger as compared to the existing datasets in this area. In all, this dataset will be a valuable resource for researchers developing algorithms to classify true from false VT alarms.  ", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "60F2cM3Inz", "reviewer": "Reviewer_2kri", "summary": "", "questions": "", "limitations": "The authors clearly mentioned that the database is not for predicting arrhythmia.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "In intensive care units, false arrhythmia alarms are problematic as it is hard to reliably detect VT and there are so many alarms. This dataset is the largest open-access, federated VT alarm dataset, annotated by clinical experts. Some annotators are highly skilled clinical experts related to the targeted disease. \n\nWhile collecting the dataset, the authors appropriately exclude uncertain or conflicted events. Even though not every record is examined by more than two annotators, the authors analyze the annotator's agreement. \n\nThe experiments include a reasonable number of baseline models and the authors also report the performance of the recent models.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "8ASJQhOzSG", "reviewer": "Reviewer_uV7r", "summary": "", "questions": "", "limitations": "The authors do discuss the limitations of their work, with a particular focus on patient privacy. Some more discussion on limitation is always welcomed. One limitation are the labelers (note, this is not a problem with the paper, but might be worth acknowledging), their biases which is greatly reduced by having multiple labelers, can still be present if two or more labelers have the same bias.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "- Dataset will be publicly accessible (in June 2024)\n- Quite high agreement on annotations, suggesting the labels are high quality \n- Well written\n- Tests a range SoTA approaches on the dataset", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "jpAZ11bzY2", "reviewer": "Reviewer_sRHE", "summary": "", "questions": "", "limitations": "In paper, limitations has already been discussed. However, i would like to know more on your saying: absence of detailed clinical information accompanying the waveform recordings, and  following sentence.", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "The availability of a large annotated database for Ventricular Tachycardia Alarms highlights the complexity of the task and emphasizes the necessity for ongoing research in this field. As a result, it remains an open and challenging problem from the perspective of machine learning.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}]

Author rebuttal:

Meta-review: This work proposes a new waveform dataset (ECG signals with additional pulsatile signals) for detecting ventricular tachycardia (VT), sourced from multiple ICUs of two hospitals in the US. The dataset contains approximately 5,000 samples with automatic alarm records, as well as expert opinions (i.e. whether the alarm was correct or not) annotated by human experts. A set of diverse models were trained on this dataset to demonstrate its utility, compared to a previous waveform dataset.
While the value of the dataset is solid, the narrow usage scope of the dataset remains a concern. The dataset contains only one label, namely the validity of the VT alarm. Therefore the dataset might not be appealing to the wider audience of NeurIPS. But the pratical/clinical utility of the dataset (which is rare in the field of ML for healthcare) is acknowledged.
Decision: Accept (Poster)