Title: RELATIONAL MULTI-INSTANCE LEARNING FOR CONCEPT ANNOTATION FROM MEDICAL TIME SERIES

Abstract: Recent advances in computing technology and sensor design have made it easier to collect longitudinal or time series data from patients, resulting in a gigantic amount of available medical data. Most of the medical time series lack annotations or even when the annotations are available they could be subjective and prone to human errors. Earlier works have developed natural language processing techniques to extract concept annotations and/or clinical narratives from doctor notes. However, these approaches are slow and do not use the accompanying medical time series data. To address this issue, we introduce the problem of concept annotation for the medical time series data, i.e., the task of predicting and localizing medical concepts by using the time series data as input. We propose Relational Multi-Instance Learning (RMIL) -a deep Multi Instance Learning framework based on recurrent neural networks, which uses pooling functions and attention mechanisms for the concept annotation tasks. Empirical results on medical datasets show that our proposed models outperform various multi-instance learning models.

1. INTRODUCTION:

Clinicians have limited time (e.g., only a few minutes (Howie et al., 1999) ) to study and treat each patient. However, they are overloaded with a lot of patient data from multiple sources and in various formats, such as patient medical history and doctor's notes in free-flowing text, vitals and monitoring data which are captured as time series, and prescriptions and drugs which appear as medical codes including ICD-9 (Organization & Corporation, 1998) , LOINC codes (Forrey et al., 1996) , etc. This rich information should be summarized and available to clinicians in easily digestible format for faster diagnosis and treatment. Graphical visualizations (Plaisant et al., 1998) are a popular approach to show patient data to doctors. However, recent studies have shown that graphical visualisations are not always helpful for clinicians' decision-making (Law et al., 2005; Van Der Meulen et al., 2010) . Text summaries on the other hand are widely embraced and are usually adopted in practice (Scott et al., 2013) . Most existing systems use natural language processing techniques (Afantenos et al., 2005; Giordano et al., 2015) to generate summaries from doctor notes which include test results, discharge reports, observational notes, etc. While these systems are useful, they only use one source of data, i.e., doctor's notes which might have noisy and erroneous entries, for text summarization. On the other hand, electronic health records have other sources of patient data such as vital signs, monitoring sensors, and lab results in the form of multivariate time-series, which can be more accurate and may contain rich information about patient's conditions. Few existing patient summarization systems actually extract information directly from these time series for concept prediction and/or summarization. Generating simple text summaries such as trends from time series has been investigated before (Sripada et al., 2003) but is marginally useful since these trends are not mapped to the medical concepts which clinicians can quickly comprehend. Recent works (Pham et al., 2016; Choi et al., 2016a; b; Lipton et al., 2015; Che et al., 2016) have successfully shown that clinical events and outcomes can be predicted using medical codes or clinical time series data. However, directly obtaining medical concept annotations and summaries from the time series data is still an open question.

In this work, we introduce the concept annotation task as the problem of predicting and localizing the medical concepts by modeling the related medical time series data. Figure 1 illustrates a concept annotation example where medical time series data such as heart rate, pH and blood gas pressure are given, and the goal is to predict the time series of concepts such as intubation, extubation and resuscitate. To solve concept annotation problem, we formulate it as a Multi-Instance Learning (MIL) problem (Dietterich et al., 1997) and propose a deep learning based framework called Relational Multi-Instance Learning (RMIL). RMIL uses Recurrent Neural Networks (RNNs) to model multivariate time series data, leverages instance relations via attention mechanisms, and provides concept predictions using pooling functions.

The main contributions of our work are the following. We present a unified view of the MIL approaches for time series data using RNN models with different pooling functions and attention mechanisms. We show that our RMIL model is capable of learning a good classifier for concept detection (bag label predictions) and concept localization tasks (instance label prediction), even though it is only trained using bag labels. We demonstrate that RMIL obtains promising results on real-world medical datasets and outperforms popular MIL approaches.

The rest of the paper is structured as follows. In the following section, we briefly discuss the related works. Afterwards, we describe MIL framework and describe how RNN can be combined with multi-instance learning framework to obtain our proposed RMIL. In Sections 4 and 5, we present experimental results and conclusions respectively. In the appendix, we demonstrate anomaly detection as another application of our RMIL framework.

2. RELATED WORK:

Discovering concept annotations from the multivariate time series is a relatively new problem in medical domain with limited prior work. In this section, we will first highlight the related works on annotation tasks and then review related works on multi-instance learning.

Concept Annotation:

In medical domain, concept annotation is usually addressed in the clinical narrative mining and biomedical text mining literature (Aggarwal & Zhai, 2012; Cohen & Hersh, 2005; Zweigenbaum et al., 2007; Vincze et al., 2008) . In other domains such as web-mining and computer vision, the concept annotation is usually analogous to semantic annotation (Kiryakov et al., 2004) , image annotation (Jeon et al., 2003) , object localization (Lampert et al., 2008) and image captioning (Karpathy et al., 2014) .

Clinical Narratives Mining Automated discovery of temporal relations from clinical narratives (Savova et al., 2009; Zhou et al., 2006; Albright et al., 2013) and doctor notes (Plaisant et al., 1996) to uncover the patterns of disease progression is an important research problem in clinical informatics. Recent efforts such as SemEval competitions (Bethard et al., 2016) have been conducted to study this problem and evaluate/benchmark clinical information extraction systems (Xu et al., 2010) . These competitions focus on discrete, well-defined tasks which allow for rapid, reliable and repeatable evaluations. However, they only consider identifying and extracting temporal relations from clinical notes and do not use the accompanying medical time series data.

Image Annotation and Captioning Successful object recognition systems have been developed in the past few decades for image annotation, object detection and localization in images and videos. ImageNet (Deng et al., 2009) and PASCAL challenges (Everingham et al., 2010) have greatly accelerated the research in this area. Image captioning and visual-to-text translation, which are more generalized image annotation tasks, have been recently studied in several works (Karpathy et al., 2014; Kojima et al., 2002; Mao et al., 2014; Yu et al., 2016) where the goal is to find a text caption for a given image. Deep learning models such as RNN and sequence-to-sequence models (Sutskever et al., 2014) have achieved excellent results for image annotation/captioning tasks.

Multi-Instance Learning Multi-Instance Learning (MIL), a well known researched topic in machine learning, was first introduced by Dietterich et al. (1997) as a form of weakly supervised learning for drug activity prediction. MIL frameworks have since been applied to many other domains including image and text annotations (Chen & Wang, 2004; Soleimani & Miller, 2017) . Andrews et al. (2003) adapted Support Vector Machines (SVM) to the MIL framework and introduced miSVM and MISVM for optimizing instance-level and bag-level classifications respectively. Zhou & Zhang (2007) further extended MIL and proposed MIMLSVM for tackling multi-label problems. Zhou et al. (2009) introduced miGraph and MIGraph to model the structure in each bag. Zhang et al. (2011) ; Guan et al. (2016) also proposed MIL framework for structure data by leveraging the relational structures at the bag and instance levels. Generative model based MIL frameworks such as Multi-Instance Mixture Model (MIMM) (Foulds & Smyth, 2011) , and Dirichlet Process Mixture of Guassians (DPMIL) (Kandemir & Hamprecht, 2014) have also been proposed for binary multi-instance classification. Guan et al. (2016) used an autoregressive hidden Markov model and proposed an MIL framework for activity recognition in time-series data. Garcez & Zaverucha (2012) used recurrent neural networks to combine instance-level preprocessing and bag-level classification in MIL setting. Comprehensive reviews of MIL approaches are provided in Amores (2013); Herrera et al. (2016) ; Soleimani & Miller (2017) . Recently, deep learning models have been successfully applied for MIL framework (Zhu et al., 2017; Wu et al., 2015; Huang et al., 2013; Yan et al., 2016; Kotzias et al., 2014; Kraus et al., 2016) and these approaches are generally termed as deep multi-instance learning models. Most of these works use either convolutional neural networks or deep neural networks in their MIL framework for image annotation, labeling, segmentation or classification tasks. Despite the popularity of deep models for MIL, there are few works which have extended deep MIL models for multivariate time series data. The goal of this paper is to propose and study deep multi-instance learning models for multivariate time series data.

3. METHODS:

In this section, we will first describe the Multi-instance learning framework, and then present our problem formulation and our proposed relational multi-instance learning models.

3.1. MULTI-INSTANCE LEARNING:

Multi-instance learning (MIL) is a form of weakly supervised learning where the training data is arranged in sets called bags, and a label is provided for the entire bag. The data points inside a bag are referred to as instances. In the MIL framework, instance labels are not provided during training. The main goal of MIL is to learn a model based on the instances in the bag and the label of the bagto make bag-level and instance-level predictions. In this work, we only focus on the classification task in MIL, leaving out other learning tasks such as regression. Generally, two broad assumptions can be used to model the relationship between instance label and bag label. In the standard MIL assumption (Dietterich et al., 1997) , the bag label is negative if all the instances in the bag have a negative label, and the bag label is positive if at-least one of the instances in the bag has a positive label. Following the notations of Carbonneau et al. (2017) , let X denote a bag with N feature vector instances i.e., X = {x 1 , ..., x N }. Let each instance x i in feature space X be mapped to a class by some process f : X → {0, 1}, where 0 and 1 correspond to negative and positive labels respectively. The bag classifier, also know as the aggregator function, g(X) is defined by:

g(X) = 1 if ∃x ∈ X : f (X) = 1; 0 otherwise

The standard assumption is quite restrictive for some problem settings, where the positive bags cannot be identified by a single instance. Thus, this assumption can be relaxed to a collective assumption which says that several positive instances in a bag are necessary to assign a positive label to that bag.

In this case, a bag classifier is given by:

g(X) = 1 if x∈X f (X) ≥ θ; 0 otherwise

where θ is a threshold which indicates the minimum number of instances with positive labels that should be present in a bag to assign a positive label to that bag Weidmann et al. (2003) . As discussed in section 2, a plethora of works have adapted machine learning models to the MIL setting to optimize instance-level and/or bag-level predictions.

3.2. PROBLEM FORMULATION:

We formulate the concept annotation task as the detection and localization of concepts given the medical time series data. Let each patient i ∈ {1, .., N } be associated with a medical time series (also referred to as feature time series) denoted by X i ∈ R T ×D , where D denotes the number of features (such as heart rate, blood pressure) and T denotes the length of time series observations (i.e., amount of time a patient is monitored). Let C denote the set of all the concepts associated with the N patients, and Y i ∈ {0, 1} K denote the concepts associated with i th patient where

K = |C|. Let C i ∈ {0, 1} T ×K denote the concept time series of X i with C jk i = 1 when concept k is present at time-stamp j.

In multi-instance learning settings, we treat each time series X i as one bag, and the observation at each time step j i.e. X j i ∈ R D as an instance in that bag. We are interested in the following tasks:

• Concept prediction task: for patient i, given X i , predict Y i . • Concept localization task: for patient i, given X i , predict C i .

Notice that during training phase, only the input X i and prediction label Y i are available. Though C i is not known, we usually have some assumptions about the relationship of prediction and localization labels. In this work,

Y k i = I 1≤j≤T C jk i ≥ η

, where I is an indicator function and η is a constant which depends on the MIL assumption. For example, in our concept annotation tasks we make the standard assumption, i.e we assume η = 1, i.e. the time series label (bag label) for a concept is positive if that concept is present at any one time-stamp (at-least one instance has positive label).

3.3. RELATIONAL MULTI-INSTANCE LEARNING FRAMEWORK:

Inspired by the recent success of recurrent neural networks in sequence modeling Bahdanau et al. (2014) ; Sutskever et al. (2014) and classification tasks Krizhevsky et al. (2012) ; Simonyan & Zisserman (2014) , we adapt these models to the MIL framework to model multivariate time series data for concept annotation tasks. We denote all the variables at every time step as an instance and the entire multivariate time series as a bag. Unlike the traditional MIL setting, where the instances within a bag are independent of each other, in our case, the instances have relationships (namely temporal dependencies) among them. To model these dependencies, we propose to combine RNN models such as Long-Short Term Memory Neural Networks (LSTM) and Sequence-to-Sequence models with MIL, and propose our Relational Multi-Instance Learning framework, abbreviated as RMIL. RMIL takes in multivariate time series as input and outputs concept annotations. In RMIL, the outputs of RNN model provide the instance label predictions (i.e. solution for concept localization task) and the aggregation of the instance labels using aggregators such as pooling layer provides the bag label predictions (i.e. solution for concept prediction task). We propose different pooling functions and attention mechanisms which can be easily incorporated into our RMIL to improve the concept annotations.

Pooling Layers for RMIL The bag-level prediction is obtained by using an aggregation gathering on all instance-level predictions. The aggregator function g(•) : [0, 1]

T → [0, 1] in RMIL can be modeled using the pooling layers. Without loss of generality we assume that RNN model computes a mapping from the feature time series to the concept time series for each of the concept k ∈ C. Let us denote the probability of an instance j belonging to concept k as p jk . Then, the bag level probability for a concept k is given by P k = g(p 1k , p 2k , . . . , p T k ). The role of aggregator function g(•) is to combine the instance probabilities from each class specific feature map {p jk } into a single bag probability P k . Several pooling mechanisms shown in Table 1 have been introduced in MIL and deep learning literature which can be used in our RMIL. In Table 1 , r, a, b k , and r k are parameters which can be fixed or are learned during training, and σ(•) denotes the sigmoid function. 

Pooling Functions:

Noisy-OR pooling (Zhang et al., 2006 )

P k = g ({p jk }) = 1 -j (1 -p jk )

Log-sum-exponention (LSE) (Ramon & De Raedt, 2000 )

P k = g ({p jk }) = 1 r log 1 T j exp (rp jk )

Generalized Mean (GM) (Keeler et al., 1991 )

P k = g ({p jk }) = 1 T j p r jk 1/r

Integrated segmentation and recognition (ISR) (Keeler et al., 1991 )

P k = g ({p jk }) = j p jk 1-p jk / 1 + j p jk 1-p jk

Noisy-AND pooling (Kraus et al., 2016 )

P k = g ({p jk }) = σ(a( j p jk /T -b k ))-σ(-abk) σ(a(1-b k ))-σ(-ab k )

Max pooling

P k = g ({p jk }) = max j (p jk ) Softmax pooling P k = g ({p jk }) = j p jk exp (r k p jk ) / j exp (r k p jk )

Attention Mechanism for RMIL Instances within each bag have temporal relations between them. We can use attention mechanism to focus on some of the instances and their relations to improve their instance-level predictions. In order to make predictions at time j, the hidden state h j ∈ R Q of RNN can be used, where Q is the hidden state dimension. However, relevant information may be captured by hidden states at other time steps as well. Thus, we may want to introduce an attention vector or matrix (a) to leverage information of hidden states H = (h 1 , • • • , h T ) ∈ R T ×Q from all time steps. Let us denote the output after attention as H ∈ R T ×Q . The attention matrix can then be modeled using H in various ways as listed below.

Feature-based Attention One idea is to design the attention matrix based on the feature and its time-stamp. Let us define a feature-based attention matrix as

A = (a 1 , • • • , a T ) ∈ R T ×Q . For each j = 1, • • • , T , we have a j = exp w j H   1≤j ≤T exp w j H  

and H = A H, where and are element-wise multiplication and division, respectively and

W = (w 1 , • • • , w T ) ∈ R T ×T

is the weight matrix which can be learned during training. We call this Attention-F mechanism. We can simplify this attention by averaging the attentions for all hidden dimensions by taking a jq ← 1/Q • 1≤q ≤Q a jq . We will denote this as Attention-FS mechanism.

Time-based Attention Attention model (Ma et al., 2017) can be designed to capture the relation between the current time step j and previous time steps j ≤ j, by solely relying on previous hidden states h j . We can define a time-based attention matrix as A ∈ R T ×T . For each j and j in [1, • • • , T ], we have

a j,j =      exp w h j / 1≤j ≤j exp w h j , j ≤ j; 0, otherwise.

and H = A • H, where w ∈ R D is the weight vector to learn. We use Attention-T to represent Time-based attention mechanism.

Interaction-based Attention:

The time-based attention can be further improved by considering both the previous and current hidden states h j and h j (Ma et al., 2017) . In this case, we have

a j,j =      exp v tanh(W1hj + W2h j ) / 1≤j ≤j exp v tanh(W1hj + W2h j ) , j ≤ j;

0, otherwise and similarly H = A • H. Here, we need to learn v ∈ R S , W 1 ∈ R S×Q , W 2 ∈ R S×Q , and we choose S = Q/2. A simplified version of interaction-based attention can be obtained if we use vector The above attention mechanisms usually help both prediction and localization tasks.

w 1 ∈ R Q , w 2 ∈ R Q instead of matrices W 1 ,

4. EXPERIMENTS:

Here, we demonstrate the performance of our proposed RMIL models on concept annotation tasks i.e. concept prediction and localization tasks, using a real-world health-care dataset and compare its performance to the popular multi-instance learning approaches. In addition, we discuss the impact of using pooling functions and attention mechanism in our RMIL framework. For our work, we extracted 21 feature time series from more than 2,000 adult patients who were diagnosed with a respiratory disorder such as Acute Hypoxemic Respiratory Failure (AHRF) (Khemani et al., 2009) at the time of admission. These 21 features are respiratory based features such as peak inspiratory pressure (PIP) and arterial partial pressure of oxygen (PaO2) and were collected during the first 3 days after admission. The feature time series has 4 time stamps and the first time stamp corresponds to the admission time. We denote this dataset as MIMIC-III RESP dataset. In addition, we also generated another feature time series with more time stamps whose results is shown in the appendix.

4.1. DATASET DESCRIPTIONS AND EXPERIMENTAL DESIGN:



Concept Annotations:

The medical time series data of MIMIC-III dataset does not come with the concept annotations, however the medical concepts are available in the doctor notes of the MIMIC-III database. To obtain the concept annotations, we extract the concept time series from the doctor notes using the NOTEEVENTS table of MIMIC-III database. The total number of doctor notes is 2,083,180, out of which 98.15% of notes (2,044,634 notes) have no timestamp and 1.85% of notes (38,546 notes) have timestamps associated with them. The total number of unique concepts in the doctor notes in the first 3 days data is 6,197. To obtain concept time series for each patient with respiratory disorder such as AHRF, we first identified respiratory-related concepts from the medical literature (Antonelli et al., 2001; Khemani et al., 2009) , and obtained their medical codes from the Unified Medical Language System (UMLS) dictionary (Bodenreider, 2004) . Then, we mined the patient's doctor notes from NOTEEVENTS table to extract all the possible medical concepts related to the respiratory system and its disorder. In total, we chose top 26 respiratory concepts to generate concept time series which has the same number of time stamps as feature time series. 

4.2. COMPARISON AND IMPLEMENTATION DETAILS:

We compare the performance of our proposed models to the popular MIL models such as MISVM (Andrews et al., 2003) , DPMIL (Kandemir & Hamprecht, 2014) and Convolutional Neural Networks (CNN) Kraus et al. (2016) , and CNN with attention. We categorize all the evaluated methods into two groups: For LSTM models, we use two LSTM layers and two dense layers. For S2S models, we use two LSTM layers for both the encoder and the decoder. For Bi-LSTM models, we use two bi-directional LSTM layers. All the models were constructed to have a comparable number of parameters. We train all the Deep learning models with the RMSProp optimization method and we use early stopping to find the best weights on the validation dataset. For baseline MIL models, we follow the suggestions of the corresponding papers to fine-tune the parameters. All the input variables in the training data are normalized to be 0 mean and 1 standard deviation. The inputs to all the models is the same feature time series data. We used Keras (Chollet, 2017) and Python to run the deep models and MISVM models. Matlab code from the original authors was used to obtain DPMIL results. We use the area under ROC (AUROC) and area under precision-recall curve (AUPRC) scores as our evaluation metrics and report the results from 5-fold cross validation for all the evaluated methods.

4.3. QUANTITATIVE RESULTS:

Table 3 shows the concept annotation results on the MIMIC-III RESP dataset. From this table, we see that RMIL models outperform the non-deep multi-instance learning models by at least 8-10% for concept localization task, and by at least 10-15% for concept prediction tasks in terms of AUROC and AUPRC. RMIL performs slightly better than CNN-based models on all the metrics. Among all the RMIL models, we find that LSTM model obtains slightly better overall results compared to the other models for localization task.

To study the impact of pooling and attention, we trained and evaluated LSTM models with different pooling functions and different attention mechanisms, which are described in Section 3. Tables 4 and 5 show the comparison results. From these tables, we observed that (i) all the attention mechanisms except feature-based attention perform similar to each other especially for the prediction task, and (ii) all the pooling functions other than ISR and Noisy-OR obtain similar overall performance. This demonstrates that choice of attention does not matter but choice of pooling has some impact in our RMIL framework. 

4.4. DISCUSSIONS:

We can study the interpretability of concept localization by looking at the localization results of our RMIL models, even though the model is trained without the labels for localization. Figure 2 shows the ground truth annotations of two respiratory concepts -intubation and extubation concepts, and the prediction probabilities of these concepts obtained by our RMIL attention-based LSTM models. From figure 2 (a) we can make the following observations, (i) intubation usually happens before extubation for the same patient, (ii) intubation and extubation could happen on the same day, and (iii) intubation and extubation occur commonly within the first 24 hours of admission. From the figure 2(b), we see that our RMIL attention based LSTM predicts that the probability of intubation happening on the first day of admission is higher (draker gray means higher probability of concept occurrence) and the probability of extubation happening within first day is lower. This indicates that the model has correctly learnt that intubation should appear before extubation. This also implicitly implies that the RMIL attention-based LSTM models have correctly learnt the instance-level relationships from the medical time series data with only bag-level labels.

5. SUMMARY:

In this paper, we presented Relational Multi-Instance Learning -a deep multi-instance learning framework using recurrent neural networks for concept annotation from the medical time series data.

Empirical results on medical dataset demonstrated that our proposed models outperform the popular state-of-the-art multi-instance learning approaches. Experiments with different pooling and attention mechanisms showed that while attention mechanism does not have a significant impact on model's performance, certain pooling functions such as ISR and Noisy-OR can negatively impact the instance prediction results.

6. APPENDIX:

6.1 MIMIC-III CONCEPT LIST Table 6 show the concept list of the MIMIC-III RESP dataset used in our experiments.

6.2. ANOMALY DETECTION FOR TIME SERIES:

Here, we will demonstrate anomaly detection from medical time series data as a use case application of our RMIL framework.

6.2.1. MOTIVATION AND RELATED WORK:

Each year, more than 1,000,000 US adults and children are put on mechanical ventilation during their stays in ICU. However, lack of effective tools to aid with ventilator weaning and extubation (removal of the breathing tube) readiness assessment results in nearly half of the patients spending unnecessary days on ventilators (Randolph et al., 2002) , and up to 20% of them having ventilators discontinued too soon (Kurachek et al., 2003) . Spending unnecessary days on ventilators can lead to hospital-acquired infections while having ventilators discontinued too soon could require painful reintubation. A work-of-breathing measure called Pressure-Rate Product (PRP), calculated from esophageal pressure, has shown potential to be used as a guideline for ventilator weaning and extubation readiness assessment (Willis et al., 2005) . However, PRP calculations are susceptible to sensor artifacts and breathing pattern anomalies. These anomalies limit realtime use of PRP for clinical decision making. Our goal is to automatically detect and remove these anomalies by using our proposed Relational Multi-instance learning models, thereby enabling real-time clinical decision making. Figure 3 shows the example of anomaly detection from the ventilator time series data. Here, anomalies appear due to patient factors (cough, movement) and instrument factors (probing, catherter drift), and should be automatically detected from the ventalitor monitoring time series data. 2016) proposes a self-learning method that learns clusters of constrained grammars based on ordinal symbolic approximation of time series value. Guigou et al. (2017) proposes incorporating experts into anomaly detection in a method inspired by immune system. In Jones et al. (2016) , the authors propose extracting exemplars from Euclidean pairwise distance as a way to speed up anomaly detection algorithm. Most of these techniques for time series anomaly detection are quite domain-specific, and few of them model the anomaly detection problem in Multi-instance learning setting.

We formulate the anomaly detection as concept annotation problem in MIL setting, where the bag corresponds to the medical time series data X i and an instance is the features at a time-stamp, and the anomaly corresponds to a concept. Thus, the prediction (predicting Y i ) and localizaton tasks (predicting C i ) correspond to predicting the presence and location of the anomalies in the time series data. For anomaly detection, we assume

Y k i = I 1≤j≤T C jk i ≥ η

, where η is chosen as η = 0.6T and T is the number of time series windows within the bag.

We conduct anomaly detection experiments on a PICU dataset described below. We annotate the concept of breathing anomaly in MIL framework by processing the dataset as follows.

The sensor recordings are split into non-overlapping 5-second windows. In MIL framework, each window becomes an instance and each recording becomes a bag. For each sensor signal of each window, we extract 20 Mel-Frequency Cepstral Coefficients (MFCCs) to be used as features of each instance; thus, each instance has 80 features in total. For each instance, its anomaly label is set to positive if at least 20% of the window are labeled as anomaly. The bag anomaly label is set to positive if at least 60% of its instances are labeled as anomaly. Results Table 8 shows the anomaly detection results using our RMIL models. We observe that (i) RMIL models mostly outperform the non-deep MIL models on both tasks, (ii) LSTM and Bi-LSTM based RMIL models obtain better overall results compared to the sequence-to-sequence models, and (iii) the attention mechanism does not help for localization task but sometimes obtains better results for the prediction task.

Remark: Cluster-MIL* is a HDBSCAN clustering (Campello et al., 2015) based approach for anomaly detection. It uses both instance and bag labels for training, while other models only use bag labels for training.

6.3. ADDITIONAL RESULTS ON MIMIC-III RESP DATASET:

We sampled the feature time series from MIMIC-III RESP dataset every 6 hours and generated a time series with more (12) time stamps. We call this MIMIC-III RESP-II dataset.

Table 9 shows the results of RMIL models on this dataset. We observe that (i) All the RMIL models have similar performance for prediction task, (ii) Bi-LSTM RMIL has better localization results compared to other the models. 

:



Figures and Tables:

Figure 1: Medical time series to concept annotations.

W 2 and by setting v = 1 in the above equation. We use Attention-I and Attention-IS to represent Interaction-based attention mechanism and simplified version of interaction-based attention mechanism respectively.

Multi-Instance Learning models (MIL): We treat MISVM, DPMIL, Convolutional Neural Networks (CNN), and CNN with Attention I as our baseline MIL models. 2. Relational Multi-Instance Learning models (RMIL): We evaluate the following deep learning models as part of our RMIL framework: (a) Long Short-Term Memory neural networks (LSTM) (Hochreiter & Schmidhuber, 1997) (b) Bi-directional LSTM (Bi-LSTM) (Graves et al., 2013) (c) Sequence to Sequence models (S2S) (Sutskever et al., 2014) (d) The above three RMIL models with different attention mechanisms (e) The above RMIL models with different pooling functions

(a) Ground truth annotations of intubation and extubation concepts. (b) Intubation and extubation concept prediction probabilities identified by attention-based LSTM.

Figure 2: Ground truth and predicted concept annotation comparison. In 2(a) white block corresponds to absence of a concept and black block corresponds to presence of a concept. In 2(b), darker gray value indicates higher chance for presence of a concept. X-axis represents time steps, Y-axis corresponds to different patients.

Figure 3: Anomaly detection from ventilator time series data.

Pooling functions for RMIL.

Table content: <table/>

MIMIC-III RESP dataset.

Table content: <table><tr><td/><td>MIMIC-III RESP</td></tr><tr><td># of samples (N )</td><td>2014</td></tr><tr><td># of variables (D)</td><td>21</td></tr><tr><td># of time steps</td><td>4</td></tr><tr><td># of concepts</td><td>26</td></tr></table>

Concept annotation results on MIMIC-III RESP dataset. Max-pooling function was used in all RMIL models.

Table content: <table><tr><td>Prediction</td><td>Localization</td></tr><tr><td colspan="2">AUROC AUPRC AUROC AUPRC</td></tr></table>

Results on RMIL LSTM models with different pooling functions and with Attention-I mechanism.

Table content: <table><tr><td/><td colspan="2">Prediction</td><td colspan="2">Localization</td></tr><tr><td/><td>AUROC</td><td colspan="2">AUPRC AUROC</td><td>AUPRC</td></tr><tr><td>LSTM with ISR Pooling</td><td>0.862</td><td>0.765</td><td>0.733</td><td>0.375</td></tr><tr><td>LSTM with Noisy-AND Pooling</td><td>0.863</td><td>0.767</td><td>0.779</td><td>0.431</td></tr><tr><td>LSTM with Generalized Mean Pooling</td><td>0.863</td><td>0.767</td><td>0.796</td><td>0.450</td></tr><tr><td>LSTM with LSE Pooling</td><td>0.863</td><td>0.767</td><td>0.792</td><td>0.452</td></tr><tr><td>LSTM with Softmax Pooling</td><td>0.863</td><td>0.767</td><td>0.790</td><td>0.450</td></tr><tr><td>LSTM with Noisy-OR Pooling</td><td>0.860</td><td>0.762</td><td>0.703</td><td>0.318</td></tr><tr><td>LSTM with Max Pooling</td><td>0.862</td><td>0.766</td><td>0.797</td><td>0.444</td></tr></table>

Results on RMIL LSTM models with different attention mechanisms and with max pooling.

Table content: <table><tr><td/><td colspan="2">Prediction</td><td colspan="2">Localization</td></tr><tr><td>Model</td><td colspan="4">AUROC AUPRC AUROC AUPRC</td></tr><tr><td>LSTM with Attention-T</td><td>0.864</td><td>0.768</td><td>0.796</td><td>0.445</td></tr><tr><td>LSTM with Attention-IS</td><td>0.862</td><td>0.767</td><td>0.798</td><td>0.446</td></tr><tr><td>LSTM with Attention-I</td><td>0.862</td><td>0.766</td><td>0.797</td><td>0.444</td></tr><tr><td>LSTM with Attention-FS</td><td>0.861</td><td>0.765</td><td>0.796</td><td>0.417</td></tr><tr><td>LSTM with Attention-F</td><td>0.861</td><td>0.764</td><td>0.780</td><td>0.408</td></tr><tr><td>LSTM with no attention</td><td>0.860</td><td>0.763</td><td>0.795</td><td>0.444</td></tr></table>

MIMIC-III respiratory concept list.

Table content: <table><tr><td colspan="2">Concept Number Concept Name</td></tr><tr><td>1</td><td>Respiratory rate</td></tr><tr><td>2</td><td>Biomedical tube device</td></tr><tr><td>3</td><td>Analysis of arterial blood gases and ph</td></tr><tr><td>4</td><td>Ventilation, function (observable entity)</td></tr><tr><td>5</td><td>Medical history</td></tr><tr><td>6</td><td>Injury wounds</td></tr><tr><td>7</td><td>Body weights</td></tr><tr><td>8</td><td>Tidal volume</td></tr><tr><td>9</td><td>Inspired fraction of oxygen</td></tr><tr><td>10</td><td>Intubation</td></tr><tr><td>11</td><td>Positive end-expiratory pressure</td></tr><tr><td>12</td><td>Artificial airways</td></tr><tr><td>13</td><td>Bicarbonates</td></tr><tr><td>14</td><td>Tracheal extubation</td></tr><tr><td>15</td><td>Chest radiograph</td></tr><tr><td>16</td><td>Mean blood pressure</td></tr><tr><td>17</td><td>Oxygen measurement, partial pressure, arterial</td></tr><tr><td>18</td><td>Cardiac arrest</td></tr><tr><td>19</td><td>Peak inspiratory pressure</td></tr><tr><td>20</td><td>Biomechanical compliance</td></tr><tr><td>21</td><td>Tachycardia ventricular</td></tr><tr><td>22</td><td>Scoliosis</td></tr><tr><td>23</td><td>Acute lung injury</td></tr><tr><td>24</td><td>Partial pressure of carbon dioxide in arterial blood</td></tr><tr><td>25</td><td>Resuscitate</td></tr><tr><td>26</td><td>Sudden onset</td></tr></table>

PICU dataset statistics. Hz sensor readings from 385 subjects collected at a leading children's hospital by a team of research clinicians. Statistics of the dataset is shown in Table7. The recordings are made on mechanically-ventilated patients in pediatrics ICU ward. Medical time series data collected from four sensors are used: flow volume spirometry, esophageal pressure sensor, and dual band respiratory inductance plethysmography. Each subject can be under one of four breathing conditions: ventilated with Continuous Positive Airway Pressure (CPAP), ventilated with Pressure Support (PS), 5 minutes after extubation, and 60 minutes after extubation. Along with the 4 sensor signals, clinicians verified binary anomaly label generated using hard-crafted state-of-the-art breathing anomaly detection algorithm is provided as ground truth.

Table content: <table><tr><td/><td>PICU</td></tr><tr><td># of samples (N )</td><td>1530</td></tr><tr><td># of variables (D)</td><td>80</td></tr><tr><td># of time steps</td><td>60</td></tr><tr><td># of concepts</td><td>1</td></tr></table>

Anomaly detection results on PICU dataset with different attention mechanisms and with max pooling.

Table content: <table><tr><td>Prediction</td><td>Localization</td></tr><tr><td colspan="2">AUROC AUPRC AUROC AUPRC</td></tr></table>

MIMIC-III RESP-II dataset with different attention mechanisms and with max pooling.

Table content: <table><tr><td>Prediction</td><td>Localization</td></tr><tr><td colspan="2">AUROC AUPRC AUROC AUPRC</td></tr></table>

Peer review (iclr2018):
 The paper addresses the classification of medical time-series data by formulating the problem as a multi-instance learning (MIL) task, where there is an instance for each timestep of each time series, labels are observed at the time-series level (i.e. for each bag), and the goal is to perform instance-level and series-level (i.e. bag-level) prediction. The main difference from the typical MIL setup is that there is a temporal relationship between the instances in each bag. The authors propose to model this using a recurrent neural network architecture. The aggregation function which maps instance-level labels to bag-level labels is modeled using a pooling layer (this is actually a nice way to describe multi-instance classification assumptions using neural network terminology). An attention mechanism is also used. The proposed time-series MIL problem formulation makes sense. The RNN approach is novel to this setting, if somewhat incremental. One very positive aspect is that results are reported exploring the impact of the choice of recurrent neural network architecture, pooling function, and attention mechanism. Results on a second dataset are reported in the appendix, which greatly increases confidence in the generalizability of the experiments. One or more additional datasets would have helped further solidify the results, although I appreciate that medical datasets are not always easy to obtain. Overall, this is a reasonable paper with no obvious major flaws. The novelty and impact may be greater on the application side than on the methodology side. Minor suggestions: -The term "relational multi-instance learning" seems to suggest a greater level of generality than the work actually accomplishes. The proposed methods can only handle time-series / longitudinal dependencies, not arbitrary relational structure. Moreover, multi-instance learning is typically viewed as an intermediary level of structure "in between" propositional learning (i.e. the standard supervised learning setting) and fully relational learning, so the "relational multi-instance learning" terminology sounds a little strange. Cf.: De Raedt, L. (2008). Logical and relational learning. Springer Science & Business Media. -Pg 3, a capitalization typo: "the Multi-instance learning framework" -The equation for the bag classifier on page 4 refers to the threshold-based MI assumption, which should be attributed to the following paper: Weidmann, N., Frank, E. & Pfahringer, B. 2003. A two-level learning method for generalized multi-instance problems. In Proceedings of the 14th European Conference on Machine Learning, Springer, 468-479. (See also: J. R. Foulds and E. Frank. A review of multi-instance learning assumptions. Knowledge Engineering Review, 25(1):1-25, 2010. ) - Pg 5, "Table 1" vs "table 1" - be consistent. -A comparison to other deep learning MIL methods, i.e. those that do not exploit the time-series nature of the problem, would be valuable. I wouldn*t be surprised if other reviewers insist on this.