Original: R ELATIONAL M ULTI -I NSTANCE L EARNING FOR C ONCEPT A NNOTATION FROM M EDICAL T IME S ERIES
Revised: R ELATIONAL M ULTI -I NSTANCE L EARNING FOR C ONCEPT A NNOTATION FROM M EDICAL T IME S ERIES

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Recent advances in computing technology and sensor design have made it easier to collect longitudinal or time series data from patients, resulting in a gigantic amount of available medical data.
Revised: Recent advances in computing technology and sensor design have made it easier to collect longitudinal or time series data from patients, resulting in a gigantic amount of available medical data.

Original: Most of the medical time series lack annotations or even when the annotations are available they could be subjective and prone to human errors.
Revised: Most of the medical time series lack annotations or even when the annotations are available they could be subjective and prone to human errors.

Original: Earlier works have developed natural language processing techniques to extract concept annotations and/or clinical narratives from doctor notes.
Revised: Earlier works have developed natural language processing techniques to extract concept annotations and/or clinical narratives from doctor notes.

Original: However, these approaches are slow and do not use the accompanying medical time series data.
Revised: However, these approaches are slow and do not use the accompanying medical time series data.

Original: To address this issue, we introduce the problem of concept annotation for the medical time series data, i.e., the task of predicting and localizing medical concepts by using the time series data as input.
Revised: To address this issue, we introduce the problem of concept annotation for the medical time series data, i.e., the task of predicting and localizing medical concepts by using the time series data as input.

Original: We propose Relational Multi-Instance
Revised: We propose Relational Multi-Instance

Original: Learning (RMIL) - a deep Multi Instance Learning framework based on recurrent neural networks, which uses pooling functions and attention mechanisms for the concept annotation tasks.
Revised: Learning (RMIL) - a deep Multi Instance Learning framework based on recurrent neural networks, which uses pooling functions and attention mechanisms for the concept annotation tasks.

Original: Empirical results on medical datasets show that our proposed models outperform various multi-instance learning models.
Revised: Empirical results on medical datasets show that our proposed models outperform various multi-instance learning models.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Clinicians have limited time (e.g., only a few minutes (Howie et al., 1999)) to study and treat each patient.
Revised: Clinicians have limited time (e.g., only a few minutes (Howie et al., 1999)) to study and treat each patient.

Original: However, they are overloaded with a lot of patient data from multiple sources and in various formats, such as patient medical history and doctor’s notes in free-ﬂowing text, vitals and monitoring data which are captured as time series, and prescriptions and drugs which appear as medical codes including ICD-9 (Organization & Corporation, 1998), LOINC codes (Forrey et al., 1996)), etc.
Revised: However, they are overloaded with a lot of patient data from multiple sources and in various formats, such as patient medical history and doctor’s notes in free-ﬂowing text, vitals and monitoring data which are captured as time series, and prescriptions and drugs which appear as medical codes including ICD-9 (Organization & Corporation, 1998), LOINC codes (Forrey et al., 1996)), etc.

Original: This rich information should be summarized and available to clinicians in easily digestible format for faster diagnosis and treatment.
Revised: This rich information should be summarized and available to clinicians in easily digestible format for faster diagnosis and treatment.

Original: Graphical visualizations (Plaisant et al., 1998) are a popular approach to show patient data to doctors.
Revised: Graphical visualizations (Plaisant et al., 1998) are a popular approach to show patient data to doctors.

Original: However, recent studies have shown that graphical visualisations are not always helpful for clinicians’ decision-making (Law et al., 2005; Van
Revised: However, recent studies have shown that graphical visualisations are not always helpful for clinicians’ decision-making (Law et al., 2005; Van

Original: Der Meulen et al., 2010).
Revised: Der Meulen et al., 2010).

Original: Text summaries on the other hand are widely embraced and are usually adopted in practice (Scott et al., 2013).
Revised: Text summaries on the other hand are widely embraced and are usually adopted in practice (Scott et al., 2013).

Original: Most existing systems use natural language processing techniques (Afantenos et al., 2005; Giordano et al., 2015) to generate summaries from doctor notes which include test results, discharge reports, observational notes, etc.
Revised: Most existing systems use natural language processing techniques (Afantenos et al., 2005; Giordano et al., 2015) to generate summaries from doctor notes which include test results, discharge reports, observational notes, etc.

Original: While these systems are useful, they only use one source of data, i.e., doctor’s notes which might have noisy and erroneous entries, for text summarization.
Revised: While these systems are useful, they only use one source of data, i.e., doctor’s notes which might have noisy and erroneous entries, for text summarization.

Original: On the other hand, electronic health records have other sources of patient data such as vital signs, monitoring sensors, lab results in the form of multivariate time-series, which can be more accurate and may contain rich information about patient’s conditions.
Revised: On the other hand, electronic health records have other sources of patient data such as vital signs, monitoring sensors, lab results in the form of multivariate time-series, which can be more accurate and may contain rich information about patient’s conditions.

Original: Few existing patient summarization systems actually extract information directly from these time series for concept prediction and/or summarization.
Revised: Few existing patient summarization systems actually extract information directly from these time series for concept prediction and/or summarization.

Original: Generating simple text summaries such as trends from time series has been investigated before (Sripada et al., 2003) but is marginally useful since these trends are not mapped to the medical concepts which  can be understood by the clinicians.
Revised: Generating simple text summaries such as trends from time series has been investigated before (Sripada et al., 2003) but is marginally useful since these trends are not mapped to the medical concepts which clinicians can quickly comprehend.

Original: Recent works (Pham et al., 2016; Choi et al., 2016a;b; Lipton et al., 2015; Che et al., 2016) have successfully shown that clinical events and outcomes can be predicted using medical codes or clinical time series data.
Revised: Recent works (Pham et al., 2016; Choi et al., 2016a;b; Lipton et al., 2015; Che et al., 2016) have successfully shown that clinical events and outcomes can be predicted using medical codes or clinical time series data.

Original: However, directly obtaining medical concept annotations and summaries from the time series data is still an open question.
Revised: However, directly obtaining medical concept annotations and summaries from the time series data is still an open question.

Original: In this work, we introduce the concept annotation task as the problem of predicting and localizing the medical concepts by modeling the related medical time series data.
Revised: In this work, we introduce the concept annotation task as the problem of predicting and localizing the medical concepts by modeling the related medical time series data.

Original: Figure 1 illustrates a concept annotation example where medical time series data such as heart rate, pH and blood gas pressure are given, and the goal is to predict the time series of concepts such as intubation, extubation and resuscitate.
Revised: Figure 1 illustrates a concept annotation example where medical time series data such as heart rate, pH and blood gas pressure are given, and the goal is to predict the time series of concepts such as intubation, extubation and resuscitate.

Original: To solve concept annotation problem, we formulate it as a Multi-Instance Learning (MIL) problem (Dietterich et al., 1997) and propose a deep learning based framework called Relational MultiInstance Learning (RMIL).
Revised: To solve concept annotation problem, we formulate it as a Multi-Instance Learning (MIL) problem (Dietterich et al., 1997) and propose a deep learning based framework called Relational MultiInstance Learning (RMIL).

Original: RMIL uses Recurrent Neural Networks (RNNs) to model multivariate time series data, leverages instance relations via attention mechanisms, and provides concept predictions using pooling functions.
Revised: RMIL uses Recurrent Neural Networks (RNNs) to model multivariate time series data, leverages instance relations via attention mechanisms, and provides concept predictions using pooling functions.

Original: The main contributions of our work are the following.
Revised: The main contributions of our work are the following.

Original: We present a uniﬁed view of the MIL approaches for time series data using RNN models with different pooling functions and attention mechanisms.
Revised: We present a uniﬁed view of the MIL approaches for time series data using RNN models with different pooling functions and attention mechanisms.

Original: We show that our RMIL model is capable of learning a good classiﬁer for concept detection (bag label predictions) and concept localization tasks (instance label prediction), even though it is only trained using bag labels.
Revised: We show that our RMIL model is capable of learning a good classiﬁer for concept detection (bag label predictions) and concept localization tasks (instance label prediction), even though it is only trained using bag labels.

Original: We demonstrate that RMIL obtains promising results on real-world medical datasets and outperforms popular MIL approaches.
Revised: We demonstrate that RMIL obtains promising results on real-world medical datasets and outperforms popular MIL approaches.

Original: The rest of the paper is structured as follows.
Revised: The rest of the paper is structured as follows.

Original: In the following section, we brieﬂy discuss the related works.
Revised: In the following section, we brieﬂy discuss the related works.

Original: Afterwards, we describe MIL framework and describe how RNN can be combined with multi-instance learning framework to obtain our proposed RMIL.
Revised: Afterwards, we describe MIL framework and describe how RNN can be combined with multi-instance learning framework to obtain our proposed RMIL.

Original: In Sections 4 and 5, we present experimental results and conclusions respectively.
Revised: In Sections 4 and 5, we present experimental results and conclusions respectively.

Original: In the appendix, we demonstrate anomaly detection as another application of our RMIL framework.
Revised: In the appendix, we demonstrate anomaly detection as another application of our RMIL framework.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Discovering concept annotations from the multivariate time series is a relatively new problem in medical domain with limited prior work.
Revised: Discovering concept annotations from the multivariate time series is a relatively new problem in medical domain with limited prior work.

Original: In this section, we will ﬁrst highlight the related works on annotation tasks and then review related works on multi-instance learning.
Revised: In this section, we will ﬁrst highlight the related works on annotation tasks and then review related works on multi-instance learning.

Original: Concept Annotation In medical domain, concept annotation is usually addressed in the clinical narrative mining and biomedical text mining literature (Aggarwal & Zhai, 2012; Cohen & Hersh, 2005; Zweigenbaum et al., 2007; Vincze et al., 2008).
Revised: Concept Annotation In medical domain, concept annotation is usually addressed in the clinical narrative mining and biomedical text mining literature (Aggarwal & Zhai, 2012; Cohen & Hersh, 2005; Zweigenbaum et al., 2007; Vincze et al., 2008).

Original: In other domains such as web-mining and computer vision, the concept annotation is usually analogous to semantic annotation (Kiryakov et al., 2004), image annotation (Jeon et al., 2003), object localization (Lampert et al., 2008) and image captioning (Karpathy et al., 2014).
Revised: In other domains such as web-mining and computer vision, the concept annotation is usually analogous to semantic annotation (Kiryakov et al., 2004), image annotation (Jeon et al., 2003), object localization (Lampert et al., 2008) and image captioning (Karpathy et al., 2014).

Original: Clinical Narratives Mining Automated discovery of temporal relations from clinical narratives (Savova et al., 2009; Zhou et al., 2006; Albright et al., 2013) and doctor notes (Plaisant et al., 1996) to uncover the patterns of disease progression is an important research problem in clinical informatics.
Revised: Clinical Narratives Mining Automated discovery of temporal relations from clinical narratives (Savova et al., 2009; Zhou et al., 2006; Albright et al., 2013) and doctor notes (Plaisant et al., 1996) to uncover the patterns of disease progression is an important research problem in clinical informatics.

Original: Recent efforts such as SemEval competitions (Bethard et al., 2016) have been conducted to study this problem and evaluate/benchmark clinical information extraction systems (Xu et al., 2010).
Revised: Recent efforts such as SemEval competitions (Bethard et al., 2016) have been conducted to study this problem and evaluate/benchmark clinical information extraction systems (Xu et al., 2010).

Original: These competitions focus on discrete, well-deﬁned tasks which allow for rapid, reliable and repeatable evaluations.
Revised: These competitions focus on discrete, well-deﬁned tasks which allow for rapid, reliable and repeatable evaluations.

Original: However, they only consider identifying and extracting temporal relations from clinical notes and do not use the accompanying medical time series data.
Revised: However, they only consider identifying and extracting temporal relations from clinical notes and do not use the accompanying medical time series data.

Original: Image Annotation and Captioning Successful object recognition systems have been developed in the past few decades for image annotation, object detection and localization in images and videos.
Revised: Image Annotation and Captioning Successful object recognition systems have been developed in the past few decades for image annotation, object detection and localization in images and videos.

Original: ImageNet (Deng et al., 2009) and PASCAL challenges (Everingham et al., 2010) have greatly accelerated the research in this area.
Revised: ImageNet (Deng et al., 2009) and PASCAL challenges (Everingham et al., 2010) have greatly accelerated the research in this area.

Original: Image captioning and visual-to-text translation, which are more generalized image annotation tasks, have been recently studied in several works (Karpathy et al., 2014; Kojima et al., 2002; Mao et al., 2014; Yu et al., 2016) where the goal is to ﬁnd a text caption for a given image.
Revised: Image captioning and visual-to-text translation, which are more generalized image annotation tasks, have been recently studied in several works (Karpathy et al., 2014; Kojima et al., 2002; Mao et al., 2014; Yu et al., 2016) where the goal is to ﬁnd a text caption for a given image.

Original: Deep learning models such as RNN and sequence-to-sequence models (Sutskever et al., 2014) have achieved excellent results for image annotation/captioning tasks.
Revised: Deep learning models such as RNN and sequence-to-sequence models (Sutskever et al., 2014) have achieved excellent results for image annotation/captioning tasks.

Original: Multi-Instance Learning Multi-Instance Learning (MIL), a well known researched topic in machine learning, was ﬁrst introduced by Dietterich et al.
Revised: Multi-Instance Learning Multi-Instance Learning (MIL), a well known researched topic in machine learning, was ﬁrst introduced by Dietterich et al.

Original: (1997) as a form of weakly supervised learning for drug activity prediction.
Revised: (1997) as a form of weakly supervised learning for drug activity prediction.

Original: MIL frameworks have since been applied to many other domains including image and text annotations (Chen & Wang, 2004; Soleimani & Miller, 2017).
Revised: MIL frameworks have since been applied to many other domains including image and text annotations (Chen & Wang, 2004; Soleimani & Miller, 2017).

Original: Andrews et al.
Revised: Andrews et al.

Original: (2003) adapted Support Vector Machines (SVM) to the MIL framework and introduced miSVM and
Revised: (2003) adapted Support Vector Machines (SVM) to the MIL framework and introduced miSVM and

Original: MISVM for optimizing instance-level and bag-level classiﬁcations respectively.
Revised: MISVM for optimizing instance-level and bag-level classiﬁcations respectively.

Original: Zhou & Zhang (2007) further extended MIL and proposed MIMLSVM for tackling multi-label problems.
Revised: Zhou & Zhang (2007) further extended MIL and proposed MIMLSVM for tackling multi-label problems.

Original: Zhou et al.
Revised: Zhou et al.

Original: introduced miGraph and MIGraph to model the structure in each bag.
Revised: introduced miGraph and MIGraph to model the structure in each bag.

Original: Zhang et al.
Revised: Zhang et al.

Original: ; Guan et al.
Revised: ; Guan et al.

Original: (2016) also proposed MIL framework for structure data by leveraging the relational structures at the bag and instance levels.
Revised: (2016) also proposed MIL framework for structure data by leveraging the relational structures at the bag and instance levels.

Original: Generative model based MIL frameworks such as Multi-Instance Mixture
Revised: Generative model based MIL frameworks such as Multi-Instance Mixture

Original: Model (MIMM)(Foulds & Smyth, 2011), and Dirichlet Process Mixture of Guassians (DPMIL) (Kandemir & Hamprecht, 2014) have also been proposed for binary multi-instance classiﬁcation.
Revised: Model (MIMM)(Foulds & Smyth, 2011), and Dirichlet Process Mixture of Guassians (DPMIL) (Kandemir & Hamprecht, 2014) have also been proposed for binary multi-instance classiﬁcation.

Original: Guan et al.
Revised: Guan et al.

Original: (2016) used an autoregressive hidden Markov model and proposed an MIL framework for activity recognition in time-series data.
Revised: (2016) used an autoregressive hidden Markov model and proposed an MIL framework for activity recognition in time-series data.

Original: Garcez & Zaverucha (2012) used recurrent neural networks to combine instance-level preprocessing and bag-level classiﬁcation in MIL setting.
Revised: Garcez & Zaverucha (2012) used recurrent neural networks to combine instance-level preprocessing and bag-level classiﬁcation in MIL setting.

Original: Comprehensive reviews of MIL approaches are provided in Amores (2013);
Revised: Comprehensive reviews of MIL approaches are provided in Amores (2013);

Original: Herrera et al.
Revised: Herrera et al.

Original: ; Soleimani & Miller (2017).
Revised: ; Soleimani & Miller (2017).

Original: Recently, deep learning models have been successfully applied for MIL framework (Zhu et al., 2017; Wu et al., 2015; Huang et al., 2013; Yan et al., 2016; Kotzias et al., 2014; Kraus et al., 2016) and these approaches are generally termed as deep multi-instance learning models.
Revised: Recently, deep learning models have been successfully applied for MIL framework (Zhu et al., 2017; Wu et al., 2015; Huang et al., 2013; Yan et al., 2016; Kotzias et al., 2014; Kraus et al., 2016) and these approaches are generally termed as deep multi-instance learning models.

Original: Most of these works use either convolutional neural networks or deep neural networks in their MIL framework for image annotation, labeling, segmentation or classiﬁcation tasks.
Revised: Most of these works use either convolutional neural networks or deep neural networks in their MIL framework for image annotation, labeling, segmentation or classiﬁcation tasks.

Original: Despite the popularity of deep models for MIL, there are few works which have extended deep MIL models for multivariate time series data.
Revised: Despite the popularity of deep models for MIL, there are few works which have extended deep MIL models for multivariate time series data.

Original: The goal of this paper is to propose and study deep multi-instance learning models for multivariate time series data.
Revised: The goal of this paper is to propose and study deep multi-instance learning models for multivariate time series data.

Original:  M ETHODS
Revised:  M ETHODS

Original: In this section, we will ﬁrst describe the Multi-instance learning framework, and then present our problem formulation and our proposed relational multi-instance learning models.
Revised: In this section, we will ﬁrst describe the Multi-instance learning framework, and then present our problem formulation and our proposed relational multi-instance learning models.

Original: 3. M ULTI -I NSTANCE L EARNING
Revised: 3. M ULTI -I NSTANCE L EARNING

Original: Multi-instance learning (MIL) is a form of weakly supervised learning where the training data is arranged in sets called bags , and a label is provided for the entire bag.
Revised: Multi-instance learning (MIL) is a form of weakly supervised learning where the training data is arranged in sets called bags , and a label is provided for the entire bag.

Original: The data points inside a bag are referred to as instances .
Revised: The data points inside a bag are referred to as instances .

Original: In the MIL framework, instance labels are not provided during training.
Revised: In the MIL framework, instance labels are not provided during training.

Original: The main goal of MIL is to learn a model based on the instances in the bag and the label of the bagto make bag-level and instance-level predictions.
Revised: The main goal of MIL is to learn a model based on the instances in the bag and the label of the bagto make bag-level and instance-level predictions.

Original: In this work, we only focus on the classiﬁcation task in MIL, leaving out other learning tasks such as regression.
Revised: In this work, we only focus on the classiﬁcation task in MIL, leaving out other learning tasks such as regression.

Original: Generally, two broad assumptions are considered to model the relationship between instance label and bag label.
Revised: Generally, two broad assumptions can be used to model the relationship between instance label and bag label.

Original: In the standard MIL assumption (Dietterich et al., 1997), the bag label is negative if all the instances in the bag have a negative label, and the bag label is positive if at-least one of the instances in the bag has a positive label.
Revised: In the standard MIL assumption (Dietterich et al., 1997), the bag label is negative if all the instances in the bag have a negative label, and the bag label is positive if at-least one of the instances in the bag has a positive label.

Original: Following the notations of Carbonneau et al.
Revised: Following the notations of Carbonneau et al.

Original: (2017), let X denote a bag with N feature vector instances
Revised: (2017), let X denote a bag with N feature vector instances

Original: i.e., X = { x 1 , ..., x N } .
Revised: i.e., X = { x 1 , ..., x N } .

Original: Let each instance x i in feature space X be mapped to a class by some process f : X → { 0 , 1 } , where 0 and 1 correspond to negative and positive labels respectively.
Revised: Let each instance x i in feature space X be mapped to a class by some process f : X → { 0 , 1 } , where 0 and 1 correspond to negative and positive labels respectively.

Original: The bag classiﬁer, also know as the aggregator function, g ( X ) is deﬁned by:
Revised: The bag classiﬁer, also know as the aggregator function, g ( X ) is deﬁned by:

Original: [Equation]
Revised: [Equation]

Original: otherwise
Revised: otherwise

Original: The standard assumption is quite restrictive for some problem settings, where the positive bags cannot be identiﬁed by a single instance.
Revised: The standard assumption is quite restrictive for some problem settings, where the positive bags cannot be identiﬁed by a single instance.

Original: Thus, this assumption can be relaxed to a collective assumption which says that several positive instances in a bag are necessary to assign a positive label to that bag.
Revised: Thus, this assumption can be relaxed to a collective assumption which says that several positive instances in a bag are necessary to assign a positive label to that bag.

Original: In this case, a bag classiﬁer is given by:
Revised: In this case, a bag classiﬁer is given by:

Original: [Equation]
Revised: [Equation]

Original: where θ is a threshold which indicates the minimum number of instances with positive labels that should be present in a bag to assign a positive label to that bag.
Revised: where θ is a threshold which indicates the minimum number of instances with positive labels that should be present in a bag to assign a positive label to that bag.

Original: As discussed in section 2, a plethora of works have adapted machine learning models to the MIL setting to optimize instance-level and/or bag-level predictions.
Revised: As discussed in section 2, a plethora of works have adapted machine learning models to the MIL setting to optimize instance-level and/or bag-level predictions.

Original: 3. P ROBLEM F ORMULATION
Revised: 3. P ROBLEM F ORMULATION

Original: We formulate the concept annotation task as the detection and localization of concepts given the medical time series data.
Revised: We formulate the concept annotation task as the detection and localization of concepts given the medical time series data.

Original: Let each patient i ∈ { 1 , .., N } be associated with a medical time series (also referred to as feature time series) denoted by X i ∈ R T × D , where D denotes the number of features (such as heart rate, blood pressure) and T denotes the length of time series observations (i.e., amount of time a patient is monitored).
Revised: Let each patient i ∈ { 1 , .., N } be associated with a medical time series (also referred to as feature time series) denoted by X i ∈ R T × D , where D denotes the number of features (such as heart rate, blood pressure) and T denotes the length of time series observations (i.e., amount of time a patient is monitored).

Original: Let C denote the set of all the concepts associated with the
Revised: Let C denote the set of all the concepts associated with the

Original: N patients, and Y i ∈ { 0 , 1 } K denote the concepts associated with i th patient where K = |C| .
Revised: N patients, and Y i ∈ { 0 , 1 } K denote the concepts associated with i th patient where K = |C| .

Original: Let
Revised: Let

Original: C i ∈ { 0 , 1 } T × K denote the concept time series of X i with C jki = 1 when concept k is present at time-stamp j .
Revised: C i ∈ { 0 , 1 } T × K denote the concept time series of X i with C jki = 1 when concept k is present at time-stamp j .

Original: In multi-instance learning settings, we treat each time series
Revised: In multi-instance learning settings, we treat each time series

Original: X i as one bag, and the observation at each time step j i.e. X ji ∈ R D as an instance in that bag.
Revised: X i as one bag, and the observation at each time step j i.e. X ji ∈ R D as an instance in that bag.

Original: We are interested in the following tasks:
Revised: We are interested in the following tasks:

Original: • Concept prediction task: for patient i , given X i , predict Y i .
Revised: • Concept prediction task: for patient i , given X i , predict Y i .

Original: • Concept localization task: for patient i , given X i , predict C i .
Revised: • Concept localization task: for patient i , given X i , predict C i .

Original: Notice that during training phase, only the input X i and prediction label Y i are available.
Revised: Notice that during training phase, only the input X i and prediction label Y i are available.

Original: Though C  is not known, we usually have some assumptions about the relationship of prediction and localization labels.
Revised: Though C i is not known, we usually have some assumptions about the relationship of prediction and localization labels.

Original: In this work, Y ki =
Revised: In this work, Y ki =

Original: I (cid:16)(cid:80) 1 ≤ j ≤ T C jki ≥ η (cid:17) , where I is an indicator function and η is a constant which depends on the MIL assumption.
Revised: I (cid:16)(cid:80) 1 ≤ j ≤ T C jki ≥ η (cid:17) , where I is an indicator function and η is a constant which depends on the MIL assumption.

Original: For example, in our concept annotation tasks we make the standard assumption, i.e we assume η = 1 , i.e. the time series label (bag label) for a concept is positive if that concept is present at any one time-stamp (at-least one instance has positive label).
Revised: For example, in our concept annotation tasks we make the standard assumption, i.e we assume η = 1 , i.e. the time series label (bag label) for a concept is positive if that concept is present at any one time-stamp (at-least one instance has positive label).

Original: 3. R ELATIONAL M ULTI -I NSTANCE L EARNING F RAMEWORK
Revised: 3. R ELATIONAL M ULTI -I NSTANCE L EARNING F RAMEWORK

Original: Inspired by the recent success of recurrent neural networks in sequence modeling Bahdanau et al.
Revised: Inspired by the recent success of recurrent neural networks in sequence modeling Bahdanau et al.

Original: ; Sutskever et al.
Revised: ; Sutskever et al.

Original: and classiﬁcation tasks Krizhevsky et al.
Revised: and classiﬁcation tasks Krizhevsky et al.

Original: (2012); Simonyan & Zisserman (2014), we adapt these models to the MIL framework to model multivariate time series data for concept annotation tasks.
Revised: (2012); Simonyan & Zisserman (2014), we adapt these models to the MIL framework to model multivariate time series data for concept annotation tasks.

Original: We denote all the variables at every time step as an instance and the entire multivariate time series as a bag.
Revised: We denote all the variables at every time step as an instance and the entire multivariate time series as a bag.

Original: Unlike the traditional MIL setting, where the instances within a bag are independent of each other, in our case, the instances have relationships (namely temporal dependencies) among them.
Revised: Unlike the traditional MIL setting, where the instances within a bag are independent of each other, in our case, the instances have relationships (namely temporal dependencies) among them.

Original: To model these dependencies, we propose to combine RNN models such as Long-Short Term Memory Neural Networks (LSTM) and Sequence-to-Sequence models with
Revised: To model these dependencies, we propose to combine RNN models such as Long-Short Term Memory Neural Networks (LSTM) and Sequence-to-Sequence models with

Original: MIL, and propose our R elational M ulti-
Revised: MIL, and propose our R elational M ulti-

Original: I nstance L earning framework, abbreviated as RMIL .
Revised: I nstance L earning framework, abbreviated as RMIL .

Original: RMIL takes in multivariate time series as input and outputs concept annotations.
Revised: RMIL takes in multivariate time series as input and outputs concept annotations.

Original: In RMIL, the outputs of RNN model provide the instance label predictions (i.e. solution for concept localization task) and the aggregation of the instance labels using aggregators such as pooling layer provides the bag label predictions (i.e. solution for concept prediction task).
Revised: In RMIL, the outputs of RNN model provide the instance label predictions (i.e. solution for concept localization task) and the aggregation of the instance labels using aggregators such as pooling layer provides the bag label predictions (i.e. solution for concept prediction task).

Original: We propose different pooling functions and attention mechanisms which can be easily incorporated into our RMIL to improve the concept annotations.
Revised: We propose different pooling functions and attention mechanisms which can be easily incorporated into our RMIL to improve the concept annotations.

Original: Pooling Layers for RMIL
Revised: Pooling Layers for RMIL

Original: The bag-level prediction is obtained by using an aggregation gathering on all instance-level predictions.
Revised: The bag-level prediction is obtained by using an aggregation gathering on all instance-level predictions.

Original: The aggregator function g ( · ) : [0 , 1] T (cid:55)→ [0 , 1] in RMIL can be modeled using the pooling layers.
Revised: The aggregator function g ( · ) : [0 , 1] T (cid:55)→ [0 , 1] in RMIL can be modeled using the pooling layers.

Original: Without loss of generality we assume that RNN model computes a mapping from the feature time series to the concept time series for each of the concept k ∈ C .
Revised: Without loss of generality we assume that RNN model computes a mapping from the feature time series to the concept time series for each of the concept k ∈ C .

Original: Let us denote the probability of an instance j belonging to concept k as p jk .
Revised: Let us denote the probability of an instance j belonging to concept k as p jk .

Original: Then, the bag level probability for a concept k is given by P k = g ( p 1 k , p 2 k , . .
Revised: Then, the bag level probability for a concept k is given by P k = g ( p 1 k , p 2 k , . .

Original: , p Tk ) .
Revised: , p Tk ) .

Original: The role of aggregator function g ( · ) is to combine the instance probabilities from each class speciﬁc feature map { p jk } into a single bag probability P k .
Revised: The role of aggregator function g ( · ) is to combine the instance probabilities from each class speciﬁc feature map { p jk } into a single bag probability P k .

Original: Several pooling mechanisms shown in Table 1 have been introduced in MIL and deep learning literature which can be used in our RMIL.
Revised: Several pooling mechanisms shown in Table 1 have been introduced in MIL and deep learning literature which can be used in our RMIL.

Original: In table 1, r , a , b k , and r k are parameters which can be ﬁxed or are learned during training, and σ ( · ) denotes the sigmoid function.
Revised: In table 1, r , a , b k , and r k are parameters which can be ﬁxed or are learned during training, and σ ( · ) denotes the sigmoid function.

Original: [Table] [Table] [Equation]
Revised: Pooling Functions Noisy-OR pooling

Original: [Table]
Revised: (Zhang et al., 2006)

Original: [Equation]
Revised: [Equation]

Original: [Table] [Table]
Revised: Log-sum-exponention (LSE) (Ramon & De Raedt, 2000)

Original: [Equation]
Revised: [Equation]

Original: [Table] [Table] et al., 1991)
Revised: Generalized Mean (GM) (Keeler et al., 1991)

Original: [Equation]
Revised: [Equation]

Original: [Table]

Original:  and [Table] [Table] ) (Keeler et al., 1991)
Revised: Integrated segmentation and recognition (ISR) (Keeler et al., 1991)

Original: [Equation]
Revised: [Equation]

Revised: Noisy

Original: [Table] [Table] et al., 2016)
Revised: -AND pooling (Kraus et al., 2016)

Original: [Equation]
Revised: [Equation]

Original: [Table]

Revised: Max pooling

Original: [Equation]
Revised: [Equation]

Original: [Table] [Equation] [Table]
Revised: Softmax

Original:  [Equation][Table]
Revised: pooling [Equation]

Original: Attention Mechanism for RMIL Instances within each bag have temporal relations between them.
Revised: Attention Mechanism for RMIL Instances within each bag have temporal relations between them.

Original: We can use attention mechanism to focus on some of the instances and their relations to improve their instance-level predictions.
Revised: We can use attention mechanism to focus on some of the instances and their relations to improve their instance-level predictions.

Original: In order to make predictions at time j , the hidden state h j ∈ R Q of
Revised: In order to make predictions at time j , the hidden state h j ∈ R Q of

Original: RNN can be used, where Q is the hidden state dimension.
Revised: RNN can be used, where Q is the hidden state dimension.

Original: However, relevant information may be captured by hidden states at other time steps as well.
Revised: However, relevant information may be captured by hidden states at other time steps as well.

Original: Thus, we may want to introduce an attention vector or matrix ( a ) to leverage information of hidden states H =
Revised: Thus, we may want to introduce an attention vector or matrix ( a ) to leverage information of hidden states H =

Original: ( h 1 , · · · , h T ) (cid:62) ∈ R T × Q from all time steps.
Revised: ( h 1 , · · · , h T ) (cid:62) ∈ R T × Q from all time steps.

Original: Let us denote the output after attention as ˜ H ∈ R T × Q .
Revised: Let us denote the output after attention as ˜ H ∈ R T × Q .

Original: The attention matrix can then be modeled using ˜ H in various ways as listed below.
Revised: The attention matrix can then be modeled using ˜ H in various ways as listed below.

Original: Feature-based Attention One idea is to design the attention matrix based on the feature and its time-stamp.
Revised: Feature-based Attention One idea is to design the attention matrix based on the feature and its time-stamp.

Original: Let us deﬁne a feature-based attention matrix as A
Revised: Let us deﬁne a feature-based attention matrix as A

Original: = ( a 1 , · · · , a T ) (cid:62) ∈ R T × Q .
Revised: = ( a 1 , · · · , a T ) (cid:62) ∈ R T × Q .

Original: For each j = 1 , · · · , T , we have
Revised: For each j = 1 , · · · , T , we have

Original: [Equation]
Revised: [Equation]

Original: and ˜ H =
Revised: and ˜ H =

Original: A (cid:12) H , where (cid:12) and (cid:11) are element-wise multiplication and division, respectively and
Revised: A (cid:12) H , where (cid:12) and (cid:11) are element-wise multiplication and division, respectively and

Original: W = ( w 1 , · · · , w T ) (cid:62) ∈ R T × T is the weight matrix which can be learned during training.
Revised: W = ( w 1 , · · · , w T ) (cid:62) ∈ R T × T is the weight matrix which can be learned during training.

Original: We call this Attention-F mechanism.
Revised: We call this Attention-F mechanism.

Original: We can simplify this attention by averaging the attentions for all hidden dimensions by taking a jq ← 1 /Q · (cid:80) 1 ≤ q (cid:48) ≤ Q a jq (cid:48) .
Revised: We can simplify this attention by averaging the attentions for all hidden dimensions by taking a jq ← 1 /Q · (cid:80) 1 ≤ q (cid:48) ≤ Q a jq (cid:48) .

Original: We will denote this as Attention-FS mechanism.
Revised: We will denote this as Attention-FS mechanism.

Original: Time-based Attention Attention model (Ma et al., 2017) can be designed to capture the relation between the current time step j and previous time steps j (cid:48) ≤ j , by solely relying on previous hidden states h j (cid:48) .
Revised: Time-based Attention Attention model (Ma et al., 2017) can be designed to capture the relation between the current time step j and previous time steps j (cid:48) ≤ j , by solely relying on previous hidden states h j (cid:48) .

Original: We can deﬁne a time-based attention matrix as A ∈ R T × T .
Revised: We can deﬁne a time-based attention matrix as A ∈ R T × T .

Original: For each j and j (cid:48) in [1 , · · · , T ] , we have
Revised: For each j and j (cid:48) in [1 , · · · , T ] , we have

Original: [Equation]
Revised: [Equation]

Original: otherwise.
Revised: otherwise.

Original: and ˜ H =
Revised: and ˜ H =

Original: A · H , where w ∈ R D is the weight vector to learn.
Revised: A · H , where w ∈ R D is the weight vector to learn.

Original: We use Attention-T to represent
Revised: We use Attention-T to represent

Original: Time-based attention mechanism.
Revised: Time-based attention mechanism.

Original: Interaction-based Attention
Revised: Interaction-based Attention

Original: The time-based attention can be further improved by considering both the previous and current hidden states h j (cid:48) and h j (Ma et al., 2017).
Revised: The time-based attention can be further improved by considering both the previous and current hidden states h j (cid:48) and h j (Ma et al., 2017).

Original: In this case, we have a j,j [Equation]
Revised: In this case, we have a j,j (cid:48) =

Original: =
Revised:  

Original: [Equation] [Table]
Revised: [Table] [Table]

Original: 0 , otherwise and

Original:  similarly ˜ H =
Revised: and similarly ˜ H =

Original: A · H .
Revised: A · H .

Original: Here, we need to learn v ∈ R S , W 1 ∈ R S × Q , W 2 ∈ R S × Q , and we choose S =
Revised: Here, we need to learn v ∈ R S , W 1 ∈ R S × Q , W 2 ∈ R S × Q , and we choose S =

Original: Q/ 2 .
Revised: Q/ 2 .

Original: A simpliﬁed version of interaction-based attention can be obtained if we use vector w 1 ∈ R Q , w 2 ∈ R Q instead of matrices W 1 , W 2 and by setting v = 1 in the above equation.
Revised: A simpliﬁed version of interaction-based attention can be obtained if we use vector w 1 ∈ R Q , w 2 ∈ R Q instead of matrices W 1 , W 2 and by setting v = 1 in the above equation.

Original: We use Attention-I and Attention-IS to represent Interaction-based attention mechanism and simpliﬁed version of interaction-based attention mechanism respectively.
Revised: We use Attention-I and Attention-IS to represent Interaction-based attention mechanism and simpliﬁed version of interaction-based attention mechanism respectively.

Original: The above attention mechanisms usually help both prediction and localization tasks.
Revised: The above attention mechanisms usually help both prediction and localization tasks.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Here, we demonstrate the performance of our proposed RMIL models on concept annotation tasks i.e. concept prediction and localization tasks, using a real-world health-care dataset and compare its performance to the popular multi-instance learning approaches.
Revised: Here, we demonstrate the performance of our proposed RMIL models on concept annotation tasks i.e. concept prediction and localization tasks, using a real-world health-care dataset and compare its performance to the popular multi-instance learning approaches.

Original: In addition, we discuss the impact of using pooling functions and attention mechanism in our RMIL framework
Revised: In addition, we discuss the impact of using pooling functions and attention mechanism in our RMIL framework.

Original: . 4. D ATASET D ESCRIPTIONS AND E XPERIMENTAL D ESIGN
Revised: 4. D ATASET D ESCRIPTIONS AND E XPERIMENTAL D ESIGN

Original: [Table]
Revised: [Table]

Original: To evaluate our RMIL, we ran experiments on MIMIC-III
Revised: To evaluate our RMIL, we ran experiments on MIMIC-III

Original: RESP dataset whose statistics is shown in Table 2.
Revised: RESP datasets whose statistics is shown in Table 2.

Original: MIMIC-III RESP Dataset MIMIC-III is a public dataset (Johnson et al., 2016) which has deidentiﬁed clinical care data collected at Beth Israel Deaconess Medical
Revised: MIMIC-III RESP Dataset MIMIC-III is a public dataset (Johnson et al., 2016) which has deidentiﬁed clinical care data collected at Beth Israel Deaconess Medical

Original: Center from 2001 to 2012.
Revised: Center from 2001 to 2012.

Original: It contains over 58,000 hospital admission records of 38,645 adults and 7,875 neonates.
Revised: It contains over 58,000 hospital admission records of 38,645 adults and 7,875 neonates.

Original: For our work, we extracted 21 feature time series from more than 2,000 adult patients who were diagnosed with a respiratory disorder such as Acute Hypoxemic Respiratory Failure (AHRF)
Revised: For our work, we extracted 21 feature time series from more than 2,000 adult patients who were diagnosed with a respiratory disorder such as Acute Hypoxemic Respiratory Failure (AHRF)

Original: (Khemani et al., 2009) at the time of admission.
Revised: (Khemani et al., 2009) at the time of admission.

Original: These 21 features are respiratory based features such as peak inspiratory pressure (PIP) and arterial partial pressure of oxygen (PaO2) and were collected during the ﬁrstdays after admission.
Revised: These 21 features are respiratory based features such as peak inspiratory pressure (PIP) and arterial partial pressure of oxygen (PaO2) and were collected during the ﬁrstdays after admission.

Original: The feature time series has 4 time stamps and the ﬁrst time stamp corresponds to the admission time.
Revised: The feature time series has 4 time stamps and the ﬁrst time stamp corresponds to the admission time.

Original: We denote this dataset as MIMIC-III RESP dataset.
Revised: We denote this dataset as MIMIC-III RESP dataset.

Revised: In addition, we also generated another feature time series with more time stamps whose results is shown in the appendix.

Original: Concept Annotations
Revised: Concept Annotations

Original: The medical time series data of MIMIC-III dataset does not come with the concept annotations, however the medical concepts are available in the doctor notes of the MIMIC-III database.
Revised: The medical time series data of MIMIC-III dataset does not come with the concept annotations, however the medical concepts are available in the doctor notes of the MIMIC-III database.

Original: To obtain the concept annotations, we extract the concept time series from the doctor notes using the NOTEEVENTS table of MIMIC-III database.
Revised: To obtain the concept annotations, we extract the concept time series from the doctor notes using the NOTEEVENTS table of MIMIC-III database.

Original: The total number of doctor notes is 2,083,180, out of which 98.15% of notes (2,044,634 notes) have no timestamp and 1.85% of notes (38,546 notes) have timestamps associated with them.
Revised: The total number of doctor notes is 2,083,180, out of which 98.15% of notes (2,044,634 notes) have no timestamp and 1.85% of notes (38,546 notes) have timestamps associated with them.

Original: The total number of unique concepts in the doctor notes in the ﬁrst 3 days data is 6,197.
Revised: The total number of unique concepts in the doctor notes in the ﬁrst 3 days data is 6,197.

Original: To obtain concept time series for each patient with respiratory disorder such as AHRF, we ﬁrst identiﬁed respiratory-related concepts from the medical literature (Antonelli et al., 2001; Khemani et al., 2009), and obtained their medical codes from the Uniﬁed Medical Language
Revised: To obtain concept time series for each patient with respiratory disorder such as AHRF, we ﬁrst identiﬁed respiratory-related concepts from the medical literature (Antonelli et al., 2001; Khemani et al., 2009), and obtained their medical codes from the Uniﬁed Medical Language

Original: System (UMLS) dictionary (Bodenreider, 2004).
Revised: System (UMLS) dictionary (Bodenreider, 2004).

Original: Then, we mined the patient’s doctor notes from
Revised: Then, we mined the patient’s doctor notes from

Original: NOTEEVENTS table to extract all the possible medical concepts related to the respiratory system and its disorder.
Revised: NOTEEVENTS table to extract all the possible medical concepts related to the respiratory system and its disorder.

Original: In total, we chose top 26 respiratory concepts to annotate the feature time series, and generated the concepts time series for 4 time-stamps (ﬁrst 3 days after admission) for each patient.
Revised: In total, we chose top 26 respiratory concepts to generate concept time series which has the same number of time stamps as feature time series. [Table]

Original: 4. C OMPARISON AND I MPLEMENTATION D ETAILS
Revised: 4. C OMPARISON AND I MPLEMENTATION D ETAILS

Original: We compare the performance of our proposed models to the popular MIL models such as MISVM (Andrews et al., 2003) and DPMIL (Kandemir & Hamprecht, 2014).
Revised: We compare the performance of our proposed models to the popular MIL models such as MISVM (Andrews et al., 2003) and DPMIL (Kandemir & Hamprecht, 2014).

Original: We categorize all the evaluated methods into two groups:
Revised: We categorize all the evaluated methods into two groups:

Original: [Table]

Original: Multi-Instance Learning models (MIL) :
Revised: Multi-Instance Learning models (MIL) :

Original: We treat MISVM and DPMIL as our baseline MIL models.
Revised: We treat MISVM and DPMIL as our baseline MIL models.

Original: Relational Multi-Instance Learning models (RMIL) :
Revised: Relational Multi-Instance Learning models (RMIL) :

Original: We evaluate the following deep learning models as part of our RMIL framework: (a) Long Short-Term Memory neural networks (LSTM) (Hochreiter & Schmidhuber, 1997) (b) Bi-directional LSTM (Bi-LSTM) (Graves et al., 2013) (c) Sequence to Sequence models (S2S) (Sutskever et al., 2014)
Revised: We evaluate the following deep learning models as part of our RMIL framework: (a) Long Short-Term Memory neural networks (LSTM) (Hochreiter & Schmidhuber, 1997) (b) Bi-directional LSTM (Bi-LSTM) (Graves et al., 2013) (c) Sequence to Sequence models (S2S) (Sutskever et al., 2014)

Original: (d) The above three RMIL models with different attention mechanisms (e) The above RMIL models with different pooling functions 
Revised: (d) The above three RMIL models with different attention mechanisms (e) The above RMIL models with different pooling functions For LSTM models, we use

Original: For LSTM models, we use two LSTM layers and two dense layers.
Revised: two LSTM layers and two dense layers.

Original: For S2S models, we use two
Revised: For S2S models, we use two

Original: LSTM layers for both the encoder and the decoder.
Revised: LSTM layers for both the encoder and the decoder.

Original: For Bi-LSTM models, we use two bi-directional
Revised: For Bi-LSTM models, we use two bi-directional

Original: LSTM layers.
Revised: LSTM layers.

Original: All the models were constructed to have a comparable number of parameters.
Revised: All the models were constructed to have a comparable number of parameters.

Original: We train all the Deep learning models with the RMSProp optimization method and we use early stopping to ﬁnd the best weights on the validation dataset.
Revised: We train all the Deep learning models with the RMSProp optimization method and we use early stopping to ﬁnd the best weights on the validation dataset.

Original: For baseline MIL models, we follow the suggestions of the corresponding papers to ﬁne-tune the parameters.
Revised: For baseline MIL models, we follow the suggestions of the corresponding papers to ﬁne-tune the parameters.

Original: All the input variables in the training data are normalized to be 0 mean and 1 standard deviation.
Revised: All the input variables in the training data are normalized to be 0 mean and 1 standard deviation.

Original: The inputs to all the models is the same feature time series data.
Revised: The inputs to all the models is the same feature time series data.

Original: We used Keras (Chollet, 2017) and Python to run the deep models and
Revised: We used Keras (Chollet, 2017) and Python to run the deep models and

Original: MISVM models.
Revised: MISVM models.

Original: Matlab code from the original authors was used to obtain DPMIL results.
Revised: Matlab code from the original authors was used to obtain DPMIL results.

Original: We use the area under ROC (AUROC) and area under precision-recall curve (AUPRC) scores as our evaluation metrics and report the results from 5-fold cross validation for all the evaluated methods.
Revised: We use the area under ROC (AUROC) and area under precision-recall curve (AUPRC) scores as our evaluation metrics and report the results from 5-fold cross validation for all the evaluated methods.

Original: 4. Q UANTITATIVE R ESULTS
Revised: 4. Q UANTITATIVE R ESULTS

Original: Table 3 shows the concept annotation results on the MIMIC-III RESP dataset.
Revised: Table 3 shows the concept annotation results on the MIMIC-III RESP dataset.

Original: From this table, we see that RMIL models outperform the non-deep multi-instance learning models by at least 8-10% for concept localization task, and by at least 10-15% for concept prediction tasks in terms of AUROC and AUPRC.
Revised: From this table, we see that RMIL models outperform the non-deep multi-instance learning models by at least 8-10% for concept localization task, and by at least 10-15% for concept prediction tasks in terms of AUROC and AUPRC.

Original: Among all the RMIL models, we ﬁnd that LSTM model obtains slightly better overall results compared to the other models for localization task.
Revised: Among all the RMIL models, we ﬁnd that LSTM model obtains slightly better overall results compared to the other models for localization task.

Original: To study the impact of pooling and attention, we trained and evaluated LSTM models with different pooling functions and different attention mechanisms, which are described in Section 3.
Revised: To study the impact of pooling and attention, we trained and evaluated LSTM models with different pooling functions and different attention mechanisms, which are described in Section 3.

Original: Tables 4 and 5 show the comparison results.
Revised: Tables 4 and 5 show the comparison results.

Original: From these tables, we observed that (i) all the attention mechanisms except feature-based attention perform similar to each other especially for the prediction task, and (ii) all the pooling functions other than ISR and Noisy-OR obtain similar overall performance.
Revised: From these tables, we observed that (i) all the attention mechanisms except feature-based attention perform similar to each other especially for the prediction task, and (ii) all the pooling functions other than ISR and Noisy-OR obtain similar overall performance.

Original: This demonstrates that choice of attention does not matter but choice of pooling has some impact in our
Revised: This demonstrates that choice of attention does not matter but choice of pooling has some impact in our

Original: RMIL framework.
Revised: RMIL framework.

Revised: [Table]

Revised: [Table]

Revised: [Table]

Original: 4. D ISCUSSIONS
Revised: 4. D ISCUSSIONS

Original: We can study the interpretability of concept localization by looking at the localization results of our
Revised: We can study the interpretability of concept localization by looking at the localization results of our

Original: RMIL models, even though the model is trained without the labels for localization.
Revised: RMIL models, even though the model is trained without the labels for localization.

Original: Figure 2 shows the ground truth annotations of two respiratory concepts - intubation and extubation concepts, and the prediction probabilities of these concepts obtained by our RMIL attention-based LSTM models.
Revised: Figure 2 shows the ground truth annotations of two respiratory concepts - intubation and extubation concepts, and the prediction probabilities of these concepts obtained by our RMIL attention-based LSTM models.

Original: From ﬁgure 2(a) we can make the following observations, (i) intubation usually happens before

Original: [Table]

Original: [Table] [Table] [Table] extubation for the same patient, (ii) intubation and extubation could happen on the same day, and (iii) intubation and extubation occur commonly within the 1st 24 hours of admission.
Revised: From ﬁgure 2(a) we can make the following observations, (i) intubation usually happens before extubation for the same patient, (ii) intubation and extubation could happen on the same day, and (iii) intubation and extubation occur commonly within the ﬁrst 24 hours of admission.

Original: From the ﬁgure 2(b), we see that our RMIL attention based LSTM predicts that the probability of intubation happening on the ﬁrst day of admission is higher (draker gray means higher probability of concept occurrence) and the probability of extubation happening within ﬁrst day is lower.
Revised: From the ﬁgure 2(b), we see that our RMIL attention based LSTM predicts that the probability of intubation happening on the ﬁrst day of admission is higher (draker gray means higher probability of concept occurrence) and the probability of extubation happening within ﬁrst day is lower.

Original: This indicates that the model has correctly learnt that intubation should appear before extubation.
Revised: This indicates that the model has correctly learnt that intubation should appear before extubation.

Original: This also implicitly implies that the
Revised: This also implicitly implies that the

Original: RMIL attention-based LSTM models have correctly learnt the instance-level relationships from the medical time series data with only bag-level labels. 
Revised: RMIL attention-based LSTM models have correctly learnt the instance-level relationships from the medical time series data with only bag-level labels. [Figure]

Original:  S UMMARY
Revised:  S UMMARY

Original: In this paper, we presented Relational Multi-Instance Learning - a deep multi-instance learning framework using recurrent neural networks for concept annotation from the medical time series data.
Revised: In this paper we presented Relational Multi-Instance Learning - a deep multi-instance learning framework using recurrent neural networks for concept annotation from the medical time series data.

Original: Empirical results on medical dataset demonstrated that our proposed models outperform the popular state-of-the-art multi-instance learning approaches.
Revised: Empirical results on medical dataset demonstrated that our proposed models outperform the popular state-of-the-art multi-instance learning approaches.

Original: Experiments with different pooling and attention mechanisms showed that while attention mechanism does not have a signiﬁcant impact on model’s performance, certain pooling functions such as ISR and Noisy-OR can negatively impact the instance prediction results.[Figure]
Revised: Experiments with different pooling and attention mechanisms showed that while attention mechanism does not have a signiﬁcant impact on model’s performance, certain pooling functions such as ISR and Noisy-OR can negatively impact the instance prediction results.

Original:  A PPENDIX 6. MIMIC-III C ONCEPT L IST
Revised:  A PPENDIX 6. MIMIC-III C ONCEPT L IST

Original: Table 6 show the concept list of the MIMIC-III RESP dataset used in our experiments [Table]
Revised: Table 6 show the concept list of the MIMIC-III RESP dataset used in our experiments.

Original: pressureBiomechanical complianceTachycardia ventricularScoliosisAcute lung injuryPartial pressure of carbon dioxide in arterial bloodResuscitateSudden onset

Original: 6. A NOMALY D ETECTION FOR T IME S ERIES
Revised: 6. A NOMALY D ETECTION FOR T IME S ERIES

Original: Here we will demonstrate anomaly detection from medical time series data as an use case application of our RMIL framework
Revised: Here, we will demonstrate anomaly detection from medical time series data as a use case application of our RMIL framework.

Original: 6.2. M OTIVATION AND R ELATED W ORK
Revised: 6.2. M OTIVATION AND R ELATED W ORK

Original: Each year, more than 1,000,000 US adults and children are put on mechanical ventilation during their stays in ICU.
Revised: Each year, more than 1,000,000 US adults and children are put on mechanical ventilation during their stays in ICU.

Original: However, lack of effective tools to aid with ventilator weaning and extubation (removal of the breathing tube) readiness assessment results in nearly half of the patients spending unnecessary days on ventilators (Randolph et al., 2002), and up to 20% of them having ventilators discontinued too soon (Kurachek et al., 2003).
Revised: However, lack of effective tools to aid with ventilator weaning and extubation (removal of the breathing tube) readiness assessment results in nearly half of the patients spending unnecessary days on ventilators (Randolph et al., 2002), and up to 20% of them having ventilators discontinued too soon (Kurachek et al., 2003).

Original: Spending unnecessary days on ventilators can lead to hospital-acquired infections while having ventilators discontinued too soon could require painful reintubation.
Revised: Spending unnecessary days on ventilators can lead to hospital-acquired infections while having ventilators discontinued too soon could require painful reintubation.

Original: A work-of-breathing measure called Pressure-Rate Product (PRP), calculated from esophageal pressure, has shown potential to be used as a guideline for ventilator weaning and extubation readiness assessment (Willis et al., 2005).
Revised: A work-of-breathing measure called Pressure-Rate Product (PRP), calculated from esophageal pressure, has shown potential to be used as a guideline for ventilator weaning and extubation readiness assessment (Willis et al., 2005).

Original: However, PRP calculations are susceptible to sensor artifacts and breathing pattern anomalies.
Revised: However, PRP calculations are susceptible to sensor artifacts and breathing pattern anomalies.

Original: These anomalies limit realtime use of PRP for clinical decision making.
Revised: These anomalies limit realtime use of PRP for clinical decision making.

Original: Our goal is to automatically detect and remove these anomalies by using our proposed Relational Multi-instance learning models, thereby enabling real-time clinical decision making.
Revised: Our goal is to automatically detect and remove these anomalies by using our proposed Relational Multi-instance learning models, thereby enabling real-time clinical decision making.

Original: Figure 3 shows the example of anomaly detection from the ventilator time series data.
Revised: Figure 3 shows the example of anomaly detection from the ventilator time series data.

Original: Here, anomalies appear due to patient factors (cough, movement) and instrument factors (probing, catherter drift), and should be automatically detected from the ventalitor monitoring time series data.
Revised: Here, anomalies appear due to patient factors (cough, movement) and instrument factors (probing, catherter drift), and should be automatically detected from the ventalitor monitoring time series data.

Original: There is a long history and a rich body of research work on anomaly detection in time series data.
Revised: There is a long history and a rich body of research work on anomaly detection in time series data.

Original: See Hodge & Austin (2004); Chandola et al.
Revised: See Hodge & Austin (2004); Chandola et al.

Original: (2009) for a quick survey on generic anomaly detection  algorithms.
Revised: (2009) for a quick survey on generic anomaly detection [Table] algorithms.

Original: Gupta et al.
Revised: Gupta et al.

Original: (2014) also author a survey on anomaly detection for time series data.
Revised: (2014) also author a survey on anomaly detection for time series data.

Original: In their follow-up survey, Chandola et al.
Revised: In their follow-up survey, Chandola et al.

Original: (2012) gave a summary on discrete sequence anomaly detection algorithms.
Revised: (2012) gave a summary on discrete sequence anomaly detection algorithms.

Original: Unfortunately, techniques for time series anomaly detection are quite domain-speciﬁc.
Revised: Unfortunately, techniques for time series anomaly detection are quite domain-speciﬁc.

Original: This is due to highly-varied nature of time series characteristics.
Revised: This is due to highly-varied nature of time series characteristics.

Original: In Chan & Mahoney (2005), a set of k minimum bounding rectangles between each time step is used as a model of normal data generating distribution.
Revised: In Chan & Mahoney (2005), a set of k minimum bounding rectangles between each time step is used as a model of normal data generating distribution.

Original: In Camerra et al.
Revised: In Camerra et al.

Original: ; Lin et al. (2003), continuous time series are converted to ordinal symbolic representation, allowing faster approximation of Euclidean distance between time series windows and early pruning.
Revised: ; Lin et al. (2003), continuous time series are converted to ordinal symbolic representation, allowing faster approximation of Euclidean distance between time series windows and early pruning.

Original: Then, anomaly detection can be performed by setting a threshold on distance value.
Revised: Then, anomaly detection can be performed by setting a threshold on distance value.

Original: Recently, Wang et al.
Revised: Recently, Wang et al.

Original: (2016) proposes a self-learning method that learns clusters of constrained grammars based on ordinal symbolic approximation of time series value.
Revised: (2016) proposes a self-learning method that learns clusters of constrained grammars based on ordinal symbolic approximation of time series value.

Original: Guigou et al.
Revised: Guigou et al.

Original: (2017) proposes incorporating experts into anomaly detection in a method inspired by immune system.
Revised: (2017) proposes incorporating experts into anomaly detection in a method inspired by immune system.

Original: In Jones et al.
Revised: In Jones et al.

Original: (2016), the authors propose extracting exemplars from Euclidean pairwise distance as a way to speed up anomaly detection algorithm.
Revised: (2016), the authors propose extracting exemplars from Euclidean pairwise distance as a way to speed up anomaly detection algorithm.

Original: Most of these techniques for time series anomaly detection are quite domain-speciﬁc, and few of them model the anomaly detection problem in Multi-instance learning setting.
Revised: Most of these techniques for time series anomaly detection are quite domain-speciﬁc, and few of them model the anomaly detection problem in Multi-instance learning setting.

Original: 6.2. P ROBLEM F ORMULATION
Revised: 6.2. P ROBLEM F ORMULATION

Original: We formulate the anomaly detection as concept annotation problem in MIL setting, where the bag corresponds to the medical time series data X i and an instance is the features at a time-stamp, and the anomaly corresponds to a concept.
Revised: We formulate the anomaly detection as concept annotation problem in MIL setting, where the bag corresponds to the medical time series data X i and an instance is the features at a time-stamp, and the anomaly corresponds to a concept.

Original: Thus, the prediction (predicting Y i ) and localizaton tasks (predicting C i ) correspond to predicting the presence and location of the anomalies in the time series data.
Revised: Thus, the prediction (predicting Y i ) and localizaton tasks (predicting C i ) correspond to predicting the presence and location of the anomalies in the time series data.

Original: For anomaly detection, we assume Y ki
Revised: For anomaly detection, we assume Y ki

Original: =
Revised: =

Original: I (cid:16)(cid:80) 1 ≤ j ≤ T C jki ≥ η (cid:17) , where η is chosen as η = 0 .
Revised: I (cid:16)(cid:80) 1 ≤ j ≤ T C jki ≥ η (cid:17) , where η is chosen as η = 0 .

Original: 2 T and T is the number of time series windows within the bag.
Revised: 6 T and T is the number of time series windows within the bag.

Original: 6.2. E XPERIMENTS
Revised: 6.2. E XPERIMENTS

Original: We conduct anomaly detection experiments on a PICU dataset described below.
Revised: We conduct anomaly detection experiments on a PICU dataset described below.

Original: [Table]
Revised: [Table]

Original: PICU Dataset This dataset consists of 1,530 recorded 5minute 200-Hz sensor readings from 385 subjects collected at a leading children’s hospital by a team of research clinicians.
Revised: PICU Dataset This dataset consists of 1,530 recorded 5minute 200-Hz sensor readings from 385 subjects collected at a leading children’s hospital by a team of research clinicians.

Original: Statistics of the dataset is shown in Table 7.
Revised: Statistics of the dataset is shown in Table 7.

Original: The recordings are made on mechanically-ventilated patients in pediatrics ICU ward.
Revised: The recordings are made on mechanically-ventilated patients in pediatrics ICU ward.

Original: Medical time series data collected from four sensors are used: ﬂow volume spirometry, esophageal pressure sensor, and dual band respiratory inductance plethysmography.
Revised: Medical time series data collected from four sensors are used: ﬂow volume spirometry, esophageal pressure sensor, and dual band respiratory inductance plethysmography.

Original: Each subject can be under one of four breathing conditions: ventilated with Continuous Positive Airway Pressure (CPAP), ventilated with Pressure Support (PS), 5 minutes after extubation, and 60 minutes after extubation.
Revised: Each subject can be under one of four breathing conditions: ventilated with Continuous Positive Airway Pressure (CPAP), ventilated with Pressure Support (PS), 5 minutes after extubation, and 60 minutes after extubation.

Original: Along with the 4 sensor signals, clinician veriﬁed binary anomaly label generated using hard-crafted state-of-the-art breathing anomaly detection algorithm is provided as ground truth.
Revised: Along with the 4 sensor signals, clinicians veriﬁed binary anomaly label generated using hard-crafted state-of-the-art breathing anomaly detection algorithm is provided as ground truth.

Original: We annotate the concept of breathing anomaly in MIL framework by processing the dataset as follows.
Revised: We annotate the concept of breathing anomaly in MIL framework by processing the dataset as follows.

Original: The sensor recordings are split into non-overlapping 5-second windows.
Revised: The sensor recordings are split into non-overlapping 5-second windows.

Original: In MIL framework, each window becomes an instance and each recording becomes a bag.
Revised: In MIL framework, each window becomes an instance and each recording becomes a bag.

Original: For each sensor signal of each window, we extract 20 Mel-Frequency Cepstral Coefﬁcients (MFCCs) to be used as features of each instance; thus, each instance has 80 features in total.
Revised: For each sensor signal of each window, we extract 20 Mel-Frequency Cepstral Coefﬁcients (MFCCs) to be used as features of each instance; thus, each instance has 80 features in total.

Original: For each instance, its anomaly label is set to positive if at least 20% of the window are labeled as anomaly.
Revised: For each instance, its anomaly label is set to positive if at least 20% of the window are labeled as anomaly.

Original: The bag anomaly label is set to positive if at least 60% of its instances are labeled as anomaly.
Revised: The bag anomaly label is set to positive if at least 60% of its instances are labeled as anomaly.

Original:  Results Table 8 shows the anomaly detection results using our RMIL models.
Revised: [Table] Results Table 8 shows the anomaly detection results using our RMIL models.

Original: We observe that (i)
Revised: We observe that (i)

Original: RMIL models mostly outperform the non-deep MIL models on both tasks, (ii) LSTM and Bi-LSTM based RMIL models obtain better overall results compared to the sequence-to-sequence models, and (iii) the attention mechanism does not help for localization task but sometimes obtains better results for the prediction task [Table] [Table]
Revised: RMIL models mostly outperform the non-deep MIL models on both tasks, (ii) LSTM and Bi-LSTM based RMIL models obtain better overall results compared to the sequence-to-sequence models, and (iii) the attention mechanism does not help for localization task but sometimes obtains better results for the prediction task.

Original: Remark: Cluster-MIL* is a HDBSCAN clustering (Campello et al., 2015)
Revised: Remark: Cluster-MIL* is a HDBSCAN clustering (Campello et al., 2015)

Original: based approach for anomaly detection.
Revised: based approach for anomaly detection.

Original: It uses both instance and bag labels for training, while other models only use bag labels for training.
Revised: It uses both instance and bag labels for training, while other models only use bag labels for training.

Revised: 6. A DDITIONAL R ESULTS ON MIMIC-III RESP DATASET

Revised: We sampled the feature time series from MIMIC-III RESP dataset every 6 hours and generated a time series with more (12) time stamps.

Revised: We call this MIMIC-III RESP-II dataset.

Revised: Table 9 shows the results of RMIL models on this dataset.

Revised: We observe that (i) All the RMIL models have similar performance for prediction task, (ii) Bi-LSTM RMIL has better localization results compared to other the models.

Revised: [Table]