Paper:
Recent advances in computing technology and sensor design have made it easier to collect longitudinal or time series data from patients, resulting in a gigantic amount of available medical data. Most of the medical time series lack annotations or even when the annotations are available they could be subjective and prone to human errors. Earlier works have developed natural language processing techniques to extract concept annotations and/or clinical narratives from doctor notes. However, these approaches are slow and do not use the accompanying medical time series data. To address this issue, we introduce the problem of concept annotation for the medical time series data, i.e., the task of predicting and localizing medical concepts by using the time series data as input. We propose Relational Multi-Instance Learning (RMIL) -a deep Multi Instance Learning framework based on recurrent neural networks, which uses pooling functions and attention mechanisms for the concept annotation tasks. Empirical results on medical datasets show that our proposed models outperform various multi-instance learning models.

Review:
The paper addresses the classification of medical time-series data by formulating the problem as a multi-instance learning (MIL) task, where there is an instance for each timestep of each time series, labels are observed at the time-series level (i.e. for each bag), and the goal is to perform instance-level and series-level (i.e. bag-level) prediction.  The main difference from the typical MIL setup is that there is a temporal relationship between the instances in each bag.  The authors propose to model this using a recurrent neural network architecture.  The aggregation function which maps instance-level labels to bag-level labels is modeled using a pooling layer (this is actually a nice way to describe multi-instance classification assumptions using neural network terminology).  An attention mechanism is also used.
The proposed time-series MIL problem formulation makes sense.  The RNN approach is novel to this setting, if somewhat incremental.  One very positive aspect is that results are reported exploring the impact of the choice of recurrent neural network architecture, pooling function, and attention mechanism.  Results on a second dataset are reported in the appendix, which greatly increases confidence in the generalizability of the experiments.  One or more additional datasets would have helped further solidify the results, although I appreciate that medical datasets are not always easy to obtain.  Overall, this is a reasonable paper with no obvious major flaws.  The novelty and impact may be greater on the application side than on the methodology side.
Minor suggestions:
-The term "relational multi-instance learning" seems to suggest a greater level of generality than the work actually accomplishes.  The proposed methods can only handle time-series / longitudinal dependencies, not arbitrary relational structure.  Moreover, multi-instance learning is typically viewed as an intermediary level of structure "in between" propositional learning (i.e. the standard supervised learning setting) and fully relational learning, so the "relational multi-instance learning" terminology sounds a little strange. Cf.:
De Raedt, L. (2008). Logical and relational learning. Springer Science & Business Media.
-Pg 3, a capitalization typo: "the Multi-instance learning framework"
-The equation for the bag classifier on page 4 refers to the threshold-based MI assumption, which should be attributed to the following paper:
Weidmann, N., Frank, E. & Pfahringer, B. 2003. A two-level learning method for generalized multi-instance problems. In Proceedings of the 14th European Conference on Machine Learning,
Springer, 468-479.
(See also: J. R. Foulds and E. Frank. A review of multi-instance learning assumptions. Knowledge Engineering Review, 25(1):1-25, 2010. )
- Pg 5, "Table 1" vs "table 1" - be consistent.
-A comparison to other deep learning MIL methods, i.e. those that do not exploit the time-series nature of the problem, would be valuable.  I wouldn't be surprised if other reviewers insist on this.