DROPS: Deep Retrieval of Physiological Signals via Attribute-specific Clinical Prototypes
Authors: Dani Kiyasseh
Tingting Zhu
David A. Clifton
Venue:  (2021)
Keywords: Contrastive learning
information retrieval
clustering
physiological signals
healthcare

Abstract: The ongoing digitization of health records within the healthcare industry results in large-scale datasets. Manually extracting clinically-useful insight from such datasets is non-trivial. However, doing so at scale while simultaneously leveraging patient-specific attributes such as sex and age can assist with clinical-trial enrollment, medical school educational endeavours, and the evaluation of the fairness of neural networks. To facilitate the reliable extraction of clinical information, we propose to learn embeddings, known as clinical prototypes (CPs), via supervised contrastive learning. We show that CPs can be efficiently used for large-scale retrieval and clustering of physiological signals based on multiple patient attributes. We also show that CPs capture attribute-specific semantic relationships.

Under review as a conference paper at ICLR 2021
DROPS: DEEP RETRIEVAL OF PHYSIOLOGICAL SIG- NALS VIA ATTRIBUTE-SPECIFCC CLINICAL PROTO- TYPES
Anonymous authors Paper under double-blind review
ABSTRACT
The ongoing digitization of health records within the healthcare industry results in large-scale datasets. Manually extracting clinically-usefin insight from such datasets is non-trivial. However, doing so at scale while simultaneously leveraging patient-specific attributes such as sex and age can assist with clinical-trial enroll- ment, medical school educational endeavours, and the evaluation of the fairness neural networks. To facilitate the reliable extraction of clinical information, we propose to learn embeddings, known as clinical prototypes (CPs), via supervised contrastive learning. We show that CPs can be efficiently used for large-scale retrieval and clustering of physiological signals based on multiple patient attributes. We also show that CPs capture autribute-specfie semantic relationships.
INTRODUCTION
Physiological data are being collected at a burgeoning rate. Such growth driven by the digitization of previous patient records, the presence of novel health monitoring and recording systems, and the recent recommendation to facilitate the exchange of health records (European Commission (019). This engenders large-scale datasets from which the manual extraction of clinically useful insight is non-trivial. Such insight can include, but is not limited to, medical diagnoses, prognoses, or treatment.
In the presence of large-scale datasets, retrieving instances based on some user-defined criteria has been longstanding goal within the machine learning community (Manning et al., 2008). This information retrieval (IR) process typically consists of a query that is used to search through large database and retrieve matched instances. Within healthcare, the importance of an IR system is threefold (Hersh & Hickam 1998 Hersh 2008). First, provides researchers with greater control over which patients to choose for clinical trial recruitment. Second, IR systems can serve as an educational and diagnostic tool, allowing physicians to identify seemingly similar patients who exhibit different clinical parameters and vice versa. Lastly, if the query attributes such as sex, age, and race, then such a system would allow researchers to more reliably evaluate the individual and counterfactual fairness of particular model (Verma & Rubin 2018). To illustrate this point, let us assume the presence of a query instance that corresponds to patient with an abnormality of the heart, atrial fibrillation, who male and under the age 25. To reliably determine the sensitivity of a model with respect to sex, one would observe its response when exposed to counterfactual instance, namely the exact same instance but with a different sex label (Kusner et al., 2017). At present, deep-learning based IR systems within the healthcare domain fail to incorporate such patient-specific attributes.
to consist of sensitive
Existing IR systems which retrieve instances from the electronic health records (Wang al. 2019 Chamberlin al. 2019) do not incorporate an sttribute-speeife search and not trivially extend to physiological signals. In this paper, propose to learn embeddings, referred to as clinical prototypes (CPs). CPs are efficient descriptors and age. We learn these embeddings via contrastive learning whereby representations of instances encouraged to be similar to their corresponding clinical prototype and dissimilar to the others. To the best of our knowledge, retrieval system for electrocardiogram (ECG) signals.
combination of patient-specice attributes, such as discase, sex,
the first to design supervised contrastive learning based large-scale
Contributions, Our contributions are the following:

---

Under review as a conference paper at ICLR 2021
Attribute-specific clinical prototypes propose supervised contrastive learning framework to learn embeddings, referred to as clinical prototypes (CPs), that are efficient descriptors set of patient attributes, e.g., disease, sex, and age.
Deep retrieval and clustering we exploit CPs to retrieve instances corresponding to a specific patient-attribute combination and assign instances various clusters.
2 RELATED WORK
Clinical representation learning involves meaningfully representing clinical data for solving tasks.
Most research attempts to learn representations of electronic health records (EHRs) (Miotto et al. 2016: Gee et al., 2019; Liu al. 2019; Li et al., 2020b Biswal et al. 2020: Darabi et al. 2020) in a generative manner. For example, Landi et al. (2020) and Huang et al. (019)) implement an autoencoder to learn patient representations. These representations are then clustered either in hierarchical manner or via K-means, Other methods involve learning prototypes. For example, Li et al. (2020a) propose to do via the ProtoNCE loss. Our approach, unlike theirs, exploits readily-available patient-attribute data and is not dependent upon the K-means algorithm. Moreover, Van Looveren & Klaise (2099) learn perturb prototypes derive interpretable counterfactual instances. Most similar to our work is that of Kiyasseh et al. (2020). which learns patient-speciff
representations while pre-trae pre-training via contrastive learning. and Garnot & Landrieu (2020) where the
distance between class prototypes learned in an end-to-end manner is regularized based on a pre- defined tree hierarchy. contrast, we learn attribute-speciff prototypes via supervised contrastive learning and capture their semantic relationships via distance- based regularization.
Clinical information retrieval whereby instances similar to query are retrieved was first introduced in 1990 (Hersh & Greenes 1990). Most research in this domain revolves around text (Gurulingappa
et al. [016:[aang et al. 201 Rhine 2017: Wallace et al. 2016). For example, D Avolio al. (2010)
map text to SNOMED concepts retneve clinical documents. More recently, IR has been performed
with biomedical images, and is referred to as content-
image retrieval (Saritha et al. 2019
Chittajallu et al.
2019). Others have extended this concept EHR data (Goodwin & Harabagiu
2018: Wang et al. 2019 Chamberlin et al. 2019). For example, Chamberlin et al. (2019) implement rudimentary IR methods such as divergence from randomness on the UPMC and MIMIC III (Johnson ct al. 2016) datasets with the aim of discovering patient cohorts. In contrast to such methods, we implement deep-learning based clinical information retrieval system for physiological signals.
METHODS
3.1 SERIIBTTESSPECHHE CLINICAL PROTOTYPES
Information retrieval systems typically necessitate a query of some sort that is exploited to search through large database and retrieve instances that satisfy criteria outlined by the initial query. Such a query can take on a multitude of forms (e.g., text, image, audio, etc.) depending on the modality of instances in the database. As are primarily interested in large databases comprising physiological signals, we design a query that is based on such signals. Moreover, the type and specificity of instances that are retrieved highly depend on the criteria outlined by query. In our context, these criteria comprise patient attribute information such as disease class, sex, and age. As a result, our query should be capable of retrieving physiological instances that are associated with the aforementioned patient attributes. To achieve this, we propose to learn set of query embeddings, P. analogous to word embeddings in natural language processing. where |P| M, representing each of the M possible patient-atrribut combinations within dataset. Each embedding, PA E P. is an efficient descriptor of a set of attributes A (or1, 022, 03) where an disease class, Q2 = sex and a3 age. Given this stribute-specific interpretation, refer to such embeddings as attribute- specific clinical prototypes or CPs. We propose to learn such CPs, in an end-to-end manner, via contrastive learning, explained next.
3.2 LEARNING ATTRIBCTE-SPECIT CLINICAL PROTOTYPES
We assume the presence of a learner, fo x E R + v E F parameterized by 6, that maps a D D-dimensional input, x, to E-dimensional representation, In information retrieval systems,
2

---

Under review as a conference paper at ICLR 2021
reliable query should accurately retrieve instances that satisfy certain criteria. When dealing with representations, such reliability can be designed for by ensuring that the query embedding is in a similar subspace, and thus in proximity, to the representations of instances that satisfy said criteria. To achieve this proximity, we can attract query embeddings to representations in the database which share patient attribute information and repel them from those which do not. More specifically, attract each representation of an instance associated with set of attributes, A, to the clinical prototype, that describes that same set. This can be achieved in two ways.
Hard assignment. We encourage each representation, v; - [xx)), of an instance, Iú, associated with particular set of attributes, A, to be similar to the single clinical prototype, PAs that describes the exact same set of attributes, and dissimilar to the remaining clinical prototypes, Py. where j * A. We quantify this similarity, a(1; Pa), using the cosine similarity, with a temperature parameter, Ts- By mapping each representation to a single CP, we refer to this as hard assignment. To encourage this behaviour, one would optimize the following objective function for mini-batch of size, B:
B Σ B
1
8 (A.. = 1/3 P
Coontra - hard
log
(1)
(2)
galer.p))
|0 |A
The hard assignment approach assumes a bijective relationship between each representation, V,, and each clinical prototype, P.a- This many-to-one mapping implies that CPs without perfect attribute match (i.e., a near miss) do not leverage potentially useful information from representations that exhibit some, albeit imperfect, overlap in patient attributes.
Soft assignment. To overcome the limitations of a hard assignment, we propose soft assignment approach whereby each representation, v; attracted to subset of disease class-specific clinical pro- totypes, L C. P. We opted for this class-specific setup to avoid erroneously attracting representations CPs from different class. This attraction would hinder retrievals based on disease class.
Recall, though, that the CPs in L still describe different sets of non-class attributes. By attracting each representation to these CPs uniformly, CPs within class will collapse to a single point, and thus be unable to distinguish between various patient attributes. We support this claim in Fig. [a where we illustrate the t-SNE projection of the CPs for five different disease classes. To avoid such collapse, we modulate the attraction between each representation, e;. associated with the attribute set, A,, and each clinical prototype, Pk. associated with the attribute set, AІ. More specifically, we introduce weight, Wik. that is dependent the discrepancy between the respective attribute sets, d(A,, As). This formulation, characterized by greater attraction between CPs and representations with more similar attribute sets, results in the CPs shown in Fig. 1b following objective function.
Formally, we optimize the
B
M
Leantra-son saft =
Was log
(3)
B
Σ es(ue.p))
a(AriA)) Σ 14T ((A
if disease class; disease class;:
Wik
(4)
otherwise
d(A,,AA) [8(disease class; discase class;) + 5((ex, = sexx) + 5(age, = age:))]
(5)
Tw
where d is the Kronecker delta function that evaluates one if the argument is true and zero otherwise and Tu is a temperature parameter that determines how soft the assignment is. For example, as + O. the loss-term approaches the hard assignment formulation.
Arrangement of clinical prototypes. Clinical prototypes, learned in an end-to-end manner, as presented will exhibit high and desirable degree of inter-class separability. However, prototypes within a class are still at risk of collapsing to select few points. This would decrease their utility for attribute-basse based querying of instances. We would prefer to learn CPs that not only distinguish between different sets of attributes, but also reflect the semantic relationships between one another, as is done with word embeddings natural language processing. Capturing semantic relationships is desirable as it allows for improved interpretability of the retrieval process. For example, representations that are equidistant from two CPs can indicate a graded difference in particular attribute such as age. In Fig. 2 (left), we illustrate the desired arrangement of 16 CPs for two arbitrary classes.
3

---

Under review as a conference paper at ICLR 2021
Uniform Weighting
Modulated Weighting
Modulated Weighting with Regularization
collapse to single point
Class
Increasing Age
Male
Cless
Class
Female
intra-class
Class
Class 2
separation
(a)
(b)
(c)
Figure I: (a) Uniform weighting between each representation and CPs in the same class results in the collapse of CPs to single point. (b) Proposed modulated weighting introduces some desirable CP intra-class separation. (c) Modulated weighting in addition to the regularization term exhibits healthy inter and intra-class separation in addition to reflecting the semantic relationships between CPs.
To arrive at this arrangement, propose a distance- based regularization term, derived as follows. First, we use the L2 norm normalize CPs in the set, P, and calculate the empirical pairwise Euclidean distance between them. As result, we generate the matrix Á E RMMx (Fig. 2 right). As we are only interested in the pairwise distances for prototypes within the same class, we locus on the sub-matrices, D. E IR M/CxM//C with [1...., C1 where C' is the number of class-specific clusters. Each class' corresponding ground-truth sub-matrix, De. is populate lated with distances, dg, that reflect the semantic relationships between CPs. We choose dg 3 x S where 3 R is user-defined distance and S E 2 is the integer distance between attributes. For example, pair of CPs that differ according sex with A (AFIB, M, 25} and A₂ {AFIB, F, < 25), results in S 1 and dæ 3 1. We then calculate the mean-squared error between D. and De Ve E [1, [1,.... CJ. Our final objective function thus includes the soft contrastive loss (Eq. 3 and the regression loss.
C M
Greg
Dr. Dz)²
(6)
Leomb Leontra-soff soft + freg
(7)
Arrangement of Clinical Prototypes
Pairwise Distance Matrix
Female
Clinical Prototypes
Male
Female
D; Ground-truth intra-cluster distances
Increasing Age
nter-cluster distances
8 D2
Cless
Class
Inter-cluster distances
Ground-truth intra-cluster distances
Attractive Force
Instance Representation
Repuisive Force
Figure 2: (Left) Visualization of the attractive and repulsive forces present between an instance representation and clinical prototypes that belong to two different clusters/classes. The clinical prototypes are coloured according to sex and shaded in ascending order of age. All repulsive forces are treated equally whereas attractive forces are weighted according to the degree of attribute matching between the instance and the respective clinical prototype. (Right) Matrix Ò E T [ illustrating the Euclidean distance between each pair of clinical prototypes. The mean- mean-squared error between the observed intra-cluster distances, De. and the ground-truth intra-cluster distances, D is minimized.
4

---

Under review as a conference paper at ICLR 2021
4 EXPERIMENTAL DESIGN
4.1 DATASETS
We conduct our experiments using PyTorch (Paszke et al. 2019) on two large-scale datasets that con- sist of physiological time-series, such the electrocardiogrann (ECG), alongside cardiac arrhythmia labels. Chapman (Zheng et al. 2020) consists of 12-lead ECG recordings from 10,646 patients alongside labels which we group into 4 major classes. PTB-XL (Wagner et al. 2020) consists of 12-lead ECG recordings from 18,885 patients alongside labels which we group into major classes (Strodthoff al. 2020). Further details can be found in Appendix A
4.2 DEPLOYING CLINICAL PROTOT FOR RETRIEVAL
We aim to exploit clinical prototypes to retrieve physiological instances, that satisfy certain criteria, from large database. One could argue that a simple text-based query, instead of CPs, in modern databases will allow for the retrieval of appropriate instances. However, this only holds if all instances in the database are labelled. Although we have used, as an exemplar, seudil--vvailable attributes such sex and age, our method extends other attributes which cannot be trivially searched in a database. In other words, the maximal utility of retrieval via CPs is realized when the troves of instances the database are unlabelled. To reiterate, the clinical motivation for such retrieval is multi-fold. First, it allows researchers to discover patients that may be suitable for clinical trial recruitment. This, in turn, may allow pharmaceutical companies trial their drugs on a more diverse patient cohort that is more representative of the population. Second, CPs can be deployed across datasets from different clinical institutions, for example, retrieve patient sub-cohorts that satisfy set of attributes. These patient sub-cohorts can now be further analysed to identify sub-cohort variability, which in turn can guide future clinical treatment. With that in mind, to reliably and quantitatively evaluate the retrieval capabilities of the CPs, must have access the ground-truth attributes associated with the instances in the held-out set.
To perform retrieval, we treat the CPs the query set and retrieve the closest instances (1 Euclidean distance) from a held-out dataset. In evaluating this task of retrieval, we deploy a commonly used metric known Precision K (P@ K). This metric quantifies whether at least one of the K retrieved instances matches the query. In our context, however, a match can occur according to any of the three attributes: class, sex, and age. Therefore, we extract single attribute, Oms from the set of attributes, A, for each of the M clinical prototypes, and the same attribute, 0g for each of the K retrieved instances, and define the following attribute-specift P@ K.
M
K
(@K(aa: = M
1
Σ
8
Σ 1==
8(a: = am)) 21)
(8)
m==1
4.3 DEPLOYING CLINICAL PROTOTYPES FOR CLUSTERING
We also exploit clinical prototypes cluster instances. In this setting, and in contrast to the retrieval setting, we treat each representation of an instance as a query and search across the CPs to assign each representation a set of attributes. From this perspective, CPs can be thought as multiple centroids of cluster which are used to label representations. To evaluate these clusters, calculate the pairwise Euclidean distance between each representation and CP before assigning each representation the specific attribute, ag pred of the CP to which it is closest. Given the ground-truth attribute for each representation, arrue, can calculate the accuracy, Acc(a), of the assignments. We can also quantify the agreement between the attribute assignments of the ground-truth, attue {atreer arrue ... N } and those obtained via clustering, qured preee afred: ..., a (eed by calculating the adjusted mutual information AMI(a) E [0,
first
N pred = a true
1
Acc(a) = N
5(0
(9)
AM](a) = MI(artre Grredd E EEMMI (EMMIlatrur strue @pred
(10)
H((āpred)) (f apred))
5

---

Under review as a conference paper at ICLR 2021
where (a ( represents the mutual information between the ground-truth and the predicted set of attributes, and (()) represents the entropy the set of attributes.
4.4 BASELINES
Where appropriate, we compare our method the following approaches:
K-Means implements Expectation Maximization to arrive at class-specific centroids. Instances are then assigned to the class of their nearest centroid. We perform K-Means a) on the input instances (K-Means Raw) and b) representations of instances learned using Eq. (K-Mens Combined).
Mean Representations takes the mean of representations, learned via Eq. 7 that belong to particular attribute combination and treats such mean representations the centroids of the clusters.
DeepCluster (DC) (Caron et al. 2018) is an iterative method that performs K-Means on rep- resentations, pseudo-labed instances according to their assigned cluster, and then exploits such labels for supervised training. The final set of instance labels is taken from the epoch with the lowest validation loss.
Information Invariant Clustering (IIC) (Ji et al., 2019) maximizes the mutual information between the posterior class probabilities assigned to an instance and its perturbed counterpart. We adapt IIC to the time-series domain by perturbing instances with additive Gaussian noise, c (O, r). We also incorporate auxiliary over-clustering it was shown to significantly improve performance. The final set of instance labels is chosen by taking the argmax of the output probabilities.
SeLA (Asano al. 2020) pseudo-labels instances by implementing the Sinkhorn Bankhom--Koopp al- gorithm to allow for supervised training. We pseudo-label instances after each epoch of training and use prior class information determine the number of clusters, which should boost perform- ance.
Deep Transfer Cluster (DTC) Han et al. 2019) calculates the distance between representations and cluster prototypes generate probability distribution over classes. The KL divergence between this distribution and a target distribution is minimized.
4.5 HYPERPARAMETERS
We chose the temperature parameters, = 0.1, as per (Kiyassch et al.) 2020), and Tu = 1. The regression regularization term (Eq. 6 requires the specification of B. We chose B 0.2 due to our choice distance metric (squared Euclidean distance) and the number of attribute groups. were too small in magnitude, the intra-cluster separability of clinical prototypes would be too small. If 8 were too large, then the clinical prototypes from different classes would begin to overlap with one another and class separability diminishes. For both datasets, sex E {M, F), and age is converted to quartiles. For the Chapman and PTB-XL datasets, |class| and 5, respectively, and thus M [class] X |sex| x lgel 32 and 40. respectively. We use a network with ID convolutional modules to generate representations. Further network and implementation details can be found in Appendix B
EXPERIMENTAL RESULTS
5.1 DEPLOYING CLINICAL PROTOTYPES FOR RETRIEVAL.
We exploit CPs to retrieve iitribute-specinee physiological signals from database. In this section. we task the CPs with retrieving the closest K = [1, 5, 10] instances in held-out set and, in Table I illustrate the resulting P@ K. We also stratify the results according the number of matched attributes between the query (CPs) and retrieved instances. For example, # of matched attributes = implies that a perfect match has occurred for all attributes: class, sex, and age.
We find that the Mean Representation approach is most likely to retrieve physiological instances that satisfy the desired patient attribute criteria. For example, Chapman, Mean Representatinn

---

Under review as a conference paper at ICLR 2021
Table 1: Retrieval performance on Chapman and PTB-XL datasets for K 1,5, 10. Results are stratified according to the number of matched attributes between the query (CP) and retrieved instances and are averaged across 5 seeds. Bold indicates best performing method for retrieving instances with perfect attribute match.
Chapman
PTB-XI.
Method
# of matched attributes
K = 1
5
10
1
5
10
2 1
71.9
100
100
70
90
100
DTC (Han al. 2019
2 2
25
71.9
90
22.5
52.5
80.5
3
3.1
15
36.9
2.5
9.5
16.5
2
91.9
97.5
100
99.0
100
100
Mean Representation
2 2
55
79.4
90
71.5
94.5
100
3
10.6
23.8
36.9
15.5
32
43.5
1
76.3
99.4
100
70
100
100
DROPS Combined
2 2
26.9
74.4
93.8
19.5
60
82
= 3 2.5
12.5
21.3
2.5
7
10
achieves a P1(1) 91.9 whereas DTC and DROPS Combined achieve P@(1) = 71.9 and 76.3, respectively, when evaluated based at least one attribute match. This implies that 91.9% of the representations used as the query set were able to retrieve a physiological instance with typically interested in perfect
at least one attribute match. In retrieval settings, however, attribute matches (i.e., # of matched attributes = 3). In our context, the superior performance of Mean Representation extends the # of matched attributes
+ For example, on Chapman, Mean Representation achieves a (ū) = 10.6 whereas DTC and DROPS Combined achieve P@(1) 3.1 and 2.5, respectively. Recall that with Mean Representation, we are exploiting the average of the representations, learned via our method, that belong to a particular attribute combination as the query set. These mean representations can be visualized in Fig. (top row). With this mind, the aforementioned findings illustrate the potential utility of our supervised contrastive learning setup in obtaining rich representations for attribute-specific retrieval. We hypothesize that the poorer performance of DROPS Combined relative to Mean Representation is due to the more extreme embedding characterized by the former's s clinical prototypes. This can be seen in Fig, 4 (top row). Although such clinical prototypes, which are farther from the class decision boundary, may be beneficial for retrieving instances from the same class, they will perform worse on other attributes.
To qualitatively evaluate clinical prototypes' retrieval capabilities, choose CPs random to form the query set, calculate their Euclidean distance to the representations of instances in a held-our dataset, and retrieve the which closest. In Fig. 3 we visualize the ECG signals that correspond to these closest representations and colour their borders green their class label matches that of the query, and red otherwise. We can that least 60% of the retrieved instances match the class label.
Query
Retrieved
Query CP
Retrieved Instances
AFIB- F-72
NORM- M-47
M spplen
SR- M-49
MI-F- 95
(a) Chapman
(b) PTB-XL
Figure 3: Top 5 retrieved instances based on query CPs from the (a) Chapman and (b) PTB-XL datasets. Green and red borders indicate retrieved instances whose ground-truth class attribute matches that of the CPs and those whose ground-truv -trutt class attribute does not match, respectively.
7

---

Under review as a conference paper at ICLR 2021
5.2 DEPLOYING CLINICAL PROTOTYPES FOR CLUSTERING
So far, have illustrated the utility of exploiting clinical prototypes for retrieval purposes. In this section, we evaluate the utility of CPs for clustering. In Table. 2 we illustrate the clustering performance of DROPS relative that of state-of-the-art clustering methods.
There are several takeaways from Table 2 First, we find that our supervised contrastive learning paradigm generates rich representations, This can be seen by the superiority of K-Means Combined relative to K-Means Raw where the Acc(class) = 81.7 and 29.2, respectively. Note that although the former approach learns clinical prototypes, does not yet exploit them directly for clustering. This, in turn, allows for the reliable evaluation of the representations alone. We provide further evidence in support of the richness of these representations in Sec. [5. When we directly leverage clinical prototypes for clustering, performance. For example, DROPS Combined achieves an Acc(class)
is done with DROPS, we find that this further improves clustering
90.3 and 76.0 on the Chapman and PTB-XL datasets, respectively, which outperforms not only its K-Means counterpart but also recent state-of-the-art methods such as DTCluster, DeepCluster, and SeLA.
To further illustrate the utility of clinical prototypes, we com- pare DROPS Combined to the associated with each of the attribute combina- tions. We find that that the former Ace(class) 10% both Chap- man and PTB-XL. We hypothesize that this discrepancy is partially due to the asymmetric definition of the contrastive objective function (Eq. 3 in which negative samples are only considered from the set of clinical prototypes and not from the representations. Nonetheless, the aforementioned findings underscore the vital role clinical prototypes play in clustering. We also note that method can be trivially incorporated into DeepCluster, for example, to generate pseudo-labels. Lastly, in Table 2b] we show that DROPS is flexible enough to simultaneously cluster according to multiple attrib- utes, feature that does not trivially extend to other methods. Such flexibility provide researchers with improved control over sensitive attributes.
Table 2: Clustering performance of representations in the validation set based on (a) class attribute and (b) sex and age attributes. Results shown across 5 seeds.
Mean Representation
(a) Cardiac arrhythmia class attribute
Chapman AMI
PTB-X XL
Method
Acc
Acc
AMI
Unsupervised Clustering
K-Means Raw
29.2
0.3
SeLA [Asano et al. 2020
21.0
10.2
10.5
1.6
IIC (Ji et al 2019)
27.0
22.0
0.5
DC (Caron et al.
2018
32.3
3.0
10.5
4.9
Supervised Clustering
DTC Han al. 2019
79.8
58.8
67.3
22.1
K-Means Combined
81.7
61.9
36.5
27.4
Mean Representation
80.3
65.0
64.7
29.1
DROPS Contrastive
89.8
72.1
76.1
36.0
DROPS Combined
90.3
72.8
76.0
35.9
(b) Sex and age attributes
Chapman
PTB- -BB
Method
Sex
Age
Sex
Age
DTC (Han al. 2019)
54.8
25.3
58.6
25.7
Mean Kepresentatron
54.8
30.3
69.7
39.4
DROPS Contrastive
56.8
37.4
74.3
19.3
DROPS Combined
57.4
38.0
73.5
19.5
5.3 CAPTURING SEMANTIC RELATIONSHIPS BETWEEN CLINICAL PROTOTYPES
Up until now, we have shown that clinical prototypes can be successfully deployed for retrieval and clustering. We encouraged such prototypes to exhibit inter and intra-cluster separability, with the aim of capturing semantic relationships between them. this section, we look confirm whether such semantic relationships representations in the training set alongside the average class-specific CP. In Fig. (bottom row), we illustrate the t-SNE projection of the CPs colour-coded and shaded according respectively. For clarity, we follow the same coding scheme presented Fig. (left).
indeed captured. In Fig. (top row), illustrate the t-SNE projection of
and age groups,
We show that training paradigm leads to clinical prototypes that satisfy the semantic relation- ships between the class, sex, and age attributes. This can be seen by the high separability between, and ordering of, the and age attributes in Fig. (bottom row). This observation holds regardless
8

---

Under review as a conference paper at ICLR 2021
Chapman
PTB-XL
AFIB
GSVT
S8
SR
CD
HYP
M
NORM
STTC
Clinical Prototype
Mean
Representation
GSVT
AFIB
Male
STTC
Famale
Age
NORM
MI
SB
Figure 4: (Top Row) t-SNE projection of the representations in Chapman and PTB-XL, respectively. These are shown alongside the average clinical prototype and the mean representation per class (shown in larger size). (Bottom Row) i-SNE projection of the corresponding clinical prototypes colour-coded and shaded according to sex and age, respectively (see Fig, left).
of the class attribute or the dataset that is experimented with. We further note the high degree of similarity between these projections, empirically-deriver and expected (see Fig. [[eft). We claim that the adoption of this formation by the CPs is driven primarily by our weighting mechanism (Eq and distance- based regularization term (Eq. [6)
DISCUSSION AND FUTURE WORK
In this paper, we propose a supervised contrastive learning framework, DROPS, for the retrieval and clustering of physiological signals. In the process, we learn representations, entitled clinical prototypes, that efficiently describe the state of a patient associated with a set of attributes, e.g.. disease, sex, age. We show that representations learned via DROPS can be exploited to retrieve instances that reflect desired criteria. Moreover, capturing semantic relationships between patient attributes, can simultancously cluster physiological instances based on multiple attributes. We now elucidate several avenues worth exploring.
show that that clinical prototypes, in addition to
Learning disentangled clinical prototypes. CPs are attribute-specffic representations. Disen- tangling these representations into their constituent attributes might facilitate the interpretability of CPs and their application to fair ML. Preliminary exploratory work is performed in Appendix C
Incorporate continuous attributes. CPs currently limited to discrete attributes. Scaling CPs to a continuous set and large number attributes will allow for more grained retrieval and further increase their utility.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Yuki Asano, Christian Rupprecht, and Andrea Vedaldi. Self-labelling via simultancous clustering and representation learning. In International Conference on Learning Representations, 2020. URL
https:///ppenret
net forum? poorummid=Hyx-.. -i= PF
Siddharth Biswal, Cao Xiao, I Lucas M Glass, Elizabeth Milkovits, and Jimeng Sun. Doctor2vec: Dynamic doctor representation learning for clinical trial recruitment. Proceedings of the AAAI Conference Artificial Intelligence, volume 34, pp. 557-564, 2020.
Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for unsuper- vised learning of visual features. In Proceedings of the European Conference on Computer Vision
(ECCV), pp- 132-149, 2018.
Steve Chamberlin. Steven Bedrick, Aaron M Cohen, Yanshan Wang, Andrew Wen, Sijia Liu, Hongfang Liu, and William Hersh. Evaluation of patient-level retrieval from electronic health record data for a cohort discovery task. medRxiv, pp. 19005280, 2019.
Deepak Roy Chittajallu, Bo Dong, Paul Tunison, Roddy Collins, Katerina Wells, James Fleshman, Ganesh Sankaranarayanan Steven Schwaitzberg, Lora Cavuoto, and Andinet Enquobahrie. Xai- cbir: Explainable ai system for content based retrieval of video frames from minimally invasive surgery videos. In 2019 IEEE 16th International Symposium Biomedical Imaging (ISBI 2019), pp. 66-69. IEEE, 2019.
Sajad Darabi, Mohammad Kachuee, Shayan Fazeli, and Majid Sarrafzadeh. Taper: Time-aware patient ehr representation. IEEE Journal of Biomedical and Health Informaties, 2020.
Leonard W D' Avolio, Thien M Nguyen, Wildon Farwell, Yongming Chen, Felicia Fitzmeyer, Owen M Harris, and Louis D Fiore. Evaluation of generalizable approach to clinical information retrieval using automated retrieval console (arc). Journal of the American Medical Informatics Association, 17(4):375-38,, 2010.
European URL
Commission.
2019.
exchange-elect
hea hewotheettttttttteaeoooaoeee across
Vivien Sainte Fare Garnot and Loic Landrieu. Metric-guided prototype learning, 2020.
Alan Gee, Diego Gareia- Olano, Joydeep Ghosh, and David Paydarfar. Explaining deep classifica- tion of time-series data with learned prototypes. arXiv preprint arXiv:1904.08935, 2019.
Travis R Goodwin and Sanda M Harabagiu. Learning relevance models for patient cohort retrieval. JAMIA open, 1 1(2):265-277) 275, 2018.
Harsha Gurulingappa, Luca Toldo, Claudia Schepers, Alexander Bauer, and Gerard Megaro. Semi- supervised information retrieval system for clinical decision support. TREC, 2016.
Kai Han, Andrea Vedaldi, and Andrew Zisserman. Learning discover novel visual categories via deep transfer clustering. In Proceedings of the IEEE International Conference Computer Vision, pp. 8401-8409, 2019.
William Hersh. Information retrieval: a health and biomedical perspective. Springer Science & Business Media, 2008
William R Hersh and Robert Greenes. Information retrieval in medicine: state of the art. MD Computing: Computers in Medical Practice, 7(5):302-311, 1990.
William R Hersh and David H Hickam. How well do physicians use electronic information retrieval systems?: A framework for investigation and systematic review. Jama, (80(15):1347-133 1998.
Li Huang, Andrew L Shea, Huining Qian, Aditya Masurkar, Hao Deng, and Dianbo Liu. Patient clustering improves efficiency of federated machine learning to predict mortality and hospital stay time using distributed electronic medical records. Journal of biomedical informatics, 99:103291, 2019.
10

---

Under review as a conference paper at ICLR 2021
Xu Ji, João F Henriques, and Andrea Vedaldi. Invariant information clustering for unsupervised image classification and segmentation. In Proceedings of the IEEE International Conference on Computer Vision. pp.
2019.
Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-Wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-iii, freely accessible critical care database. Scientific Data, 3(1):1-9, 2016.
Dani Kiyasseh, Tingting Zhu, and David Clifton. Clocs: Contrastive learning of cardiac signals. arXiv preprint arXiv:2005. 1324. 2020.
Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual faimess. In Advances in neural information processing systems, pp. 4066-4076, 2017.
Isotta Landi, Benjamin S Glicksberg, Hao-Chih Lee, Sarah Cherng, Giulia Landi, Mattco Danicletto,
Joel Dudley, Cesare Furlanello, and Riccardo Miotto. Deep representation learning of electronic health records to unlock patient stratification at scale. arXiv preprint 06Xiv:2003.065 16, 2020.
Junnan Li, Pan Zhou, Caiming Xiong, Richard Socher, and Steven CH Hoi. Prototypical contrastive learning of unsupervised representations. arXiv preprint (XXiv:0005004960 2020a.
Yue Li, Pratheeksha Nair, Xing Han Lu, Zhi Wen, Yuening Wang, Amir Ardalan Kalantari Dehaghi, Yan Miao, Weigi Liu, Tamas Ordog, Joanna Biemaack,, al. Inferring multimodal latent topies from electronic health records. Nature Communications, 11(1):1-17, 2020b.
Dianbo Liu, Dmitriy Dligach, and Timothy Miller. Two-stage federated phenotyping and patient representation learning. arXiv preprint arXiv: 1908. 05596, 2019.
Christopher D Manning, Hinrich Schütze, and Prabhakar Raghavan. Introduction to information retrieval. Cambridge university press, 2008
Riccardo Miotto, Li Li, Brian A Kidd, and Joel T Dudley. Deep patient: an unsupervised representa- tion to predict the future of patients from the electronic health records. Scientific Reports, 6(1): 1-10, 2016.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. In Advances Neural Information Processing Systems, pp. 8024-8035, 2019.
Adam M. Rhine. Information Retrieval for Clinical Decision Support. PhD thesis, 2017.
Rani Saritha, Varghese Paul, and Ganesh Kumar. Content based image retrieval using deep learning process. Cluster Computing, 22(2):4187-4200, 2019.
Nils Strodthoff, Patrick Wagner, Tobias Schaeffter, and Wojeiech Samek. Deep learning for ECG analysis: Benchmarks and insights from PTB-XL. arXiv preprini arXiv:2004. 13701, 2020.
Arnaud Van Looveren and Janis Klaise. Interpretable counterfactual explanations guided by proto- 2019.
types. arXiv preprint arXiv: arXiv:
Sahil Verma and Julia Rubin. Fairness definitions explained. In 2018 IEEE/ACM International Workshop Software Fairness (FairWare), pp. 1-7. IEEE, 2018.
Patrick Wagner, Nils Strodthoff, Ralf-Dieter Bousseljot, Wojeiech Samek, and Tobias Schaeffter. PTB- XL, a large publicly available electrocardiography dataset, 2020, URL htttts::///hhsiorrtr org/conter /ptp- x1/1.0.17
Byron Wallace, Joël Kuiper, Aakash Sharma, Mingxi Zhu, and lain J Marshall. Extracting pico sentences from clinical trial reports using supervised distant supervision. The Journal of Machine Learning Research, 17(1):4572-4596 2016.
Haolin Wang. Qingpeng Zhang, and Jiahu Yuan. Semantically enhanced medical information retrieval system: tensor factorization based approach. IEEE Access, 5:7584-7593, 2017.
11

---

Under review as a conference paper at ICLR 2021
Yanshan Wang, Andrew Wen, Sijia Liu, William Hersh, Steven Bedrick, and Hongfang Liu. Test collections for electronic health record-based clinical information retrieval. JAMIA open, 2(3): 360-368, 2019.
Jianwei Zheng, Jianming Zhang, Sidy Danioko, Hai Yao, Hangyuan Guo, and Cyril Rakovski. A 12-lead electrocardiogram database for arrhythmia research covering more than 10,000 patients. Scientific Data, 7 2020.
12


Reviews:
[{"review_id": "tBKcvoOiUV0", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\nReasons for score: \n\nThe authors propose a novel contribution of designing a contrastive learning based attribute specific retrieval system for physiological signals, but the experiments in the paper aren’t sufficient in showing the proposal’s utility relative to appropriate baselines for the two tasks specified ( IR and clustering ) and  don’t discuss the prior work the physiological signal representation mechanism is based on which is central to the work.  \n\nThe work could be greatly improved by clarifying some important aspects of the proposed system and by conducting more appropriate experiments to back the result claims made.\n\n \n##########################################################################\nPros:\n1) The authors propose a novel contribution of designing a contrastive learning based attribute specific retrieval system for physiological signals  and do a fairly good job of showing how it differs in regards to existing works ( with the exception of 1 or 2 important ones )\n2) The contrastive learning approach of the contrastive-soft loss is motivated and explained well and shown to improve \nupon solely using K-Means on representations learned via the L_contrastive−soft + L_regression  combined loss objective.\n \n##########################################################################\nCons: \n\n1) There are no comparisons for how the system performs in the IR case relative to other works and the baselines for the clustering case are not appropriate ( as they are all unsupervised ) and this method explicitly needs supervision via attributes to derive attribute specific prototypes for learning.\n\t\n   Also, in the retrieval section the use of R@K doesn’t seem appropriate as used.\n   Usually R@K means does your result (document / entity / etc ) appear in the top K results, but here its different.\n\n   For instance, if we look at the sex attribute, we have a set of M total attributes one per prototype (either 32 or 40) that is split \n   evenly between male and female [ M, F, M , F … ]  =  [a1, a2, a3, a4 .. aM ] for all M combinations of attributes.\n   Given my understanding of the formulation in 4.2, we take a particular CP (lets say its the 15th of 32 ), then find its K nearest neighbors by euclidean distance, and then compare the true sex of those K returned neighbors with that of the CP.\n   However this “sex” attribute is either M or F and doesn’t pertain to the exact sex prototype of CP 15 ( ie, the task isn’t find the correct 1 out of 32, but rather find any of the correct 16 out of 32 ).. Is this correct?  \n   \n   In the medical field, the need for exact matches seems like it would preclude just expanding the number of returned items K in hopes of finding one correct results especially when there is a simple possibility for querying with structured data anyways ( ie filter signals by age / sex / class first )  and then just use retrieval of similar signals that way.  Is that incorrect? \n\t\n    In section 5.1 60% at K=5 is hard to assess for quality without comparing it against other retrieval methods.  \n\n    Specific to the clustering section, the external baselines described are all unsupervised and not directly comparable to the \n    supervised DROPS or K-means Combined methods as such.  Effectively the work shows that doing k-means on representations learned by the paper is not as good in terms of ACC/AMI as compared with those learned from two proposed proposed variants of contrastive learning.  \n\n  The two proposed variants of attribute specific contrastive learning are interesting, but hard to size up against other methods.\n    For instance  taking latent space reps from the  autoencoder + prototypical classifier network  from Gee et al19 and comparing how K-Means on those representations does with K-means Combined or DROPS would be directly comparable and appropriate ( there are probably other supervised learning methods to construct representations that you could use from the prior works section as well ).\n\n\n2) Kiyasseh et al., 2020 “ Clocs: Contrastive learning of cardiac signals”  is not mentioned in the prior work section and mentioned once in section 4.5 to discuss the use of a hyper parameter.  \n\tThis work seems quite similar in the methods, datasets, etc used particularly in regards to contrastive learning of cardiac signals for representation learning so in the background section the authors should clarify and say if this is an extension to that work or explain the differences. \n\n3) Points that were unclear to me:  \n\t3.1 The construction of D_hat and the ground truth D in section 3.3 in “Arrangement of clinical prototypes” .  It seems that in order to regularize with respect to the CPs, in order to calculate pairwise distances between them to make Dhat, you need to be given the prototypes initially ( otherwise where does the ground truth D come from ).  Its very possible this is a misunderstanding on my end, but this section could have been clearer.  \n\n       3.2 Similarly, is C set to 4 and 5 for the two datasets specifically or are they set to 8 in both cases ( sex |2| x age |4| = |8| )?  The wording left that a little unclear to me and specifying that in section 4.5 would resolve that. \n          \n\t3.3 How are the \"v\" representations and prototypes actually learned?   The architecture given in section C of the supplement outputs an E x C, which seems its intended for the prototypes, but its not entirely clear.\n          \n       3.4  There are two different architectures used for the Chapman and PLB-XL datasets and the reason given is the size of the later, \n            but the first has ~10k examples ( each of size 2500 ) and 4 classes whereas the later has 18k each with size 2500 and 5 classes so they seem to be relatively similar in size no?   \n\n4)  The work Gee et al 19 “Explaining Deep Classification of Time-Series Data with Learned Prototypes”  ( ICML 19 https://arxiv.org/abs/1904.08935 ) seems that it should possibly be cited in the prior works section since it learns prototypes on clinical signals and shows how the t-sne representations of the prototypes lie with respect to the learned latent representations of the patient clinical signals.  The work doesn't explicitly describe a retrieval mechanism, but it does assign classes to the prototypes by finding the nearest neighbor in euclidean distance over the training set and giving the prototype its class which is similar in nature to the retrieval mechanism proposed here ( though with a different and non-contrastive learning mechanism ).\n\n5)  If the Beta parameter is distance metric dependent then it would be useful to show an experiment showing what occurs in terms of ACC / AMI by adjusting this parameter to justify setting it to .2.\n\n6) Table 1 this table should really be  split to be attribute specific  ( have additional columns for age / sex / class for each K=1 , K=5 and K=10 for the two datasets)\n    It seems like the approach would do best at the class task and then do worse ( as is 5.1 ) at getting the sex or age attributes, but without that information we can’t discern this for the retrieval task ( though its shown for the clustering task in Table 2 ) \n\n7) In Table 2 , the Combined method doesn’t improve performance in (a) or (b) for the PTB-XL case and it would be useful to hear discussion on that.\n\n8) Below Table 2, I disagree with the statement “Note that this approach does not yet exploit clinical prototypes” since the training setup which learns these representations in fact does leverage CPs for learning purposes  since they are explicitly used in the contrastive soft loss objective.\n\n9) K-Means could be used to provide clustering by different attributes ( though possibly not simultaneously ) and adding that to Table 2 (b) could help elucidate if its advantageous or not to learn the clusters simultaneously in fact.  \n\n10) In Section 5.3, the contrastive learning representations are separated pretty well in the Chapman set, but it seems to a much lesser extent in the PTB-XL data.  Why is that?  \n   What does plotting the k-means centroids of these representations with T-SNE give?   \n   How would this differ from setting K=32 ( for the Chapman case ) and then using the average of the representations fo each \n   example in a specific attribute combination grouping ( sex - age - class ) as the centroid for each K?  \n       \n##########################################################################\n\nQuestions during rebuttal period: \nPlease address and clarify the cons above \n \n#########################################################################\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "syOyRxg288S", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The manuscript focuses on clinical natural language processing of electronic health records. More precisely, it addresses a text classification task called information extraction or named entity recognition from these clinical records. Its contributions include developing an embedding model to capture clinical prototypes (CPs), via supervised contrastive learning, and presenting experimental evidence of these learnt CPs capturing attribute-specific semantic relationships and being helpful in subsequent clinical natural language processing task of information retrieval and clustering of clustering of physiological signals. I find this text processing methodology interesting, carefully described, and supplementary to other studies.\n\nHowever, unfortunately, the authors demonstrate limited understanding of the related literature. First, the second and third paragraph of the Introduction section have many sentences that require references to be inserted. Second, and more importantly, the Related work section seems to not capture the key papers and trends of the field (I suggest reading some systematic reviews or surveys on clinical natural language processing, information extraction, and information extraction by, for example, Wendy Chapman, Carol Friedman, and Pierre Zweigenbaum), and, for example, as illustrated by the ImageCLEF and CLEFeHealth evaluation labs, computer vision tends to proceed faster than text analytics (see, e.g., https://www.researchprotocols.org/2018/7/e10961/), and not the other way around as claimed by this section.\n\nIn addition, to feel convinced of the presented experimental evidence, I would have wanted to see statistical significance tests, confidence intervals, effect sizes, or similar presented. I could not find this methodology described in the manuscript or its outcomes, although the narrative repeatedly referred to significant performance gains. Please clarify.\n\nAs my main minor comment, I would like to see a clearer separation of the materials, methods, and experiments sections from results, as well as including clearer justifications of this study design. For example, the aforementioned significance topic has not been addressed sufficiently. Another illustration of somewhat difficult task for the reader is to understand the experimental design as a whole and be convinced of this study being rigorous is the Experimental Results section including a lot of methodological details as opposed to only obtained outcomes.\n\nI also suggest including a conclusion statement as well as embedding more evidence (e.g., evaluation materials and methods plus obtained indicators of performance gains, such as measure values and effect sizes) to convince the reader in the abstract. To continue, please remember to punctuate equations and formulae. \n\nFinally, typically the Methods, Experiments, and Experimental Results sections would have been written using a past tense to emphasise a finished (as opposed to an ongoing) study where materials, methods, and experiments have already been chosen, justified, and completed. Most importantly, please avoid having inconsistent tense in these sections (see, e.g., Section 4.5. using a past tense whereas almost all others are having a present tense).", "extra_scores": {}, "extra_text": {}}, {"review_id": "YrVRR_bKDSb", "reviewer": "AnonReviewer6", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################  \nSummary:\n\nThis paper proposes to tackle the problem of retrieving and clustering physiological signals by learning clinical prototypes via supervised contrastive learning. Three readily available patient attributes, disease, age, and sex, are used to assist the learning. A hard assignment of samples to prototype is proposed, and further relaxed to a soft one to utilize the samples that do not have exactly matched attribute set. A regularization term is also proposed to encourage intra-cluster distances. Two ECG datasets are used to evaluate the proposed model.\n\n##########################  \nPros:\nThe target problem to tackle is important and the idea of using prototype learning approach is interesting.\n\n\n##########################  \nCons/Questions: \n1. Overall, the writing can be improved. I find several places confusing and many details are not fully described. For example, in Section 3.1, \"To that end, we propose to learn a set of embeddings, $p\\in P$\", what is space $P$?\n2. It's unclear to me what needs to be learned, $p$ or $v$, or both?\n3. In Eq (4), what is $\\text{class}_i$? is it already known?\n4. In Eq (8), the $\\delta(\\cdot)$ function is undefined. Is it the indicator function?\n5. For the retrieval task, why the CPs are used as the query set to retrieve instances from the held-out set, instead of the reverse (given unseen sample as query and retrieve samples from database)?\n6. For the retrieval task, it is unclear to me if the attributes are available for the held-out set. If yes, then the evaluation metric of the retrieval task seems weird to me: If the task is to simply retrieve samples with similar attributes, it is straightforward to just query the attributes and just retrieve the samples with a maximal number of matched attributes. Why do we need to learn embeddings for such task? If the attributes are not available, what is the clinical motivation for retrieving such instances?\n7. Usually the ECG signals are processed with CNN or RNN models, it's unclear if and what models are used by the authors to generate the feature $x$ for the input ECG.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes to learn clinical prototypes via supervised contrastive learning to facilitate the reliable retrieval of clinical information and clustering in large datasets. The presentation of the paper could be substantially improved – e.g., the overview and motivation of the paper, the definition of clinical prototypes, selection of certain evaluation criteria, clarification of terminology in equations, the description of the motivations and settings of the experiments, etc.  In addition to the need to substantial improvement in clarity, major concerns include lack of comparison with more supervised approaches and discussion of relevant literatures raised by reviewers.  
Decision: Reject