CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients

1 INTRODUCTION . At present , the healthcare system is unable to sufficiently leverage the large , unlabelled datasets that it generates on a daily basis . This is partially due to the dependence of deep learning algorithms on high quality labels for good generalization performance . However , arriving at such high quality labels in a clinical setting where physicians are squeezed for time and attention is increasingly difficult . To overcome such an obstacle , self-supervised techniques have emerged as promising methods . These methods exploit the unlabelled dataset to formulate pretext tasks such as predicting the rotation of images ( Gidaris et al. , 2018 ) , their corresponding colourmap ( Larsson et al. , 2017 ) , and the arrow of time ( Wei et al. , 2018 ) . More recently , contrastive learning was introduced as a way to learn representations of instances that share some context . By capturing this high-level shared context ( e.g. , medical diagnosis ) , representations become invariant to the differences ( e.g. , input modalities ) between the instances . Contrastive learning can be characterized by three main components : 1 ) a positive and negative set of examples , 2 ) a set of transformation operators , and 3 ) a variant of the noise contrastive estimation loss . Most research in this domain has focused on curating a positive set of examples by exploiting data temporality ( Oord et al. , 2018 ) , data augmentations ( Chen et al. , 2020 ) , and multiple views of the same data instance ( Tian et al. , 2019 ) . These methods are predominantly catered to the image-domain and central to their implementation is the notion that shared context arises from the same instance . We believe this precludes their applicability to the medical domain where physiological time-series are plentiful . Moreover , their interpretation of shared context is limited to data from a common source where that source is the individual data instance . In medicine , however , shared context can occur at a higher level , the patient level . This idea is central to our contributions and will encourage the development of representations that are patient-specific . Such representations have the potential to be used in tasks that exploit patient similarity such as disease subgroup clustering and discovery . As a result of the process , medical practitioners may receive more interpretable outputs from networks . In this work , we leverage electrocardiogram ( ECG ) signals to learn patient-specific representations in a self-supervised manner via contrastive learning . To do so , we exploit the fact that ECG signals summarize both temporal and spatial information . The latter can be understood in terms of projections of the same electrical signal onto multiple axes , also known as leads . Contributions . Our contributions are the following : 1 . We propose a family of patient-specific contrastive learning methods , entitled CLOCS , that exploit both temporal and spatial information present within ECG signals . 2 . We show that CLOCS outperforms state-of-the-art methods , BYOL and SimCLR , when performing a linear evaluation of , and fine-tuning on , downstream tasks involving cardiac arrhythmia classification . 2 RELATED WORK . Contrastive Learning . In contrastive predictive coding , Oord et al . ( 2018 ) use representations of current segments to predict those of future segments . More recently , Tian et al . ( 2019 ) propose contrastive multi-view coding where multiple views of the same image are treated as ‘ shared context ’ . He et al . ( 2019 ) ; Chen et al . ( 2020 ) ; Grill et al . ( 2020 ) exploit the idea of instance discrimination ( Wu et al. , 2018 ) and interpret multiple views as stochastically augmented forms of the same instance . They explore the benefit of sequential data augmentations and show that cropping and colour distortions are the most important . These augmentations , however , do not trivially extend to the time-series domain . Shen et al . ( 2020 ) propose to create mixtures of images to smoothen the output distribution and thus prevent the model from being overly confident . Time Contrastive Learning ( Hyvarinen & Morioka , 2016 ) performs contrastive learning over temporal segments in a signal and illustrate the relationship between their approach and ICA . In contrast to our work , they formulate their task as prediction of the segment index within a signal and perform limited experiments that do not exploit the noise contrastive estimation ( NCE ) loss . Bachman et al . ( 2019 ) Time Contrastive Networks ( Sermanet et al. , 2017 ) attempt to learn commonalities across views and differences across time . In contrast , our work focuses on identifying commonalities across both spatial and temporal components of data . Self-Supervision for Medical Time-Series . Miotto et al . ( 2016 ) propose DeepPatient , a 3-layer stacked denoising autoencoder that attempts to learn a patient representation using electronic health record ( EHR ) data . Although performed on a large proprietary dataset , their approach is focused on EHRs and does not explore contrastive learning for physiological signals . Sarkar & Etemad ( 2020 ) apply existing self-supervised methods on ECG recordings in the context of affective computing . The methods implemented include defining pretext classification tasks such as temporal inversion , negation , time-warping , etc . Their work is limited to affective computing , does not explore contrastive learning , and does not exploit multi-lead data as we do . Lyu et al . ( 2018 ) explore a sequence to sequence model to learn representations from EHR data in the eICU dataset . In the process , they minimize the reconstruction error of the input time-series . Li et al . ( 2020 ) leverage the aforementioned unsupervised learning technique on a large clinical dataset , CPRD , to obtain uncertainty estimates for predictions . 3 BACKGROUND . 3.1 CONTRASTIVE LEARNING . Assume the presence of a learner fθ : x ∈ RD −→ h ∈ RE , parameterized by θ , which maps a D-dimensional input , x , to an E-dimensional representation , h. Further assume the presence of an unlabelled dataset , X ∈ RNxD , where N is the total number of instances . Each unlabelled instance , xi ∈ X , is exposed to a set of transformations , TA and TB , such that xiA = TA ( x i ) and xiB = TB ( x i ) . Such transformations can consist of two different data augmentation procedures such as random cropping and flipping . These transformed instances now belong to an augmented dataset , X ′ ∈ RNxDxV , where V is equal to the number of applied transformations . In contrastive learning , representations , hiA = fθ ( x i A ) and h i B = fθ ( x i B ) , are said to share context . As a result of this shared context , these representations constitute a positive pair because ( a ) they are derived from the same original instance , xi , and ( b ) the transformations applied to the original instance were class-preserving . Representations within a positive pair are encouraged to be similar to one another and dissimilar to representations of all other instances , hjA , h j B ∀j j 6= i . The similarity of these representations , s ( hiA , h i B ) , is quantified via a metric , s , such as cosine similarity . By encouraging high similarity between representations in the positive pair , the goal is to learn representations that are invariant to different transformations of the same instance . 4 METHODS . 4.1 POSITIVE AND NEGATIVE PAIRS OF REPRESENTATIONS . Representations that are derived from the same instance are typically assumed to share context . This approach , however , fails to capture commonalities present across instances . In the medical domain , for example , multiple physiological recordings from the same patient may share context . It is important to note that if the multitude of physiological recordings associated with a patient were collected over large time-scales ( e.g. , on the order of years ) and in drastically different scenarios ( e.g. , at rest vs. during a stress test ) , then the shared context across these recordings is likely to diminish . This could be due to changing patient demographics and disease profiles . With the previous caveat in mind , we propose to leverage commonalities present in multiple physiological recordings by redefining a positive pair to refer to representations of transformed instances that belong to the same patient . We outline how to arrive at these transformed instances next . 4.2 TRANSFORMATION OPERATORS . When choosing the transformation operators , T , that are applied to each instance , the principal desideratum is that they capture invariances in the ECG recording . Motivated by the observation that ECG recordings reflect both temporal and spatial information , we propose to exploit both temporal and spatial invariance . We provide an intuition for such invariances in Fig . 1 . As is pertains to temporal invariance ( Fig . 1 left ) , we assume that upon splitting an ECG recording , associated with Class 1 , into several segments , each of them remain associated with Class 1 . We justify this assumption based on human physiology where abrupt changes in cardiac function ( on the order of seconds ) are unlikely to occur . If these segments were collected years apart , for example , our assumption may no longer hold . As for spatial invariance ( Fig . 1 right ) , we leverage the hexiaxial diagram which illustrates the location of the leads relative to the heart . We assume that temporallyaligned ECG recordings from different leads ( views ) are associated with the same class . This is based on the idea that multiple leads ( collected at the same time ) will reflect the same underlying cardiac function . Occasionally , this assumption may not hold , if , for example , a cardiac condition afflicts a specific part of the heart , making it detectable by only a few leads . We now describe how to exploit these invariances for contrastive learning . Contrastive Multi-segment Coding ( CMSC ) . Given an ECG recording , xi , with duration S seconds , we can extract V non-overlapping temporal segments , each with duration S/V seconds . If V = 2 , for example , xit1 = Tt1 ( x i ) and xit2 = Tt2 ( x i ) where t indicates the timestamp of the temporal segment ( see Fig . 1 left ) . We exploit temporal invariances in the ECG by defining representations of these adjacent and non-overlapping temporal segments as positive pairs . Contrastive Multi-lead Coding ( CMLC ) . Different projections of the same electrical signal emanating from the heart are characterized by different leads , L. For example , with two leads , L1 and L2 , then xiL1 = TL1 ( x i ) and xiL2 = TL2 ( x i ) ( see Fig . 1 right ) . We exploit spatial invariances in the ECG by defining temporally-aligned representations of these different projections as positive pairs . Contrastive Multi-segment Multi-lead Coding ( CMSMLC ) . We simultaneously exploit both temporal and spatial invariances in the ECG by defining representations of non-overlapping temporal segments and different projections as positive pairs . For example , in the presence of two temporal segments with timestamps , t1 and t2 , that belong to two leads , L1 and L2 , then xit1 , L1 = Tt1 , L1 ( x i ) and xit2 , L2 = Tt2 , L2 ( x i ) .

Summary: This paper proposes to use contrastive learning to learn representations from cardiac signals (ECGs). The model incorporates ECG domain knowledge, patient-specific, and relationships between multiple leads (channels), in the learning process. The targeted task is very important, enormous ECGs are collected and stored, but seldom people are mining them as they are unlabelled. This paper might reinvigorate them.