CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients

1 INTRODUCTION . At present , the healthcare system is unable to sufficiently leverage the large , unlabelled datasets that it generates on a daily basis . This is partially due to the dependence of deep learning algorithms on high quality labels for good generalization performance . However , arriving at such high quality labels in a clinical setting where physicians are squeezed for time and attention is increasingly difficult . To overcome such an obstacle , self-supervised techniques have emerged as promising methods . These methods exploit the unlabelled dataset to formulate pretext tasks such as predicting the rotation of images ( Gidaris et al. , 2018 ) , their corresponding colourmap ( Larsson et al. , 2017 ) , and the arrow of time ( Wei et al. , 2018 ) . More recently , contrastive learning was introduced as a way to learn representations of instances that share some context . By capturing this high-level shared context ( e.g. , medical diagnosis ) , representations become invariant to the differences ( e.g. , input modalities ) between the instances . Contrastive learning can be characterized by three main components : 1 ) a positive and negative set of examples , 2 ) a set of transformation operators , and 3 ) a variant of the noise contrastive estimation loss . Most research in this domain has focused on curating a positive set of examples by exploiting data temporality ( Oord et al. , 2018 ) , data augmentations ( Chen et al. , 2020 ) , and multiple views of the same data instance ( Tian et al. , 2019 ) . These methods are predominantly catered to the image-domain and central to their implementation is the notion that shared context arises from the same instance . We believe this precludes their applicability to the medical domain where physiological time-series are plentiful . Moreover , their interpretation of shared context is limited to data from a common source where that source is the individual data instance . In medicine , however , shared context can occur at a higher level , the patient level . This idea is central to our contributions and will encourage the development of representations that are patient-specific . Such representations have the potential to be used in tasks that exploit patient similarity such as disease subgroup clustering and discovery . As a result of the process , medical practitioners may receive more interpretable outputs from networks . In this work , we leverage electrocardiogram ( ECG ) signals to learn patient-specific representations in a self-supervised manner via contrastive learning . To do so , we exploit the fact that ECG signals summarize both temporal and spatial information . The latter can be understood in terms of projections of the same electrical signal onto multiple axes , also known as leads . Contributions . Our contributions are the following : 1 . We propose a family of patient-specific contrastive learning methods , entitled CLOCS , that exploit both temporal and spatial information present within ECG signals . 2 . We show that CLOCS outperforms state-of-the-art methods , BYOL and SimCLR , when performing a linear evaluation of , and fine-tuning on , downstream tasks involving cardiac arrhythmia classification . 2 RELATED WORK . Contrastive Learning . In contrastive predictive coding , Oord et al . ( 2018 ) use representations of current segments to predict those of future segments . More recently , Tian et al . ( 2019 ) propose contrastive multi-view coding where multiple views of the same image are treated as ‘ shared context ’ . He et al . ( 2019 ) ; Chen et al . ( 2020 ) ; Grill et al . ( 2020 ) exploit the idea of instance discrimination ( Wu et al. , 2018 ) and interpret multiple views as stochastically augmented forms of the same instance . They explore the benefit of sequential data augmentations and show that cropping and colour distortions are the most important . These augmentations , however , do not trivially extend to the time-series domain . Shen et al . ( 2020 ) propose to create mixtures of images to smoothen the output distribution and thus prevent the model from being overly confident . Time Contrastive Learning ( Hyvarinen & Morioka , 2016 ) performs contrastive learning over temporal segments in a signal and illustrate the relationship between their approach and ICA . In contrast to our work , they formulate their task as prediction of the segment index within a signal and perform limited experiments that do not exploit the noise contrastive estimation ( NCE ) loss . Bachman et al . ( 2019 ) Time Contrastive Networks ( Sermanet et al. , 2017 ) attempt to learn commonalities across views and differences across time . In contrast , our work focuses on identifying commonalities across both spatial and temporal components of data . Self-Supervision for Medical Time-Series . Miotto et al . ( 2016 ) propose DeepPatient , a 3-layer stacked denoising autoencoder that attempts to learn a patient representation using electronic health record ( EHR ) data . Although performed on a large proprietary dataset , their approach is focused on EHRs and does not explore contrastive learning for physiological signals . Sarkar & Etemad ( 2020 ) apply existing self-supervised methods on ECG recordings in the context of affective computing . The methods implemented include defining pretext classification tasks such as temporal inversion , negation , time-warping , etc . Their work is limited to affective computing , does not explore contrastive learning , and does not exploit multi-lead data as we do . Lyu et al . ( 2018 ) explore a sequence to sequence model to learn representations from EHR data in the eICU dataset . In the process , they minimize the reconstruction error of the input time-series . Li et al . ( 2020 ) leverage the aforementioned unsupervised learning technique on a large clinical dataset , CPRD , to obtain uncertainty estimates for predictions . 3 BACKGROUND . 3.1 CONTRASTIVE LEARNING . Assume the presence of a learner fθ : x ∈ RD −→ h ∈ RE , parameterized by θ , which maps a D-dimensional input , x , to an E-dimensional representation , h. Further assume the presence of an unlabelled dataset , X ∈ RNxD , where N is the total number of instances . Each unlabelled instance , xi ∈ X , is exposed to a set of transformations , TA and TB , such that xiA = TA ( x i ) and xiB = TB ( x i ) . Such transformations can consist of two different data augmentation procedures such as random cropping and flipping . These transformed instances now belong to an augmented dataset , X ′ ∈ RNxDxV , where V is equal to the number of applied transformations . In contrastive learning , representations , hiA = fθ ( x i A ) and h i B = fθ ( x i B ) , are said to share context . As a result of this shared context , these representations constitute a positive pair because ( a ) they are derived from the same original instance , xi , and ( b ) the transformations applied to the original instance were class-preserving . Representations within a positive pair are encouraged to be similar to one another and dissimilar to representations of all other instances , hjA , h j B ∀j j 6= i . The similarity of these representations , s ( hiA , h i B ) , is quantified via a metric , s , such as cosine similarity . By encouraging high similarity between representations in the positive pair , the goal is to learn representations that are invariant to different transformations of the same instance . 4 METHODS . 4.1 POSITIVE AND NEGATIVE PAIRS OF REPRESENTATIONS . Representations that are derived from the same instance are typically assumed to share context . This approach , however , fails to capture commonalities present across instances . In the medical domain , for example , multiple physiological recordings from the same patient may share context . It is important to note that if the multitude of physiological recordings associated with a patient were collected over large time-scales ( e.g. , on the order of years ) and in drastically different scenarios ( e.g. , at rest vs. during a stress test ) , then the shared context across these recordings is likely to diminish . This could be due to changing patient demographics and disease profiles . With the previous caveat in mind , we propose to leverage commonalities present in multiple physiological recordings by redefining a positive pair to refer to representations of transformed instances that belong to the same patient . We outline how to arrive at these transformed instances next . 4.2 TRANSFORMATION OPERATORS . When choosing the transformation operators , T , that are applied to each instance , the principal desideratum is that they capture invariances in the ECG recording . Motivated by the observation that ECG recordings reflect both temporal and spatial information , we propose to exploit both temporal and spatial invariance . We provide an intuition for such invariances in Fig . 1 . As is pertains to temporal invariance ( Fig . 1 left ) , we assume that upon splitting an ECG recording , associated with Class 1 , into several segments , each of them remain associated with Class 1 . We justify this assumption based on human physiology where abrupt changes in cardiac function ( on the order of seconds ) are unlikely to occur . If these segments were collected years apart , for example , our assumption may no longer hold . As for spatial invariance ( Fig . 1 right ) , we leverage the hexiaxial diagram which illustrates the location of the leads relative to the heart . We assume that temporallyaligned ECG recordings from different leads ( views ) are associated with the same class . This is based on the idea that multiple leads ( collected at the same time ) will reflect the same underlying cardiac function . Occasionally , this assumption may not hold , if , for example , a cardiac condition afflicts a specific part of the heart , making it detectable by only a few leads . We now describe how to exploit these invariances for contrastive learning . Contrastive Multi-segment Coding ( CMSC ) . Given an ECG recording , xi , with duration S seconds , we can extract V non-overlapping temporal segments , each with duration S/V seconds . If V = 2 , for example , xit1 = Tt1 ( x i ) and xit2 = Tt2 ( x i ) where t indicates the timestamp of the temporal segment ( see Fig . 1 left ) . We exploit temporal invariances in the ECG by defining representations of these adjacent and non-overlapping temporal segments as positive pairs . Contrastive Multi-lead Coding ( CMLC ) . Different projections of the same electrical signal emanating from the heart are characterized by different leads , L. For example , with two leads , L1 and L2 , then xiL1 = TL1 ( x i ) and xiL2 = TL2 ( x i ) ( see Fig . 1 right ) . We exploit spatial invariances in the ECG by defining temporally-aligned representations of these different projections as positive pairs . Contrastive Multi-segment Multi-lead Coding ( CMSMLC ) . We simultaneously exploit both temporal and spatial invariances in the ECG by defining representations of non-overlapping temporal segments and different projections as positive pairs . For example , in the presence of two temporal segments with timestamps , t1 and t2 , that belong to two leads , L1 and L2 , then xit1 , L1 = Tt1 , L1 ( x i ) and xit2 , L2 = Tt2 , L2 ( x i ) .

Summary: This work presents a new self-supervised training framework for multi-channel ECG signals. The authors use contrastive learning by exploiting the fact that a single patient can generate multiple ECG signals, and there are multiple views (i.e. leads) for the same ECG signals. Compared to popular self-supervised training methods BYOL and SimCLR, the proposed method shows superior performance on the arrhythmia classification task for 4 different datasets in various scenarios.