Unsupervised Deep Metric Learning for the inference of hemodynamic value with Electrocardiogram signals
Abstract
An objective assessment of intrathoracic pressures remains an important diagnostic method for patients with heart failure.
Although cardiac catheterization is the gold standard for estimating central hemodynamic pressures, it is an invasive procedure where a pressure transducer is inserted into a great vessel and threaded into the right heart chambers.
Approaches that leverage non-invasive signals – such as the electrocardiogram (ECG) – have the promise to make the routine estimation of cardiac pressures feasible in both inpatient and outpatient settings.
Prior models that were trained in a supervised fashion to estimate central pressures have shown good discriminatory ability over a heterogeneous cohort when the number of training examples is large.
As obtaining central pressures (the labels) requires an invasive procedure that can only be performed in an inpatient setting, acquiring large labeled datasets for different patient cohorts is challenging.
In this work, we leverage a dataset that contains over 5.4 million ECGs, without concomitant central pressure labels, to improve the performance of models trained with sparsely labeled datasets.
Using a deep metric learning (DML) objective function, we develop a procedure for building latent 12-lead ECG representations and demonstrate that these latent representations can be used to improve the discriminatory performance of a model trained in a supervised fashion on a smaller labeled dataset.
More generally, our results show that training with DML objectives with both labeled and unlabeled ECGs showed the downstream performance on par with the supervised baseline.
Introduction
Cardiac catheterization is a procedure for measuring hemodynamics, and is an important diagnostic tool.
For instance, intracardiac pressures play an important role in the assessment of hemodynamic severity in patients with heart failure.
The mean pulmonary capillary wedge pressure (mPCWP) - an estimate of the left atrial pressure - is one measurement that is used for treatment and prognostication in heart failure patients.
The mPCWP, however, is acquired via right heart catherization (RHC), which is an invasive procedure that is only performed in an inpatient setting.
To provide readily available diagnostic tools to detect and understand heart failure, we need safe routine screening methods that reliably predict patient hemodynamics.
One universally available piece of clinical data is the electrocardiogram, which has shown promise with respect to helping to guide the care of patients with suspected heart failure [1].
To reduce the risk of invasive procedures, a ECG-based model can be constructed to infer when the mean pulmonary capillary wedge pressure (mPCWP) is above normal [2].
Previous studies [3, 4, 5, 6, 7] have leveraged large unlabeled datasets to infer cardiac abnormalities using the similarity information between ECGs.
In particular, several approaches have used contrastive learning to construct latent representations of ECG data for downstream inference tasks [3, 4, 5].
1) Using larger unlabeld ECG corpus (ECGs without matched mPCWP), we learn upstream metric embedding f Φ with deep metric learning objective.
Then we finetune this embedding f Φ with smaller ECG dataset with matched mPCWP (labeled ECGs) to 3) infer mPCWP.
Figure 2 includes the diagram for supervised deep metric learning with the joint learning of both triplet and finetuning task loss.
However, Deep Metric Learning (DML), a procedure that maps data to latent representations in a manner such that similar datapoints are mapped to similar points in the latent space, has not been used to develop learned representations of ECG data.
Yet the use of DML for representation learning [8, 9, 10] has yielded frutiful results for other tasks such as few-shot image retrieval [11] or facial re-identification [12].
In this work, we apply DML to a real-world clinical task, intracardiac pressure classification and regression using the 12-lead ECGs.
The goal is to arrive at learned representations that adequately represent the information within ECG signals, using a large resource of unlabeled ECGs (Figure 1).
DML objective functions (e.g., triplet loss and angular loss) learn an embedding that preserves the inherent distance between ECGs by pulling similar or positive samples together and pushing dissimilar samples away in the embedding space.
We define similar/positive labels by ranking the ECGs in a batch using the dynamic time warping distance between two ECGs which represents the inherent characteristics of ECG data.
We finetune the model from upstream DML on downstream task to esimate intracardiac pressures.
Our preliminary results showed improved performance compared to the supervised baseline, which shows promise that utilization of unlabeled dataset helps improve generation of embedding with useful information for downstream task.
Related Works 2.1 Deep Metric Learning
Deep Metric Learning (DML) is a similarity driven learning method for building a representation by learning a distance metric.
DML works by optimizing the loss function defined on pairs or tuples generated from training examples, where the objectives are often distance metric on embedding space.
One of the example distance metrics used for the objectives are ranking-based contrastive training, including a triplet loss function [13, 14] and a pairwise loss funciton [15].
DML has been demonstrated to improve generalization performance in representation learning [8, 9, 10].
However,
DML has been applied mostly in the supervised setting, and only few recent works proposed ways for establishing DML in an unsupervised manner [16, 17, 18].
Furthermore, as DML generally requires class of labels to form the tuplets of data, only a few studies have applied DML to regression problems [19].
Here we apply unsupervised DML to a real-world regression problem to determine if similarity information within ECG dataset helps create an embedding space that improves the performance of downstream hemodynamics inference.
2.2 Self-Supervised Contrastive Learning for Electrocardiogram
Self-supervised contrastive learning performs pre-training with pretext tasks to generate a global embedding over an unlabeled dataset, by modeling consistency while ensuring invariance to perturbations.
There have been several attempts to leverage unlabeled ECG dataset using self-supervised representation learning for downstream ECG classification task [3, 4, 5, 6, 7, 20] and inferrence in biosignals [21].
However, to the best of our knowledge, unsupervised DML has not been applied to downstream tasks involving ECGs.
In this work, we apply DML on unlabeled ECG for downstream hemodynamics inference task and compare the test performance with supervised learning.
Methods
At the core of our project is utilizing Deep Metric Learning to build a better representation by leveraging rich information from unlabeled ECG data.
We build representation space by pulling similar samples into similar metric spaces and pushing different samples apart.
3.1 Supervised Deep Metric Learning for Classification
Given a n labeled ECG dataset D = { ( x 1 , y 1 ) , · · · , ( x n , y n ) } , where x ∈ R d ×T and y ∈ { 0 , 1 } with d being the number of ECG leads and T is the number of ECG timestep.
Binary label y describes whether a patient has elevated mPCWP or not, which is defined by the cutoff mPCWP thresholdmmHg (1 if the patient has elevated mPCWP ( > 15 mmHg).
Per each anchor ECG x a , we define the positive sample x p to be the ECGs having the same label with the anchor ECG and the negative sample x n is defined to be the one that has different label.
The triplet ( x a , x p , x n ) are then sampled from the batch B to calculate the triplet loss L triplet .
The model f Φ that has D -dimensional output is optimized with triplet loss ( L Triplet ) and f Φ with the last fully-connected layer g .
Thus, g · f Φ is optimized by the learning objective combining cross-entropy loss ( L CE ) for a batch size of N with the triplet loss scaling factor of α .
[Equation]
where α is the scaling coefficient of triplet loss and σ is the sigmoid function.
3.2 Supervised Deep Metric Learning for Regression
Similar to the case of supervised DML classification task, where we have N labeled ECG dataset
D = { ( x 1 , y 1 ) , · · · , ( x N , y N ) } , where x ∈ R d ×T and y ∈ R with d ECG leads.
We define the positive sample x p to be the input sample with the smallest absolute label difference with the anchor: x p = { x i | i =
arg min y i ∈B |
y i − y a |} where B is the batch.
Then the negative sample x n is then randomly sampled from the set of ECGs in a batch except the positive sample: N B = { x j ∈ B| x j ̸ = x p } .
Again, we construct the triplet ( x a , x p , x n ) and let the model f Φ be optimized by triplet loss ( L Triplet ) while at the same time the final classifier g · f Φ is jointly optimized with Root Mean Square Error (RMSE) with the triplet scaling factor of α : L tot =
L RMSE + α L triplet .
See Appendix Section A. for the full loss function.
3.3 Unsupervised Deep Metric Learning
Now we expand this setting to unsupervised metric learning where we nave M ECG data D = { x 1 , · · · , x M } , where x ∈ R d without the matched mPCWP labels.
Given an anchor x a in a batch, a positive sample corresponding to each anchor is determined as the one with the smallest distance metric which compares the ECG signal inputs.
Here we measure the distance between two inputs using distance measures suitable for signal input and define the similarity between inputs, dynamic [Table] time warping (DTW) [22]: d DTW .
As the complexity of DTW calculation is quadratic ( O ( n 2 ) ), we use pretrained DTW model to get the surrogate DTW ( ˆ d DTW , See Appendix B for more detail).
We can then define the positive sample to be x p
= { x i | i = arg min x i ∈B ˆ d DTW ( x i , x a ) } .
Then the negative sample x n is randomly sampled from the set containing dataset from the batch excluding the positive sample: N B = { x j ∈ B| x j ̸ = x p } .
The model f
Φ is optimized with the upstream deep metric learning with L triplet .
The input triplets ( x a , x p , x n ) are mapped on D-dimensional representation space Φ , where we finetune the model with the downstream finetuing task (See Appendix A.2, Figure 1).
3.4 Datasets, Tasks, and Model
Dataset with mPCWP Labels (labeled dataset) We construct the 12 lead ECG dataset with corresponding right heart catheterization procedure from the data warehouse of Massachusetts General
Brigham where we have total of 8 , 172 right heart catheterization procedure data and ECGs from 4 , 051 patients.
Our data includes the patients who had ECG recordings the same date they had right heart catheterization from 2010 to 2020.
The ECG data contains 10 seconds of signal data which was sampled with [-
-]250 Hz sampling rate, and the mPCWP label was obtained from the right heart catheterization procedure.
We divided the dataset into train, valid, and test set which contains 4,680, 1,667, 1,825 data points and 2,430, 810, and 811 patients, respectively.
Dataset without mPCWP Labels The larger 12 lead ECG dataset without mPCWP mapping has provided by the agreement from Massachusetts General Hospital and Brigham and Women’s Hospital where we have total of 5 , 426 , [-
-]614 ECGs with 1 , 195 , 268 patients who had ECG recordings while their daytime visit or admission.
We removed patient cohort overlapping with the ECG datasets with mPCWP labels, and excluded abnormal ECG signals with continued zero amplitude (mV) at any point of recording at any lead.
Task We train a model to predict the mPCWP of patient in either classification and regression task.
Binary classification task is to infer the elevation of mPCWP with the threshold of 15 mmHg, and the regression task would be to predict the value of mPCWP itself.
Model For the basic model architecture, we used ResNet18 [23] backbone as encoder for generating
D -dimensional space (metric learning experiments), classifier in combination with the last fully connected layer with sigmoid function (supervised learning, downstream finetuning).
See Appendix
C for more detail.
Results and Future Works
In this work, we have demonstrated with our baseline experiment that unsupervised DML improves the performance of downstream wedge pressure prediction.
We summarize the test performance (Supervised baseline, Supervised DML) and downstream performance on hemodynamics inference (Unsupervised DML) in Table 1.
The results show that supervised and unsupervised DML helps improve the performance of the supervised baseline, specifically more in regression task.
This implies that the underlying similarity information in the ECG signal pattern captured by DTW metric helps inferring the downstream wedge pressure in both classification and regression settings.
Although we obtained the RMSE value of 6.9 mmHg in supervised DML and 7.16 mmHg in unsupervised DML, the regressinon error is not sufficiently small to be applied in clinical ward given that the mean mPCWP measure of our dataset is 15.6 mmHg and standard deviation is 8.07 mmHg.
In the future, we will generalize our experimental setting to the full 5 million ECG dataset we have in the dataset warehouse, and expect the deep metric embedding achieves better performance as it builds embedding with richer similarity based information.
Furthermore, we will add more baselines onto our dataset to compare the result with contrastive learning settings.
One additional baseline and direction we can adopt in our future work would be to apply self-training with noisy student [24], where we generate pseudolabels from unlabeled input ECG with learned model and keep validate with supervised test set.
We will release our final code base and model weights trained on our ECG datasets upon release of our final paper.  Acknowledgements We would like to thank members of the Computational Cardiovascular Research Group and HealthyML
Lab for their invaluable feedback.
A Deep Metric Learning A.1 Supervised Deep Metric Learning for Regression
The full loss function for supervised DML regression task {+uses the Root Mean Square Loss (RMSE) for downstream classification loss, L RMSE = (cid:115) N (cid:80) i =
 ||+}
y
i − f Φ ( x i ) ||n where the equation for the full loss term is:
[Equation]
A.2 Downstream Finetuning
We finetune the model f Φ with the downstream finetuning task with either supervised classification ( L CE ) or regression task ( L RMSE ) on the ECG dataset D
= { ( x 1 , y 1 ) , · · · , ( x N , y N ) } introduced in the Sections 3.1 and 3.2: L CE = − (cid:80) Ni[-=
 y i log ( σ ( f Φ ( x i )) , L RMSE = (cid:115) N (cid:80) i-]
=
y i log ( σ ( f Φ ( x i )) , L RMSE
[Equation]
B Dynamic Time Warping for similarity-based ECG Ranking
DTW calculates the amount of dissimilarity between two waveforms in terms of distance (e.g.
Euclidean) under admissible temporal alignments of those waveforms.
The algorithm attempts to minimize the distance for finding the optimal alignment (warping).
Using discrete matching between existing elements of the waveform, the admissible warping paths are defined; DTW finds the best warping path that minimizes the overall distance using dynamic programming.
This recursive approach is computationally expensive as it costs near O ( n 2 ) .
To use DTW as our ranking metric for the ECGs, we calculate the DTW ( d DTW ) between respective leads of two ECGs and take the average metric over the 12 leads.
This approach adds a large computational burden for each batch processing, given the complexity of the DTW calculation.
Hence, we design a surrogate model (Figure 3) to calculate the d DTW without using the dynamic programming-based recursive algorithm, as described below.
B.1 Surrogate Model to infer DTW between two ECGs
To bypass the cost of calculating the DTW which is up to O ( n 2 ) , we build a surrogate model to get the surrogate DTW value ˆ d DTW , which is the prediction output of the model.
We build the surrogate model with ResNet18 encoders and two fully connected layers (FC layer) (Figure 3).
We first embed two ECGs using the ResNet18 encoder to get 50 dimension output, respectively.
Then we concat the two embeddings to generate 100-dimensional input to two FC layers, getting the final output.
The surrogate model is updated with RMSE loss =
[Figure]
N where N is the number of pairs in a batch B .
Corresponding RMSE loss and R 2 value is evaluated to check if the model performs well to be applied for DTW-based similarity ranking.
We run the model for 50 epochs and got RMSE of 0 .
711 , PearsonR 0 .
939 , and p-value of < 0 .
01 which was sufficient to be used to generate surrogate DTW ˆ d DTW .
C Implementation Details
For the supervised learning baseline we divided the whole ECG dataset with mPCWP labels into train/val/test set and reported the evaluation performance on test set.
For Supervised DML baseline we jointly trained cross entropy loss or RMSE for mPCWP predition task with triplet loss on labeled dataset.
Lastly, for unsupervised DML baseline we utilized ECG dataset without mPCWP labels to generate learned D -dimensional embedding then we finetuned this metric embedding using labeled
ECGs for downstream mPCWP prediction task.
We used the scaling coefficient value of triplet loss, α , to be 1 for the experiments we have presented here.
For the downstream mPCWP inference, we used 2 fully connected layers with batch normalization, ReLU, and Dropout rate of 0.3.