Transfer Knowledge from Natural Language to Electrocardiography: Can We Detect Cardiovascular Disease Through Language Models? (2023)

Abstract: Recent advancements in Large Language Models (LLMs) have drawn increasing attention since the learned embeddings pretrained on large-scale datasets have shown powerful ability in various downstream applications. However, whether the learned knowledge by LLMs can be transferred to clinical cardiology remains unknown. In this work, we aim to bridge this gap by transferring the knowledge of LLMs to clinical Electrocardiography (ECG). We propose an approach for cardiovascular disease diagnosis and automatic ECG diagnosis report generation. We also introduce an additional loss function by Optimal Transport (OT) to align the distribution between ECG and language embedding. The learned embeddings are evaluated on two downstream tasks: (1) automatic ECG diagnosis report generation, and (2) zero-shot cardiovascular disease detection. Our approach is able to generate high-quality cardiac diagnosis reports and also achieves competitive zero-shot classification performance even compared with supervised baselines, which proves the feasibility of transferring knowledge from LLMs to the cardiac domain.

## introduction heart and cardiovascular diseases are the leading global cause of death, with 80% of cardiovascular disease-related deaths due to heart attacks and strokes. the clinical 12-lead ecg, when correctly interpreted, is the primary tool to detect cardiac abnormalities and heart-related issues. ecg provides unique information about the structure and electrical activity of the heart and systemic conditions through changes in the timing and morphology of the recorded waveforms. achievements of ecg interpretation, such that critical and timely ecg interpretations of cardiac conditions, will lead to efficient and cost-effective intervention. * * marked as equal contribution llm starts from the transformer model @xcite and grows quickly with a wide range of applications @xcite @xcite . recently, llm has shown great potential for accelerating learning in many other domains since the learned embeddings can provide meaningful representation for downstream tasks. examples include transferring the knowledge of llm to, i.e., robotics control @xcite , multimodal reasoning and interaction @xcite , robotics planning @xcite @xcite , decision-making @xcite , robotics manipulation @xcite @xcite @xcite , code generation @xcite , laws @xcite , computer vision @xcite , and so on. some previous works explored llm and biological protein @xcite , or health records @xcite . however, the medical or healthcare domains contain so much domain knowledge that different sources preserve unique data characteristics without a unified paradigm. to the best of our knowledge, no previous work explores the knowledge transfer from llm to cardiovascular disease with ecg signals. in this work, we bridge the gap between llm and clinical ecg by investigating the feasibility of transferring knowledge of llm to the cardiology domain. our contributions are listed as follows: deep learning in ecg deep learning approaches have been rapidly adopted in many fields for their accuracy and flexibility, including ecg domain @xcite @xcite @xcite @xcite @xcite . transformer @xcite has recently been adopted in several ecg applications, i.e., arrhythmia classification, abnormalities detection, stress detection, etc @xcite @xcite @xcite . ## dataset and prepossessing dataset we conducted the experiments on the ptb-xl dataset @xcite , which contains clinical 12-lead ecg signals of 10-second length. there are five conditions in total, including normal ecg (norm), myocardial infarction (mi), st/t change (sttc), conduction disturbance (cd), and hypertrophy (hyp). the waveform files are stored in waveform database (wfdb) format with 16-bit precision at a resolution of 1µv/lsb and a sampling frequency of 100hz. the ecg statements conform to the scp-ecg standard and cover diagnostic, form, and rhythm statements. prepossessing the raw ecg signals are first processed by the wfdb library @xcite and fast fourier transform (fft) to process the time series data into the spectrum, which is shown in fig. 2 . then we perform n-points window filtering to filter the noise within the original ecg signals and adopt notch processing to filter power frequency interference (noise frequency: 50hz, quality factor: 30). the ecg signals are segmented by dividing the 10-second ecg signals into individual ecg beats. we first detect the r peaks of each signal by ecg detectors @xcite , and then slice the signal at a fixed-sized interval on both sides of the r peaks to obtain individual beats. more details can be found in appendix c. feature extraction instead of directly using the time-series signals, we extract time domain and frequency domain features to better represent ecg signals. the time-domain features include: maximum, minimum, range, mean, median, mode, standard deviation, root mean square, mean square, k-order moment and skewness, kurtosis, kurtosis factor, waveform factor, pulse factor, and margin factor. the frequency-domain features include: fft mean, fft variance, fft entropy, fft energy, fft skew, fft kurt, fft shape mean, fft shape std, fft shape skew, fft kurt. more details can be found in appendix c. an analysis of the statistics of the processed ecg data can also be found in ## conclusion in this paper, we bridge the gap between llms and cardiovascular ecg by transferring knowledge of llms into the cardiovascular domain. the transferred knowledge embeddings can be used for downstream applications, including cardiovascular disease diagnosis and automatic ecg diagnosis report generation. our results demonstrate the effectiveness of knowledge transfer, as the proposed method shows excellent performance in both downstream tasks, where our zero-shot classification approach even achieved competitive performance with supervised learning baselines, showing the feasibility of using llm to enhance applications in the cardiovascular domain.