ICML 2025

H-Tuning: Toward Low-Cost and Efficient ECG-based Cardiovascular Disease
Detection with Pre-Trained Models
Rushuang Zhou 1 2 Yuan-Ting Zhang 3 4 5 Yining Dong 6 2
Abstract
Fine-tuning large-scale pre-trained models pro-
vides an effective solution to alleviate the la-
bel scarcity problem in cardiovascular diseases
(CVDs) detection using electrocardiogram (ECG).
However, as the pre-trained models scale up, the
computational costs for fine-tuning and inference
become unaffordable on low-level devices de-
ployed for clinical applications.
Additionally,
maintaining the model performance under low
budgets in computational resources remains a sig-
nificant challenge. However, a comprehensive
study that can address them in a joint framework is
still lacking. Here, we propose a holistic method
(H-Tuning) for low-cost and efficient fine-tuning
of pre-trained models on downstream datasets.
Then, the inference costs of the models fine-tuned
by H-Tuning are further reduced significantly us-
ing a knowledge distillation technique. Exper-
iments on four ECG datasets demonstrate that
H-Tuning reduces the GPU memory consumption
during fine-tuning by 6.34 times while achieving
comparable CVDs detection performance to stan-
dard fine-tuning. With the knowledge distillation
technique, the model inference latency and the
memory consumption are reduced by 4.52 times
and 19.83 times. As such, the proposed joint
framework allows for the utilization of pre-trained
models with high computation efficiency and ro-
bust performance, exploring a path toward low-
cost and efficient CVDs detection. Code is avail-
able at https://github.com/KAZABANA/H-Tuning
1Department of Biomedical Engineering, City University
of Hong Kong, Hong Kong, China 2Hong Kong Center for
Cerebro-Cardiovascular Health Engineering, Hong Kong, China
3Department of Electronic Engineering, Chinese University of
Hong Kong, Hong Kong, China 4Hong Kong Institutes of Medical
Engineering, Hong Kong, China 5The AICARE Bay Lab, Gua...