xjc1022/MELP-ECG-FM-init

ECG-FM checkpoint for MELP pretraining

This is the ECG-FM checkpoint that initialises MELP's ECG tower during pretraining. It is passed to MELP's main_pretrain.py as --ecg_encoder_weight. It is the starting point of the run behind xjc1022/MELP-repro and xjc1022/MELP-Encoder-repro. Without it, the ECG tower starts from random weights.

Use this checkpoint to pretrain MELP from the same starting point. For ECG features or zero-shot classification, use the two repos above.

Usage

hf download xjc1022/MELP-ECG-FM-init ecgfm.ckpt --local-dir ckpts

Then, from a MELP checkout with ckpts/ at the repository root:

cd scripts/pretrain
python main_pretrain.py ... --ecg_encoder_weight ../../ckpts/ecgfm.ckpt

The README's Pretraining Stage section has the full command.

Contents

file	size	sha256
ecgfm.ckpt	253 MB	35c4c7f7f81ad98ef80b90dc36f43c70dee911898460aeec27de763146f2e4a8

The file is a dict with state_dict (169 tensors, 63.1M parameters, float32), epoch (44) and global_step (131040). It loads with torch.load(path, weights_only=True).

The state_dict is that of MELP's ECGFMModel:

prefix	what it is
ecg_encoder.*	wav2vec 2.0 + CMSC encoder from fairseq-signals: 8 transformer layers, 768-d, 12 heads
attn_pool_contrast.*, ln_contrast.*, proj_contrast.*	contrastive pooling head, 10 queries
bn.*, logit_scale	batch norm and temperature from this stage

MELP loads every tensor except attn_pool_contrast.*, because MELP uses 12 queries. MELP's caption pooler (attn_pool_caption.*, ln_caption.*) is not in this checkpoint and starts from random weights.

Training

This checkpoint was trained by Fuying Wang with MELP's ECGFMModel (main_pretrain.py --model_name ecgfm), on ECG waveforms only, without reports. The objective is wav2vec 2.0 masked prediction plus CMSC, a contrastive loss between segments from the same patient. Training used 4 GPUs, batch 64 per GPU, lr 1e-4 and weight decay 0.2. The checkpoint's hyperparameters do not name the dataset, but MELP's pretraining data module reads only MIMIC-IV-ECG. The file was written by a ModelCheckpoint that monitored validation loss, at epoch 44, step 131040.

This file is the original Lightning checkpoint (epoch=44-step=131040.ckpt) with the optimizer state, scheduler, callbacks and hyperparameters removed. The state_dict is unchanged. MELP builds an identical ECG tower from either file.

Data and license

MIMIC-IV-ECG is open access on PhysioNet under the Open Data Commons Open Database License v1.0. The MELP code is MIT-licensed.

Citation

@article{wang2025token,
  title={From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining},
  author={Wang, Fuying and Xu, Jiacheng and Yu, Lequan},
  journal={arXiv preprint arXiv:2506.21803},
  year={2025}
}