ForeverBlue/RALF-SeizeIT2

PhysioWave-T-RALF: multimodal seizure detection on SeizeIT2 (EEG + ECG + EMG)

The PhysioWave-T multimodal model, built on the three released encoders (EEG, ECG, sEMG). Code: https://github.com/ForeverBlue816/PhysioWave.

Three independently pretrained C1 encoders -- EEG, ECG and sEMG -- fused by RALF (Reliability-Aware Latent Fusion) for seizure detection on SeizeIT2 (behind-the-ear EEG, ECG and deltoid EMG). Each encoder hands the fusion a few summary tokens; RALF predicts a reliability per modality and sample, gates the cross-attention with it, masks absent modalities out, and adds a residual of reliability-weighted unimodal logits.

Encoders frozen (the pretrained cores fixed; montage adapters, pools, fusion core and heads trained), 69.4 M parameters in all. Run ralf_frozen_szo, selected on validation auprc at epoch 5 counted from 0 (validation auprc 0.380). Classes: ['background', 'seizure'].

Test results

SeizeIT2's official split: patients sub-097 to sub-125 are the test set and were never seen before the final evaluation; 12% of the training patients were held out for checkpoint selection.

Continuous test recordings (778 recordings, 2,912 h, 181 seizures), the SeizeIT2 paper's scoring (any-overlap, means over recordings):

metric	value
sensitivity at <= 2 false alarms/h	32.0 %
sensitivity at <= 6.22 false alarms/h	58.1 %
sensitivity at <= 11 false alarms/h	74.4 %
AUROC (the dataset paper's epoch-based area)	0.741
AUPR (the dataset paper's epoch-based area)	0.0121

Window-level test set (4 s windows; background subsampled to 30 per hour):

modalities present	AUPRC	AUROC
EEG + ECG + EMG	0.399	0.773
EEG + ECG	0.394	0.768
EEG + EMG	0.265	0.691
ECG + EMG	0.337	0.746
EEG	0.252	0.692
ECG	0.313	0.746
EMG	0.192	0.615

Under corruption of one modality at severity 0.7, on top of the complete set, with the reliability the model assigned to each modality:

condition	AUPRC	reliability EEG	ECG	EMG
clean	0.399	0.91	0.85	0.91
EEG noise	0.368	0.63	0.85	0.91
EEG bad channel	0.358	0.29	0.85	0.91
EEG drift	0.332	0.32	0.85	0.91
ECG saturation	0.367	0.91	0.33	0.91
ECG noise	0.395	0.91	0.65	0.91
EMG dropout	0.403	0.91	0.85	0.29
EMG noise	0.398	0.91	0.85	0.24

A corrupted modality's reliability drops and the other two stay where they were; that is the mechanism the fusion is built around.

Training curves

(a) Validation AUPRC per epoch for the released run (bold) and for the five frozen-encoder RALF variants of the paper's schedule sweep; the dot on each curve is its selected epoch. The released checkpoint is epoch 5 of the original configuration, selected on validation AUPRC. (b) The released run's loss terms. The reliability loss holds at 0.04 through epoch 7, where the six-epoch warm-up reaches its peak learning rate; after that the reliability head drifts and the task loss follows, which is why selection stopped at epoch 5 and why the paper reports RALF as the band over all six runs rather than one curve. The variants at lr 1e-4 train stably for their full 15 epochs and select at epochs 2 to 5 with validation AUPRC 0.376 to 0.401.

Files

file	what it is
ralf_seizeit2_best.pth	the model: encoders, adapters, fusion core and heads (epoch 5)
results.json	the test matrix: every modality subset and corruption
events_cross.json	event-level scoring on the continuous test recordings
history.json	training and validation per epoch
encoders.json	which pretrained keys each encoder took
config_resolved.yaml	the run's config; encoders point at the released files

The encoders it was built on

modality	file	repository
EEG	pretrain_eeg_c1_moe_best.pth	ForeverBlue/EEG
ECG	pretrain_ecg_c1_moe_best.pth	ForeverBlue/ECG
EMG	pretrain_emg_c1_moe_epoch38.pth	ForeverBlue/EMG

The weights here already contain all three encoders as they were loaded, so they are not needed to RUN the model. They are needed to BUILD it: build_model constructs each encoder from its pretraining checkpoint before the fusion weights are loaded on top. Download the three files into one directory and point encoders.<m> at them (the released config already names them).

Loading

The model's montage adapters and heads are sized by the task's data, so it is rebuilt from the prepared windows (Fusion/seizeit2_prep.py in the code repository) and then loaded strictly:

import torch
from physiowave.config import load_config
from physiowave.fusion.train import build_model, load_tasks

cfg = load_config("config_resolved.yaml")
cfg["data"]["tasks"]["seizeit2"]["dir"] = "path/to/seizeit2"     # train.h5 / val.h5 / test.h5
cfg["encoders"] = {m: f"path/to/encoders/{f}" for m, f in cfg["encoders"].items()
                   if m in ("eeg", "ecg", "emg") and f}

tasks = load_tasks(cfg, list(cfg["data"]["modalities"]))
model, _ = build_model(cfg, tasks, out_dir="encoders_built")
ck = torch.load("ralf_seizeit2_best.pth", map_location="cpu", weights_only=False)
model.load_state_dict(ck["model"])     # strict: every key must match
model.eval()

Only train.h5 is read to size the model; nothing is trained.

Training

fusion      ralf, width 256, 8 fusion tokens, depth 2, heads 8
summary     4 tokens per modality
encoders    frozen
lr          0.0003, batch 32, epochs 60 (patience 10), bf16
loss        task CE + 0.2 x unimodal CE + 1.0 x reliability MSE + 1.0 x subset consistency
augment     each modality corrupted with p 0.5; modalities dropped with p 0.3
seed        42

One run, one seed. The numbers above are that run's, not a mean.

Code: https://github.com/ForeverBlue816/PhysioWave (docs/fusion.md)

---

Prepared by scripts/prepare_hf_fusion_release.py from ralf_frozen_szo on 2026-10-29.