LaBraM at Scale
Scaling Data and Models for EEG Pretraining
Jun-Yu Pan · Ruicheng Yin · Bao-Liang Lu · Dongsheng Li · Wei-Long Zheng · Yansen Wang
We scale LaBraM by increasing its pretraining data and encoder size. The release includes 10 encoders, trained on 25,067–79,910 hours of EEG and ranging from 5.8M to 1.97B parameters. Evaluation covers full fine-tuning, linear probing and shared EEG–Qwen3 classification across 23 EEG tasks.
- Full-FT: Atom-L* reaches 59.05%, exceeding REVE Large by 3.61 percentage points with a 46.3M-parameter encoder.
- Linear probing: Quark-G reaches 54.33%, exceeding CodeBrain by 1.24 percentage points.
- Shared EEG–Qwen3 classification: Molecule-H + Qwen3-8B reaches 50.50 ± 2.17% across the 23 tasks.
- Larger encoders improve Full-FT more consistently than LP. Data-fraction experiments suggest that the useful data scale depends on encoder capacity. Downstream performance can decline while pretraining loss continues to fall.
Available weights
| Set | Contents | Manifest |
|---|---|---|
| Default | Ten pretrained models in quark/, atom/ and molecule/ |
Default manifest |
| Best Full-FT | Ten models selected by their highest evaluated 23-task Full-FT mean | Selected-set guide · Manifest |
Weights are provided in safetensors format and include the encoder and pretraining heads. File sizes and SHA-256 checksums are listed in the manifests. The default model set totals approximately 13.3 GB.
Download and run
From the code repository, install the dependencies and download a model:
python scripts/download_checkpoint.py --model molecule-base
python scripts/download_checkpoint.py --model atom-large --variant best-full-ft
Or download directly with the Hub client:
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
path = hf_hub_download(
repo_id="Pan-Junyu/LaBraM-at-Scale",
filename="molecule/molecule-base.safetensors",
)
state_dict = load_file(path, device="cpu")
The code's FT, LP and multitask entrypoints load these files directly. For Full-FT:
MODE=ft FAMILY=79k MODEL=labram_base_patch200_200 NPROC=1 \
CHECKPOINT=checkpoints/molecule-base.safetensors DATA_DIR=/data/tasks/TUAB \
OUTPUT_DIR=outputs/tuab_ft LR=3e-4 LAYER_DECAY=0.75 SEED=42 \
bash scripts/run_eval.sh
Use MODE=lp_pool or MODE=lp_nopool for probing. Set FAMILY=25k for Quark, 49k for Atom or 79k for Molecule. Quark and Atom scale signal by 1/100; Molecule uses per-channel z-score normalization clipped at ±15. Choose labram_base_patch200_200, labram_large_patch200_200, labram_huge_patch200_200 or labram_giant_patch200_200 to match the encoder size.
Task samples contain float32 X of shape [channels, time] at 200 Hz, integer y, and ch_names. The usage guide describes the split layout, launchers and outputs.
Tokenizers
Tokenizer weights are available for all three encoder families:
Use the tokenizer and input normalization for the corresponding encoder family.
Benchmark results
| Model | Encoder parameters | Pretraining hours | Full-FT (%) | LP (%) |
|---|---|---|---|---|
| Quark-B | 5.8M | 25,067 | 52.91 | 52.83 |
| Quark-L | 46.3M | 25,067 | 53.50 | 53.98 |
| Quark-H | 369M | 25,067 | 54.51 | 52.37 |
| Quark-G | 1.97B | 25,067 | 57.22 | 54.33 |
| Atom-B | 5.8M | 49,628 | 54.74 | 52.71 |
| Atom-L | 46.3M | 49,628 | 57.07 | 53.53 |
| Atom-H | 369M | 49,628 | 57.37 | 53.54 |
| Molecule-B | 5.8M | 79,910 | 55.34 | 51.71 |
| Molecule-L | 46.3M | 79,910 | 56.70 | 53.30 |
| Molecule-H | 369M | 79,910 | 58.02 | 53.88 |
| Atom-B* | 5.8M | 49,628 | 55.11 | 53.35 |
| Atom-L* | 46.3M | 49,628 | 59.05 | 54.09 |
| Atom-H* | 369M | 49,628 | 58.86 | 54.25 |
Parameter counts refer to the encoder; pretraining hours measure the EEG collection size. Complete FT, LP and seed-level results cover 32 FT and 29 LP configurations.
Evaluation and selection
Results are mean balanced accuracy across 23 tasks and seeds 42, 10 and 5, following OmniEEG-Bench. All downstream settings use at most 40 training windows per subject–class pair; validation and test splits retain their full samples. Single-task hyperparameters are selected using seed42 test performance and reused for the other seeds. Within each fit, validation balanced accuracy selects the fine-tuned model. LP takes the higher available pooled or no-pool three-seed mean for each model and task; SleepEDF uses pooled features.
Unstarred results use the last evaluated full-data pretraining checkpoint. Atom-B*, Atom-L* and Atom-H* use one checkpoint per encoder selected by the highest evaluated 23-task Full-FT mean, then apply that same checkpoint to FT and LP.
Metadata and checksums
manifest.json maps model IDs to architectures, files and checksums. metadata/ contains export metadata and per-tensor hashes. SHA256SUMS covers released weights and metadata; the selected set also has its own checksum file.
Fine-tuned task heads, fraction ablations and EEG–Qwen3 adapters are maintained separately.
Acknowledgements and licensing
This work builds on LaBraM (ICLR 2024). The upstream MIT notice is preserved in third_party/LaBraM-LICENSE.txt. EEG encoder/tokenizer weights and original code use the MIT License.