Eigenbrain: frozen brain-decoding inference bundle

This repository contains frozen inference assets for three independent brain-decoding demonstrations: EEG classification, neural language decoding, and NSD Subject-1 fMRI-to-image reconstruction. It is an inference/replay bundle; it does not train a model and does not fit preprocessing transforms.

Contents

Task	Input	Frozen route	Main output
EEG classification	SEED-V eigenmode EEG	temporal/mode-attention encoder + 5-class head	predicted class
Language decoding	Alice BCI observation and prior	PoE + subject layer + QFormer + BGE tokens + Phi-4 LoRA	decoded English text
Image reconstruction	NSD Subject-1 CIFTI-4096 and modes-2000	anchored PoE + MindEye2 BrainNetwork + diffusion prior + SDXL-unCLIP	reconstructed PNG

The language route uses the public base models microsoft/Phi-4-mini-instruct and BAAI/bge-m3; the files in this repository are the project-specific alignment, projector, and LoRA weights. The image route uses a large frozen unCLIP checkpoint and requires a CUDA GPU with substantial storage and memory.

Download

Download this repository together with the companion inference code. The project code expects the following local layout:

assets/
weights/
weights/alice_lora/
weights/nsd_sub01/
models/Phi-4-mini-instruct/
models/bge-m3/

The companion download_assets.py downloads the project files from this repository and the two public base models:

export EIGEN_HF_REPO=SSp1ash/Eigenbrain
python download_assets.py

For classification and language only, omit the large NSD image files with python download_assets.py --skip-nsd.

Run

python run_demo.py --task classification --device cuda

python run_demo.py --task language --device cuda \
  --phi-path models/Phi-4-mini-instruct \
  --bge-path models/bge-m3

python run_demo.py --task image --device cuda

The image command evaluates the default ten held-out NSD examples and writes the selected qualitative reconstructions and image-level metrics under outputs/nsd_sub01/.

Reference results

The fixed compact replay gives approximately 40% accuracy for EEG classification on 10 examples. The language text route gives a mean BGE sentence cosine of approximately 0.81 on 5 examples. The NSD image route has an archived full-test CLIP 2-way identification score of 0.783; the default ten-example smoke test is not a replacement for the full benchmark.

These values describe the shipped frozen replay assets and should not be interpreted as a new independent held-out evaluation. The language alignment checkpoint includes historical oracle-style provenance, and the NSD visual demo saves only a small qualitative subset of the evaluated examples.

Files

Custom project files are stored under assets/ and weights/. The largest file is weights/nsd_sub01/unclip6_epoch0_step110000.ckpt (about 18 GB). Public base models are intentionally not duplicated here.

License and limitations

The code and frozen assets are provided for research demonstration and reproducibility. Check the licenses and terms of the public Phi-4, BGE-M3, MindEye2/SDXL-unCLIP components and the NSD data before redistribution or commercial use. The NSD route is within-subject reconstruction for Subject 1, not a cross-subject general-purpose image decoder.