Towards Interpretable Visual Decoding with Attention to Brain Representations
Paper • 2509.23566 • Published
NeuroAdapter Subject 2 E109 reproduction weights
This repository preserves our independently trained Subject 2 adapter used in the completed 982-image evaluation. It is not an official author checkpoint. inference.pt contains guidance_generator, image_proj and ip_adapter; parcel_vertices.pt fixes the selected 100 parcels per hemisphere. Obtain frozen Stable Diffusion 1.5 and the matching NeuroAdapter architecture separately.
Training used seed42, BF16, two RTX4090 GPUs, effective batch32 (8 per GPU, accumulation2), AdamW LR1e-4, weight decay1e-6, clip1, original Min-SNR gamma5 and TokenDropout. Training ended intentionally after372 epochs; this release is epoch109. Full continuation checkpoints109/300/372 remain preserved separately.
Checkpoint selection and limitations
On the fixed500-image validation set, epoch372 was better on seven of the eight reconstruction metrics; epoch109 had the highest SSIM. The predeclared balanced gate required SSIM not to decrease, so it conservatively retained109. This does not establish109 as the best overall model. No later candidate passed the K2 gate, so K4 confirmation was not triggered. The released checkpoint was fixed before its final test and was not changed using test scores.
The final982-image test used50 diffusion steps, CFG4, FP32 inference with the original VAE autocast and TF32, candidate batch1 and eight fixed ID/candidate hash seeds. Candidate0 is the K1 control. Best-of-eight uses the official Subject2 sixteen-member brain encoder and measured/predicted fMRI correlation, without GT-image selection. Seven mean metrics improve over K1, while SSIM decreases; no statistical significance is claimed. See evaluation-summary.json.
Limitations include a single training seed, validation reused during development, an unknown encoder development split, and different hardware/software/random sequences from the authors. The standard982 IDs were independently derived from the first37 Subject2 sessions; brain inputs retain the registered full40-session three-presentation averages. This is not an independent blind test or an exact reproduction of historical paper execution.
Provenance and integrity
manifest.json, inference-config.json and SHA256SUMS describe the exact artifact. No raw fMRI, stimulus images, per-image features, credentials, optimizer state, or downloaded backbone weights are included. Upstream architectures and dependencies retain their original licenses.
Upstream: https://github.com/kriegeskorte-lab/NeuroAdapter Paper: https://arxiv.org/abs/2509.23566
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support