Title: MoRE-Brain: Routed Mixture of Experts for Interpretable and Generalizable Cross-Subject fMRI Visual Decoding
Venue: NeurIPS 2025
Accepted: yes
Keywords: brain decoding; visual reconstruction; fMRI
Abstract: Decoding visual experiences from fMRI offers a powerful avenue to understand human perception and develop advanced brain-computer interfaces. However, current progress often prioritizes maximizing reconstruction fidelity while overlooking interpretability, an essential aspect for deriving neuroscientific insight. To address this gap, we propose MoRE-Brain, a neuro-inspired framework designed for high-fidelity, adaptable, and interpretable visual reconstruction. MoRE-Brain uniquely employs a hierarchical Mixture-of-Experts architecture where distinct experts process fMRI signals from functionally related voxel groups, mimicking specialized brain networks. The experts are first trained to encode fMRI into the frozen CLIP space. A finetuned diffusion model then synthesizes images, guided by expert outputs through a novel dual-stage routing mechanism that dynamically weighs expert contributions across the diffusion process. MoRE-Brain offers three main advancements: First, it introduces a novel Mixture-of-Experts architecture grounded in brain network principles for neuro-decoding. Second, it achieves efficient cross-subject generalization by sharing core expert networks while adapting only subject-specific routers. Third, it provides enhanced mechanistic insight, as the explicit routing reveals precisely how different modeled brain regions shape the semantic and spatial attributes of the reconstructed image. Extensive experiments validate MoRE-Brain’s high reconstruction fidelity, with bottleneck analyses further demonstrating its effective utilization of fMRI signals, distinguishing genuine neural decoding from over-reliance on generative priors. Consequently, MoRE-Brain marks a substantial advance towards more generalizable and interpretable fMRI-based visual decoding.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces MoRE-Brain, a new model for reconstructing images from fMRI data. Instead of using a single model for the whole brain, it uses multiple smaller expert models, each assigned to different brain regions. A routing system decides which experts to use across time and space, which makes the model more interpretable and flexible. The authors provide detailed visualizations and analysis to connect model components with brain function.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper presents two novel and biologically inspired ideas. First, a hierarchical mixture-of-experts (MoE) structure that mirrors the visual hierarchy in the brain. Second, a dynamic time and space routing mechanism that could relate to how the brain integrates visual information over time and across regions. The design is not only technically interesting but also offers interpretability, which is rare in brain decoding models.\n\nThe authors provide rich visual analysis to support the interpretability of their model, especially linking components to brain functions. \n\nThe paper is well written and clearly organized.\n\nWeaknesses:\n\nWhile the paper discusses the challenge of cross-subject variability (Lines 42–45: “Due to inter-individual brain variability, models often require subject-specific training. While approaches like mapping subject data to a shared space or utilizing subject-specific tokens exist, they typically necessitate extra training and are difficult to generalize to new subjects.”), the proposed method still requires fine-tuning on each new subject, either partially or fully. This is similar to what methods like MindEye2 and MindBridge already do. So in practice, it still relies on partial fine-tuning for new subjects, and the issue is not entirely resolved. \n\nThe quantitative performance on standard image reconstruction metrics is not the best among competing methods.\n\nQUESTIONS:\nI noticed that the visual reconstructions from MoRE-Brain look better than MindEye2, but the quantitative metrics do not reflect this. Could the authors comment on why this discrepancy might exist?\n\nLIMITATIONS:\nrefer to weakness", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces MoRE-Brain, a novel framework for decoding visual stimuli from fMRI data. The core of the method is a neuro-inspired hierarchical Mixture-of-Experts (MoE) architecture, which mimics the functional specialization and hierarchical processing of the human visual system.\n\nThe framework operates in two stages: first, an MoE-based encoder is trained to map fMRI signals from distinct voxel groups to CLIP embeddings. Second, SDXL is used to generate images, guided by the fMRI embeddings via a novel dual-stage (Time and Space) routing mechanism.\n\nThe authors claim three main contributions: 1) a new brain-inspired MoE architecture for neuro-decoding, 2) efficient cross-subject generalization by adapting only subject-specific routers while sharing expert networks, and 3) enhanced interpretability by revealing how different modeled brain regions contribute to the reconstruction. The method is evaluated on the Natural Scenes Dataset (NSD), where it shows competitive performance and demonstrates strong results in generalization and interpretability analyses.\n\nSTRENGTHS AND WEAKNESSES:\n# Strength\n1. Novel and Well-Motivated Architecture: Using a hierarchical Mixture-of-Experts (MoE) architecture is well-motivated, as it is based on principles of neuroscience, specifically the concepts of functional specialization and hierarchical processing in the visual cortex. This represents a principled departure from more monolithic decoder architectures and provides a solid foundation for the paper's interpretability claims.\n\n2. Practical Generalization Framework: The approach of freezing the shared \"expert\" networks and fine-tuning only the subject-specific \"routers\" is an elegant and practical way to address cross-subject generalization. The experiments, which show good performance on new subjects with only a small fraction of their data (e.g., 1-4 sessions), effectively validate this approach.\n\n3. Extensive and Insightful Interpretability Analysis: A major highlight of this work is its deep dive into model interpretability, which goes far beyond what is typical for reconstruction papers in this field. The authors use a combination of techniques, including ICA on GradientSHAP attributions, to link model components to known brain networks. The analyses successfully demonstrate emergent functional specialization, showing that experts learn to focus on different spatial regions and develop preferences for specific semantic categories at higher hierarchical levels. This provides compelling evidence that the model is not just a black box but learns neurophysiologically plausible mappings.\n\n4. Rigorous Evaluation via Bottleneck Analysis: The inclusion of a bottleneck analysis to probe the model's reliance on fMRI signals versus generative priors is commendable. The results, which show that MoRE-Brain's performance degrades more significantly with restricted fMRI information compared to baselines, lend strong support to the claim that the model genuinely utilizes the neural data rather than over-relying on the priors of the diffusion model.\n\n\n# Weakness\n\n1. The presentation quality needs attention. The architectural diagrams (Figures 1 and 2) have distracting backgrounds that reduce clarity. And in Line 194, Figure 4 does not show numeric results as \"quantitatively\" states, a table the exact numerical values should be more precise.\n\n2. Insufficient Detail on Model Efficiency: one of the claim is the efficiency of cross-subject generalization. However, the paper fails to quantify this efficiency properly. It never states the number of tunable parameters in the fMRI routers relative to the total number of parameters in the model. Without this detail, it is hard to gauge the actual computational savings of the proposed fine-tuning strategy.\n\nQUESTIONS:\nPlease see the points raised in the Weaknesses section above.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
Introduces a mixture-of-experts architecture for fMRI-to-image reconstruction. Experts can specialize decoding from specific functional regions. Time vs. Space router allows the diffusion model to be guided by the fMRI signal in a more interpretable manner. Cross-subject decoding is achievable by simple finetuning of the routers while the experts are frozen.

STRENGTHS AND WEAKNESSES:
Strengths:
* unique MoE approach
* good interpretability results

Weaknesses:
* bottleneck analyses isn't particularly convincing, minimal quantitative comparisons to prior work
* generalization experiments aren't particularly comprehensive (see questions)

QUESTIONS:
* the bottleneck analyses is a bit suspect to me: the claim that MoRE-Brain is notably more sensitive to the amount of information allowed through the bottleneck does not seem to be strongly supported by the data. For all the models, the bottleneck level barely changes model performance, while MoRE-Brain is simply worse at most bottleneck levels except the last one. Any explanation as to why this is the case?
* how well does the generalization ability compare to other models?
* how well does does the model generalize across subjects if the experts were unfrozen?
* can you add qualitative results for the cross-subject decoding?
* can you present the quantitative benchmarks in a table rather than in a graph in the bottleneck analysis?

LIMITATIONS:
no discussion of societal impacts
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-03

comments: SUMMARY:
This paper introduces MoRE-Brain for fMRI-based visual reconstruction. It employs a Mixture-of-Experts (MoE) architecture where distinct "expert" models are assigned to process signals from functionally related brain regions.

STRENGTHS AND WEAKNESSES:
Strengths:
- The paper proposes a Mixture-of-Experts (MoE) architecture where each expert is dedicated to processing signals from a distinct, functionally related brain region.
- The paper is clearly written.

Weaknesses:
- The claims regarding "interpretability" in the title and abstract appear to be overstated. It is currently unknown whether the brain's visual pathway actually operates in a manner analogous to the dynamic routing mechanism proposed in the paper. If this neuro-inspired claim is based on existing literature, the authors should provide the relevant citations to support it.
- The experimental evaluation is missing a quantitative comparison with prior work on several standard metrics. Metrics such as PixCorr, SSIM, Alex, CLIP, Eff and SwAV are commonly reported in previous reconstruction papers and should be included here for a comprehensive comparison.
- The qualitative results in Figure 3 do not convincingly demonstrate a clear advantage of the proposed method over previous approaches. Furthermore, to substantiate the method's superiority and generalizability, the authors should provide generated results from multiple subjects.

QUESTIONS:
See the weaknesses.

LIMITATIONS:
N/A
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-03