Title: Moving Beyond Diffusion: Hierarchy-to-Hierarchy Autoregression for fMRI-to-Image Reconstruction
Primary area: applications to neuroscience & cognitive science
Keywords: fMRI-to-Image Reconstruction, Coarse-to-Fine Generation, Scale-wise Autoregressive Modeling, Scale-aware Neural Guidance

Abstract: Reconstructing visual stimuli from fMRI signals is a central challenge bridging machine learning and neuroscience.
Recent diffusion-based methods typically map fMRI activity to a single high-level embedding, using it as fixed guidance throughout the entire generation process. However, the fixed guidance collapses hierarchical neural information and mismatches the stage-dependent demands of image reconstruction.
In this paper, we propose MindHier, a coarse-to-fine fMRI-to-image reconstruction framework built on scale-wise autoregressive modeling.
MindHier introduces three components: a Hierarchical fMRI Encoder that extracts multi-level neural embeddings, a Hierarchy-to-Hierarchy Alignment scheme that enforces layer-wise correspondence with CLIP features, and a Scale-Aware Coarse-to-Fine Neural Guidance strategy that injects the neural embeddings into autoregression at matching scales.
These designs together facilitate a hierarchical reconstruction process, first synthesizing global semantics and then refining local details, akin to human visual perception. MindHier thus establishes an efficient and principled alternative to diffusion-based paradigms.
Extensive experiments on the NSD dataset demonstrate that MindHier achieves state-of-the-art semantic fidelity, 4.67$\times$ faster inference, and more deterministic results than the diffusion-based baselines. Code will be released.

TL;DR: MindHier, a coarse-to-fine autoregressive framework, uses scale-aware guidance to inject hierarchical neural features for fMRI-to-image reconstruction, surpassing diffusion models in speed, stability, and semantic accuracy.