Dynadiff: Single-stage Decoding of Images from Continuously Evolving
  fMRI
Authors: name: Marlène Careil, name: Yohann Benchetrit, name: Jean-Rémi King

Brain-to-image decoding has been recently propelled by the progress in
generative AI models and the availability of large ultra-high field functional
Magnetic Resonance Imaging (fMRI). However, current approaches depend on
complicated multi-stage pipelines and preprocessing steps that typically
collapse the temporal dimension of brain recordings, thereby limiting
time-resolved brain decoders. Here, we introduce Dynadiff (Dynamic Neural
Activity Diffusion for Image Reconstruction), a new single-stage diffusion
model designed for reconstructing images from dynamically evolving fMRI
recordings. Our approach offers three main contributions. First, Dynadiff
simplifies training as compared to existing approaches. Second, our model
outperforms state-of-the-art models on time-resolved fMRI signals, especially
on high-level semantic image reconstruction metrics, while remaining
competitive on preprocessed fMRI data that collapse time. Third, this approach
allows a precise characterization of the evolution of image representations in
brain activity. Overall, this work lays the foundation for time-resolved
brain-to-image decoding.