Seeing through the Brain: Image Reconstruction of Visual Perception from
  Human Brain Signals
Authors: name: Yu-Ting Lan; name: Kan Ren; name: Yansen Wang; name: Wei-Long Zheng; name: Dongsheng Li; name: Bao-Liang Lu; name: Lili Qiu
Published: 2023-07-27 12:54:16

Abstract: Seeing is believing, however, the underlying mechanism of how human visual
perceptions are intertwined with our cognitions is still a mystery. Thanks to
the recent advances in both neuroscience and artificial intelligence, we have
been able to record the visually evoked brain activities and mimic the visual
perception ability through computational approaches. In this paper, we pay
attention to visual stimuli reconstruction by reconstructing the observed
images based on portably accessible brain signals, i.e., electroencephalography
(EEG) data. Since EEG signals are dynamic in the time-series format and are
notorious to be noisy, processing and extracting useful information requires
more dedicated efforts; In this paper, we propose a comprehensive pipeline,
named NeuroImagen, for reconstructing visual stimuli images from EEG signals.
Specifically, we incorporate a novel multi-level perceptual information
decoding to draw multi-grained outputs from the given EEG data. A latent
diffusion model will then leverage the extracted information to reconstruct the
high-resolution visual stimuli images. The experimental results have
illustrated the effectiveness of image reconstruction and superior quantitative
performance of our proposed method.