Neural-Driven Image Editing
Authors: name: Pengfei Zhou; name: Jie Xia; name: Xiaopeng Peng; name: Wangbo Zhao; name: Zilong Ye; name: Zekai Li; name: Suorong Yang; name: Jiadong Pan; name: Yuanxiang Chen; name: Ziqiao Wang; name: Kai Wang; name: Qian Zheng; name: Xiaojun Chang; name: Gang Pan; name: Shurong Dong; name: Kaipeng Zhang; name: Yang You
Published: 2025-07-07 18:31:50

Abstract: Traditional image editing typically relies on manual prompting, making it
labor-intensive and inaccessible to individuals with limited motor control or
language abilities. Leveraging recent advances in brain-computer interfaces
(BCIs) and generative models, we propose LoongX, a hands-free image editing
approach driven by multimodal neurophysiological signals. LoongX utilizes
state-of-the-art diffusion models trained on a comprehensive dataset of 23,928
image editing pairs, each paired with synchronized electroencephalography
(EEG), functional near-infrared spectroscopy (fNIRS), photoplethysmography
(PPG), and head motion signals that capture user intent. To effectively address
the heterogeneity of these signals, LoongX integrates two key modules. The
cross-scale state space (CS3) module encodes informative modality-specific
features. The dynamic gated fusion (DGF) module further aggregates these
features into a unified latent space, which is then aligned with edit semantics
via fine-tuning on a diffusion transformer (DiT). Additionally, we pre-train
the encoders using contrastive learning to align cognitive states with semantic
intentions from embedded natural language. Extensive experiments demonstrate
that LoongX achieves performance comparable to text-driven methods (CLIP-I:
0.6605 vs. 0.6558; DINO: 0.4812 vs. 0.4636) and outperforms them when neural
signals are combined with speech (CLIP-T: 0.2588 vs. 0.2549). These results
highlight the promise of neural-driven generative models in enabling
accessible, intuitive image editing and open new directions for
cognitive-driven creative technologies. Datasets and code will be released to
support future work and foster progress in this emerging area.