Neural-Driven Image Editing
Authors: name: Pengfei Zhou, name: Jie Xia, name: Xiaopeng Peng, name: Wangbo Zhao, name: Zilong Ye, name: Zekai Li, name: Suorong Yang, name: Jiadong Pan, name: Yuanxiang Chen, name: Ziqiao Wang, name: Kai Wang, name: Qian Zheng, name: Xiaojun Chang, name: Gang Pan, name: Shurong Dong, name: Kaipeng Zhang, name: Yang You

Traditional image editing typically relies on manual prompting, making it
labor-intensive and inaccessible to individuals with limited motor control or
language abilities. Leveraging recent advances in brain-computer interfaces
(BCIs) and generative models, we propose LoongX, a hands-free image editing
approach driven by multimodal neurophysiological signals. LoongX utilizes
state-of-the-art diffusion models trained on a comprehensive dataset of 23,928
image editing pairs, each paired with synchronized electroencephalography
(EEG), functional near-infrared spectroscopy (fNIRS), photoplethysmography
(PPG), and head motion signals that capture user intent. To effectively address
the heterogeneity of these signals, LoongX integrates two key modules. The
cross-scale state space (CS3) module encodes informative modality-specific
features. The dynamic gated fusion (DGF) module further aggregates these
features into a unified latent space, which is then aligned with edit semantics
via fine-tuning on a diffusion transformer (DiT). Additionally, we pre-train
the encoders using contrastive learning to align cognitive states with semantic
intentions from embedded natural language. Extensive experiments demonstrate
that LoongX achieves performance comparable to text-driven methods (CLIP-I:
0.6605 vs. 0.6558; DINO: 0.4812 vs. 0.4636) and outperforms them when neural
signals are combined with speech (CLIP-T: 0.2588 vs. 0.2549). These results
highlight the promise of neural-driven generative models in enabling
accessible, intuitive image editing and open new directions for
cognitive-driven creative technologies. Datasets and code will be released to
support future work and foster progress in this emerging area.