Title: Neural-Driven Image Editing
Venue: NeurIPS 2025
Accepted: yes
Keywords: Brain-Computer Interface

Image Editing

Visual Generation

Generative Models

Diffusion Models
Abstract: Traditional image editing typically relies on manual prompting, making it labor-intensive and inaccessible to individuals with limited motor control or language abilities. Leveraging recent advances in brain-computer interfaces (BCIs) and generative models, we propose LoongX, a hands-free image editing approach driven by multimodal neurophysiological signals. 
LoongX utilizes state-of-the-art diffusion models trained on a comprehensive dataset of 23,928 image editing pairs, each paired with synchronized electroencephalography (EEG), functional near-infrared spectroscopy (fNIRS), photoplethysmography (PPG), and head motion signals that capture user intent.
To effectively address the heterogeneity of these signals, LoongX integrates two key modules. The cross-scale state space (CS3) module encodes informative modality-specific features. The dynamic gated fusion (DGF) module further aggregates these features into a unified latent space, which is then aligned with edit semantics via fine-tuning on a diffusion transformer (DiT).
Additionally, we pre-train the encoders using contrastive learning to align cognitive states with semantic intentions from embedded natural language.
Extensive experiments demonstrate that LoongX achieves performance comparable to text-driven methods (CLIP-I: 0.6605 vs. 0.6558; DINO: 0.4812 vs. 0.4637) and outperforms them when neural signals are combined with speech (CLIP-T: 0.2588 vs. 0.2549). These results highlight the promise of neural-driven generative models in enabling accessible, intuitive image editing and open new directions for cognitive-driven creative technologies. The code and dataset are released on the project website: https://loongx1.github.io.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces LoongX, an image editing approach based on neural and speech signals. The authors collect a comprehensive set of human stimuli for image editing including EEG, fNIRS, PPG, motion and speech signals. \nThe proposed LoongX approach demonstrates image editing capabilities when conditioned on these signals providing a new paradigm beyond textual image editing. Further, the authors provide novel ablations into the importance of different brain region signals for image manipulation.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\n- General purpose image editing with neural conditioning is a novel task and field. This paper marks a potentially seminal contribution\n- Strong user study design and data collection process. Excellent, controlled study setup, good description of participant demographics and prior IRB approval \n- LMind as strong dataset contribution \n- experimental results look promising\n- Paper provides some interesting insights on importance of different signals and brain regions for image manipulation \n\n# Weaknesses \n**Missing details and strong assumptions**\nIn a lot of places the paper fails to provide relevant information, references or experiments \n- Strong architectural assumptions on input encoding using CS3 and DGF modules. While this setup generally seems suitable there Is no clear information why these exact design choices were taken. Either a strong justification, reference to prior work demonstrating the effectiveness of certain submodules or comprehensive ablations would be needed. \n- Additionally, the LoongX seemingly arbitrarily combines EEG/PGG with T5 embeddings and fNIRS/Motion with CLIP with no clear intuition \n- There is virtually no information provided on the actual DiT backbone or how it is conditioned in the input image. What's the specific architecture, parameter count? Is it pre-trained or trained from scratch? What encoding/framework is used for input image conditioning? \n- L 205 mentions that existing near-text corpora are used for pre-training without citation \n- Information on pre-training and fine-tuning is lackluster (also not present in the appendix). for how many epochs/steps is the model trained? are their any interesting observations in terms of training stability, loss curves, etc. for this new form of conditioning \n- Also obvious questions on the overall setup are not addressed. From Eq (10) we can see that the model is trained using a flow objective. But what kind of flow? Does the model use CFG? If so, what does the null-conditioning/negative prompt look like?\n- The visualization in Fig. 6 is incredibly hard to follow \n\n**Experimental Design** \nAdditionally there are some issues with the experimental design and conclusions drawn from these results \n- Choice of metrics is unconventional/unsuitable. Specifically, pixel-based L1, L2 are ill suited in assessing image editing since they do not actually measure human perceived similarity. Consequently, the standard choice in image editing literature is to use LPIPS [1] instead. \n- Similarly, reporting performance as singular value per method in a table does not accurately reflect the performance of an image editing methods. There is an inherent tradeoff between similarity with the input image and alignment with the edit instruction. This trade-off is much better reported as a curve over different hyper parameter configurations/conditioning strengths. (See InstructPix2Pix for example).\n- It is also unclear what the DINO metric is supposed to measure here  \n- OmniControl as the sole baseline for textual editing does not make for a fair comparison. Firstly, OmniControl's strength is not even in textual image editing but strong reference usage. Consequently, the reported baseline for text-only does not actually reflect the strength of current textual image editing approaches. For reference, I performed the cherry-picked examples in Fig. 7 with Flux-Kontext and it strongly outperforms all depicted examples. Consequently, the conclusions drawn when comparing against textual conditioning are heavily skewed as well. These limitations are only exacerbated by the lack of information on DiT size and training FLOPs. \n- There are a plethora of recent textual image editing works (a lot of which the authors cited themselves) that would have given a better baseline comparison for textual image editing. \n\nAdditionally the paper would benefit from actually demonstrating \"hands-free image editing\" where users would interact with the trained model in the same way as during data collection. The transfer from performance reported on a held out test-set to actual downstream usability seems less obvious for this form of interaction. \n\n## Minor Comments\n\n- Missing related work. DreamConnect [2] also performs image editing using neural signals. \n\n[1] Zhang et al. \"The Unreasonable Effectiveness of Deep Features as a Perceptual Metric\" (2018) CVPR\n\n[2] Yasheng Sun et al. \"Connecting Dreams with Visual Brainstorming Instruction\" (2025) Visual Intelligence\n\nQUESTIONS:\nPlease address the questions posed in the weakness section .\n\nLIMITATIONS:\nThe authors discuss some key limitations in Section 5.5", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an image editing method based on multimodal neural signals, LoongX. This method breaks away from the dependence on language cues and physical interactions, allowing image editing through neurophysiological signals such as EEG, fNIRS, PPG, etc. The authors also propose a large-scale dataset, L-Mind, which contains 23,928 neural signals collected from image editing samples. Two key modules, the CS3 and DGF, combine the diffusion model (DiT) to achieve an understanding of editing intentions and image generation. Experiments show that LoongX can be compared with and even surpass text-based editing methods on multiple metrics, especially when combined with speech input.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. This work is the first to integrate EEG, fNIRS, PPG, and motion signals for image editing, introducing a novel and promising direction of neural-driven editing.\n\n2. The authors present a new large-scale multimodal dataset and commit to releasing both data and code, which will benefit future research and advance the field.\n\n3. The proposed LoongX framework is well-designed, combining the CS3 encoder and DGF fusion module to efficiently extract and integrate multimodal neural features.\n\n4. The experiments are comprehensive, including ablation studies, comparative baselines, and evaluations across multiple editing types, demonstrating the method’s effectiveness and robustness.\n\n**Weaknesses**\n\n1. The method relies on strong assumptions about the correspondence between neural signals and editing intent, without sufficient validation. During data collection, participants merely read editing instructions aloud, assuming they simultaneously imagine the edits. This unverified reliance on internal mental imagery raises concerns about whether the captured signals truly reflect editing semantics. Moreover, EEG and fNIRS signals may be dominated by language processing and speech-related activity, potentially confounding the neural representation of editing intent. No control experiments (e.g., reading unrelated text or silent imagination) are included to isolate editing-specific neural patterns.\n\n2. The claim that neural signals outperform text prompts in intuitive visual tasks (e.g., background editing) is contradicted by qualitative results. In Figure 7 (row 1), the Neural and Neural+Speech methods fail to execute the core instruction “place the cat above it,” while the Text-based method achieves the intended spatial manipulation. This inconsistency suggests that neural signals may struggle with structural understanding.\n\n3. The approach depends on specialized hardware (e.g., fNIRS, PPG, EEG), limiting its practicality and scalability in real-world settings.\n\n4. The dataset includes only 12 participants, limiting generalizability. Broader evaluation across diverse populations (e.g., users with disabilities, different ages, or cultural backgrounds) is necessary to assess robustness and applicability.\n\nQUESTIONS:\n1. I am not an expert in neural signal processing, so I am not sure whether relying solely on participants reading the editing instructions aloud is sufficient to ensure that the collected signals are truly related to the editing intent. Please clarify how to ensure that the collected neural signals represent editing intent rather than language comprehension or speech production.\n\n2. In Figure 7 (row 1), the neural-based methods do not fulfill the instruction's spatial constraint (“place the cat above it”). Please address this inconsistency with the claim that neural signals are superior for background editing.\n\n3. Discuss the practicality of your approach given the reliance on EEG, fNIRS, and PPG hardware.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces LoongX, which is a multimodal image editing approach that goes beyond the limits of text and image based editing. The authors acknowledge that the current way of instructing the models may be insufficient for certain types of edits, and offer a solution that is empowered with multiple signals from different modalities such as speech, EEG signals and head motion. The proposed model involves a diffusion transformer, which accepts multimodal signals instead of text only. As a part of the proposed method, LoongX proposes two modules to effectively encode different modalities, which are labeled as Cross-Scale State Space Module and Dynamic Gated Fusion Module. Given the quantitative evaluations with multi-modal editing methods such as OminiControl and qualitative editing results, LoongX is able to perform edits effectively while broadening the range of editing capabilities of diffusion transformers.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The proposed method expands the multi-modal inputs used in editing from image-text pairing to multiple signals involving inputs such as EEG, head motion and speech. This is a significant effort towards achieving true multimodality, and in expanding the editing tasks available.\n- LoongX introduces two modules that are crucial in interpreting multi-modal inputs where general purpose encoders are not available (which is not the case for image and text representations).\n- Over the qualitative results, the proposed method shows that edits that can not easily expressed with text prompts are made possible, such as Figure 10 (a), This is a significant expansion over the existing editing methods.\n- Quantitative results shows that the proposed approach is competitive and even surpassing the baseline in certain metrics, when compared with competing multi-modal approaches.\n\n**Weaknesses**\n- The details for the baseline diffusion transformer is not clear, the authors should provide more details on the architecture or if it is fine-tuned over an existing model or trained from scratch.\n- While acknowledging that the proposed method may fall short if compared with image editing methods trained on massive amounts of text and image, it would be still helpful to compare the proposed approach with text based editing methods, in addition to multi-modal ones. It is acceptable that the proposed approach may not outperform them, but the gap should be clear for possible future work.\n- In addition to qualitative comparisons for the use of different modalities, comparisons with competing methods should also be presented.\n- The authors reported $L_1$ and $L_2$ distances as evaluation metrics for content preservation. Given the presence of DINO and CLIP-I based evaluation, this seems redundant. Authors can make their evaluation more extensive by conducting a user study instead and providing perceptual evaluation this way (which can cover both editability and content preservation), Since image editing cab be subjective, incorporating such a study would be valuable.\n- The details of the evaluations set, and the types of editing tasks are not precisely explained. The authors are encouraged to provide sufficient clarifications and discuss the failure cases of their approach.\n\nQUESTIONS:\n- Is the diffusion transformer training conducted from scratch or a pretrained network such as FLUX or SD3 is used? In addition, what are the architectural details of the transformer? This would help understanding the scalability of the method and its further potential.\n- Are there any limitations of the method in terms of the edits that are performed? What are the common failure cases? The authors are encouraged to discuss such limitations with examples (or evaluations), which would impact my judgement positively.\n- What are the details of the quantitative evaluation/ Which types of edits are evaluated?\n\nLIMITATIONS:\nThe authors provided a short discussion on limitations, addressing that abstract concepts are still hard to edit. While acknowledging this, authors are encouraged to expand this discussion. As an example, in Figure 7 (a) it appears that the color information of the cat cannot be preserved. If there are such limitations in certain types of edits, the authors should be transparent in their discussion, by including comparisons with the baseline method (OminiControl).", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}

comments: SUMMARY:
The paper proses very first framework of image editing model using Neural brain signal condition.

STRENGTHS AND WEAKNESSES:
Strength
1. This is the very first work and overall very interesting work.

Weakness

Although the diffusion model arthitecture and training setup is nice and plausible , there are major issues which must be addressed.
1. Dataset bias problem . The authors collected paired dataset using 12 subjects, and set training and test setup. 
Are the subjects of training set data and test set data independent? If the training and test data set share same subjects, then it is intra-subject experiment which lacks generalizability. Please elaborate the dataset collection protocol. 

2. Usage of speech signal. In the method, the conditional model takes speech signal along with neural signals. Also in the results, output from only neural signal shows significantly low performance compared to neural+speech. It seems that speech signal plays major role in editing. Please elaborate this part and why neural signal only setup does not work. 

I will raise the scores if  these two points are properly addressed as these points are critical on the reliability and meaning of the overall proposed work.

Minor issue: During the data. collection, are the authors properly taken the approval from ethics board?

QUESTIONS:
See weakness

LIMITATIONS:
No
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02