Title: BrainEC-LLM: Brain Effective Connectivity Estimation by Multiscale Mixing LLM
Venue: NeurIPS 2025
Accepted: yes
Keywords: Brain effective connectivity

Large language models

functional magnetic resonance imaging
Abstract: Pre-trained Large language models (LLMs) have shown impressive advancements in functional magnetic resonance imaging (fMRI) analysis and causal discovery. Considering the unique nature of the causal discovery field, which focuses on extracting causal graphs from observed data, research on LLMs in this field is still at an early exploratory stage. As a subfield of causal discovery, effective connectivity (EC) has received even less attention, and LLM-based approaches in EC remain unexplored. Existing LLM-based approaches for causal discovery typically rely on iterative querying to assess the causal influence between variable pairs, without any model adaptation or fine-tuning, making them ill-suited for handling the cross-modal gap and complex causal structures. To this end, we propose BrainEC-LLM, the first method to fine-tune LLMs for estimating brain EC from fMRI data. Specifically, multiscale decomposition mixing module decomposes fMRI time series data into short-term and long-term multiscale trends, then mixing them in bottom-up (fine to coarse) and top-down (coarse to fine) manner to extract multiscale temporal variations. And cross attention is applied with pre-trained word embeddings to ensure consistency between the fMRI input and pre-trained natural language. The experimental results on simulated and real resting-state fMRI datasets demonstrate that BrainEC-LLM can achieve superior performance when compared to state-of-the-art baselines.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposed BrainEC-LLM to borrow the power of large-scale LLM for effective connectivity (EC) prediction. EC as a causal discovery task can naturally utilize the knowledge base of LLM to advance the performance. BrainEC-LLM is framed with multi-scale fMRI timeseries decomposition and reconstruction to address complex causal structures in brain EC task. BrainEC-LLM has shown superior performance in 4 stimulated datasets and competitive performance in one real dataset. Zero-shot learning experiments demonstrated the high compatibility of the text embedding by LLM.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Superior performance in 4 stimulated datasets and competitive performance in one real dataset.\n\n2. Enable zero-shot learning capability for brain EC task.\n\nWeaknesses:\n1. The motivation is not clear. The authors bring a new approach to LLM based brain EC prediction due to the limited capacity of LLM to capture complex causal structures. The complexity of causal structures of EC, however, is not defined or described across the manuscript. The motivation of the second component multiscale decomposition mixing (MDM) is also hard to find. \n\n2. Figure 3 shows CUTS+ has the best performance (32 correct links), while the proposed BrainEC-LLM has fewer, 22 correct links.\n\n3. Zero-shot learning in Table 2 shows a diverse performance without explain, where SCN5 keeps the same F1 score but Smith dropped a lot.\n\n4. Table 4 shows one LLM model has manifestly outperformed another one. Performance by the bad one is the worst in comparison with baselines in Table 1, and it is less accurate than w/o LLM. This demonstrates the performance of BrainEC-LLM is highly relied on text embedding from LLM.\n\n5. Downstream application is limited in one dataset. Performance evaluation is limited with only a linear classifier, SVM.\n\n6. LLM is known as a large-scle model, and thus it holds a high computational cost. However, the computational costs of BrainEC-LLM are missed.\n\n7. Performance is unstable when alterring the hyperparameters as shown in Fig. 5. Although the best hyperparameter was found with step 0.1, the high variance of performance implict there are better hyperparameters if tuning with smaller steps.\n\nQUESTIONS:\n1. Why zero-shot learning has not test for real dataset? This will demonstrate the potential of this work for real-world applications.\n\n2. BOLD signal reconstruction is shown in the method figure. Why this loss is needed, and how is this loss affecting performance? \n\n3. Does the ablation study of w/o multiscale mixing lead to the model using pure LLM features? How is this implemented?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of estimating brain effective connectivity (EC) from fMRI time series data by proposing BrainEC-LLM. This method fine-tunes LLMs for brain EC estimation to overcome the limitations of prior approaches that relied on inference-only or pairwise querying, and addresses the shortcomings of traditional machine learning or deep learning methods, which lack the ability to capture multiscale temporal dynamics and require task-specific retraining.\nBrainEC-LLM addresses these issues through a structured architecture comprising five key components: 1) a prompt generation module to provide the LLM with contextualized knowledge about the task, fMRI dataset characteristics, and prior inter-regional correlations; 2) multiscale decomposition mixing to isolate temporal trends at various scales; 3) cross-attention with pre-trained word embeddings to bridge the gap between fMRI signals and language models; 4) multiscale reconstruction mixing for feature fusion; and 5) a self-attention module for estimating directed EC graphs. Experiments were conducted on simulated datasets (Smith, Sanchez, SCN5/7), high-resolution real fMRI data, and the ABIDE I dataset for ASD classification (downstream task). The model outperformed eight baselines on different metrics and showed zero-shot generalization. Also, the proposed method achieved the highest classification accuracy (71.1%) for ASD vs. control. Ablation studies confirmed the importance of each aforementioned module, particularly the multiscale and cross-attention mechanisms. The time complexity analysis showed that most of the computational cost stems from the LLM component, while the added modules contribute minimally.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nOverall, this paper addresses an interesting problem with a well-motivated approach. The proposed method is both innovative and practically applicable and offers new insights into the intersection of fMRI analysis for brain EC estimation and language models. The paper is clearly written, thoroughly covering different aspects of the problem and methodology. Additionally, the authors provide extensive implementation details in the appendix, which supports reproducibility for further research.\n\nWeaknesses:\n\nAlthough the method itself is clearly described, one area that could be improved is explaining the intuition behind the proposed multiscale decomposition and downsampling strategy. Could wavelet transformation have been used instead, and what are the advantages of the proposed method over wavelet-based approaches? Also, the framework figure would benefit from clearer dimensional annotations to help readers better understand the data flow and transformations across modules. Finally, more detailed and transparent information about how the datasets were used for training is necessary. The descriptions in the appendix are vague, and it is difficult to fully assess the experimental setup.\n\nQUESTIONS:\n* What is the underlying intuition for the proposed multiscale decomposition and downsampling strategy?\n\n* Could a wavelet transformation have been used instead of multiscale decomposition and downsampling strategy, and what benefits does the proposed approach offer compared to wavelet-based alternatives?\n\n* How exactly were the datasets used for training and evaluation? The current explanations in the appendix are vague...\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript proposes BrainEC-LLM for fine-tuning large language models to estimate brain effective connectivity from fMRI time series data. It introduces a multiscale decomposition mixing module to capture both short- and long-term temporal trends and a cross-attention mechanism to align fMRI features with natural language embeddings. Experiments on both simulated and real fMRI datasets show that BrainEC-LLM outperforms state-of-the-art baselines in EC estimation and demonstrates strong generalization to new datasets and downstream tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Comprehensive evaluation across six diverse datasets, including both simulated and real-world fMRI data.\n2. Comprehensive ablations: testing different backbone models, removing the prompt generation, multiscale mixing, cross-attention, LLM module, and cross-scale contrastive loss, thresholds, and dropout.\n\nWeaknesses:\n1. Table 1, Table 2, Table 3, Table 4, Table 10 do not have statistical testing between the best model and others (e.g., Wilcoxon). Where there are multiple comparisons, specific correction should be applied (e.g., Holm).\n2. Tables do not have AUROC. It is important because it is not a threshold-based metric.\n3. Figure 4 is too small. It is very difficult to draw any conclusions from it. You'll need to make them better for visualization. Although it is larger in Figure 8, the authors should compare and visualize the differences, rather than just showing Healthy and ASD as examples.\n\nQUESTIONS:\n1. How sensitive is the model to the generated prompts if you make them simpler or more complex (with additional background information that would explain some terminology)?\n2. The authors have to improve visualization and discussion when comparing brain networks. See weaknesses.\n\nLIMITATIONS:\nThe authors adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes BrainEC-LLM, a novel approach that fine-tunes large language models (LLMs) for estimating effective connectivity (EC) from fMRI data. The authors introduce a multiscale decomposition mixing module and employ cross-modal alignment via pre-trained word embeddings, aiming to address the challenges posed by complex causal structures and the modality gap between fMRI signals and natural language.\n\nSTRENGTHS AND WEAKNESSES:\nWhile the research topic is timely and the application to EC is novel, the manuscript would benefit from more detailed explanations of key components, a clearer articulation of theoretical contributions, and further analysis to justify its claimed advantages.\n\nQUESTIONS:\n1.The authors should clarify how the prior knowledge used in the method is obtained. Is it extracted from external resources, domain-specific rules, or learned during training?(page 3, section 4.1)\n2. The procedure of the “Mixing” step depicted in Figure 1 lacks sufficient explanation. Please elaborate on how the mixing operation is performed, including the rationale, inputs, and expected outcomes.\n3. The distinction between bottom-up and top-down mixing is not clearly defined. (page 4, section 4.2)\n4. In Figure 3, the authors evaluate model performance based on predicting correct, spurious, and missing connections. However, the directionality and weight distribution of connections are not considered, which could be critical in many tasks. This omission weakens the validity of the evaluation. The authors are encouraged to provide an explanation for this and consider incorporating direction and weight analysis in future work.\n5. In the ablation study presented in Table 4, the proposed method does not outperform the baseline (LLaMA3-8B). The authors should provide further analysis or justification to demonstrate the practical advantages of their method, especially when the performance gains are marginal or absent.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-05"}