BrainEC-LLM: Brain Effective Connectivity Estimation by Multiscale Mixing LLM
Authors: Wen Xiong
Junzhong Ji
Jinduo Liu
Venue: NeurIPS 2025 poster ()
Keywords: Brain effective connectivity
Large language models
functional magnetic resonance imaging
Primary area: machine_learning_for_sciences
TL;DR: BrainEC-LLM

Abstract: Pre-trained Large language models (LLMs) have shown impressive advancements in functional magnetic resonance imaging (fMRI) analysis and causal discovery. Considering the unique nature of the causal discovery field, which focuses on extracting causal graphs from observed data, research on LLMs in this field is still at an early exploratory stage. As a subfield of causal discovery, effective connectivity (EC) has received even less attention, and LLM-based approaches in EC remain unexplored. Existing LLM-based approaches for causal discovery typically rely on iterative querying to assess the causal influence between variable pairs, without any model adaptation or fine-tuning, making them ill-suited for handling the cross-modal gap and complex causal structures. To this end, we propose BrainEC-LLM, the first method to fine-tune LLMs for estimating brain EC from fMRI data. Specifically, multiscale decomposition mixing module decomposes fMRI time series data into short-term and long-term multiscale trends, then mixing them in bottom-up (fine to coarse) and top-down (coarse to fine) manner to extract multiscale temporal variations. And cross attention is applied with pre-trained word embeddings to ensure consistency between the fMRI input and pre-trained natural language. The experimental results on simulated and real resting-state fMRI datasets demonstrate that BrainEC-LLM can achieve superior performance when compared to state-of-the-art baselines.

BrainEC-LLM: Brain Effective Connectivity Estimation via Multiscale Mixing LLM
Wen Xiong, Junzhong Ji, Jinduo Liu Beijing University of Technology
xi xiongweneemaln but enu {jinduc, jizc1] @bjut edu en
Abstract
Pre-trained Large language models (LLMs) have shown impressive advancements in functional magnetic resonance imaging (IMRI) analysis and causal discovery. Considering the unique nature of the causal discovery field, which focuses extracting causal graphs from observed data, research on LLMs in this field still at early exploratory stage. As subfield of causal discovery, effective connectivity (EC) has received even less attention, and LLM--ased approaches EC remain unexplored. Existing LLM-based approaches for causal discovery typically rely on iterative querying to assess the causal influence between variable pairs, without any model adaptation or fine-tuning, making them ill-suited for handling the cross-modal gap and complex causal structures. To this end, we propose BrainEC- LLM, the first method fine-tune LLMs for estimating brain EC from fMRI data. Specifically, multiscale decomposition mixing module decomposes fMRI time series data into short-term and long-term term multiscale trends, then mixing them in bottom-up (fine to coarse) and top-down (coarse to fine) manner to extract multiscale temporal variations. And cross attention applied with pre-trained word embeddings to ensure consistency between the fMRI input and pre-trained natural language. The experimental results on simulated and real resting- state fMRI datasets demonstrate that BrainEC-LLM can achieve superior performance when compared to state-of-the-art baselines. The code available at https: LLM
Introduction
Brain effective connectivity (EC) estimation has attracted significant scientific attention and has been widely used in clinical studies involving Alzheimer's disease [9] 665. schizophrenia [4], depression [52] and autism spectrum disorders [7]. Based on application needs, the most commonly used neuroimaging modality is functional magnetic resonance imaging (fMRI).
In recent years, traditional machine learning (ML) and deep learning (DL) have made significant progress in estimating brain EC using fMRI data [31] Traditional ML methods are often highly interpretable, and their relatively simple model structures lead to low computational complexity [6]]. However, this simplicity also means that they require extensive feature engineering, have limited capacity to handle complex relationships, and often necessitate different algorithms for different problems 44 In contrast, DL methods, with their more complex model structures, do not require manual feature extraction 41) They can automatically extract, process, and make decisions based on features from raw data 2 g. However, DL methods require separate training for specific tasks, large amount labeled data, and costly model training [33]]
Corresponding author: Jinduo Liu (jinduo bjut.edu.cn)
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

While numerous studies have explored the application of LLMs to IMRI data, most focus on analysis tasks such as decoding [TT]. Meanwhile, EC estimation is causal discovery task that aims to infer a directed graph representing causal relationships between brain regions, and typically lacks ground truth. Existing approaches applying L.MSs to causal discovery typically rely on iterative querying to assess the causal influence between variable pairs, without any model adaptation or fine-tuning, limiting their capacity to capture complex causal structures [78].
To address these challenges, we propose BrainEC-LLM, a novel approach that leverages LLMs for the first time to estimate brain EC from fMRI time series data through tinettuning rather than relying solely on inference. Specifically, prompts generation (PG) module produces prompts in the form of fMRI dataset description, prior knowledge, and task description to guide LLM, enhancing its ability to utilize temporal information and cross-brain connectivity from fMRI data. Next, to capture intricate fMRI multiscale temporal variations and reduce cross-modal disparities, we propose multiscale decomposition mixing (MDM) module. This module performs bottom-up (fine to coarse) and top- down (coarse to fine) mixing of short-term and long-term multiscale fMRI trends, and then aligns multiscale IMRI features with pre-trained word embeddings using cross attention. Then prompts are served as prefixes and concatenated with the multiscale fMRI embeddings before being delivered into LLM. Finally, the multiscale features output by the LLM are fed into multiscale reconstruction mixing (MRM) module to achieve the fusion of multiscale information, and the brain EC network is estimated using self attention. Our key contributions can be summarized as follows:
We propose BrainEC-LLM, the first method to fine-tune LLMs for brain EC estimation, in contrast to prior inference-only causal discovery methods.
We propose PG to generate task description, dataset description and prior knowledge, which allows LLM to comprehend temporal dependencies and cross-brain connectivity.
We present MDM, which disentangles complex fMRI multiscale temporal variations from a new perspective, enhancing LLM's understanding of these dynamics.
Extensive experiments both simulated and real resting-state fMRI datasets demonstrate that BrainEC-LLM outperforms the current state-of-the-art baselines.
2 Related Works
2.1 Brain Effective Connectivity Methods
Brain effective connectivity (EC) estimation seeks to uncover causal graphs that characterize the influence patterns among different brain regions using fMRI data. Since real fMRI datasets lack ground truth EC, autoregressive models have been widely adopted in this field. These models exploit the temporal autocorrelation within IMRI time series to predict the data itself, allowing the model to learn more representative and informative connectivity patterns.
Existing EC estimation approaches can generally be categorized into two groups: traditional machine ML methods and DL methods. Traditional ML methods for estimating brain EC networks include DCM 7 20]. SEM 17 and GC [55 These methods are advantageous for their interpretabilltyy However, their model structures are typically predefined and sensitive to noise and high-dimensional data. In contrast, DL methods have shown greater capability in estimating brain EC networks, particularly when processing high-dimensional and complex data. Examples of these methods include ACOCTE [40), RL-EC [42), CR-VAE (36), Meta (30), MetaRLEC (75) and CUTS+ (12). While these models achieve good performance on brain EC estimation tasks, they require separate training for different tasks.
2.2 Large Language Models
Numerous research efforts have shown that pre-trained LLMs can be effectively adapted to unseen tasks through fine-tuning [14 62 74. While some works raise concerns about their robustness 660 7 others have demonstrated that, with proper encoding or architectural tuning, LLMs can achieve strong performance across diverse time series tasks [24] 32) As a domain
branch of causal discovery, EC estimation aims to infer causal graphs that describe the directed influence among brain regions based on fMRI time series data. Although LLMs have been initially explored in

---

Syame
Maliisale Recosstraciion
Sct Fomuaa
Rover de A411
Promppps Generation
SUPRE
Fnderdep
ANPPEEE EIRE
Row
Degatinn Hoat
Comncct by Nervet
Fawerd
Rodeved
Multicalle Decompesiiian Mising
Figure 1: The overview of BrainEC-LLM framework. First, enhance the inference capability of the LLM using ! prompts generation module. Next, we decomposition the complex variations in IMRI time series with ② multiscale decomposition mixing module, aligning pre-training word information of LLM with fMRI time series data. Finally, the multiscale features produced by LLM are combined across different scales using multiscale reconstruction mixing module.
both IMRI analysis and causal discovery, their application EC estimation for IMRI data has not yet been investigated.
IMRI Analysis. LLMs have been increasingly explored in the context of fMRI time series data, such as decoding linguistic information from brain signals [5], reconstructing language from non-invasive brain recordings (68), and modeling the alignment between text and brain activity 19 However, directly applying L.MS to fMRI time series poses challenges due to the complex spatial-temporal patterns and modality gap between fMRI time series and language representations.
Causal Discovery. LLMs have demonstrated strong capabilities for zero-shot inference of causal structures and identifying complex dependency relationships (5 [59 47). However, current approaches require LLMS to iteratively evaluate variable pairs for causal determination [72] 6 leading to
computational inefficiency while failing to capture global dependencies and introducing model biases.
3 Notation and Problem Statement
In this paper, we utilize lowercase font (e.g., v;) to indicate the (i-th) brain region, adopt uppercase font (e.g., A) to denote matrix, use uppercase calligraphic letters (e.g., 2') to represent dirressimaii tensor, and math bold italic letters (c.g., Z) are employed to signify nur-dimensional tensor.
The problem of brain EC estimation can be formulated learning from brain data (e.g., fMRI), We then introduce the definition of brain effective connectivity (EC). Brain EC can be represented as directed graph =< v, A where stands for the set of nodes, and each node brain region or region of interest (ROI). denotes brian EC adjacency matrix, where As symbolizes the effective connectivity from brain region i; to Vj, indicating that us has a causal influence on uz.
represents
4 Methodology
BrainEC-LLM consists of three key components as shown in Figure | A prompt generation module first guides pre-trained LLMs (e.g., Llama 3, Mistral) by generating prompts tailored the input IMRI time series. Then, multiscale decomposition mixing module performs short-term and long- term mixing, followed by cross attention with pre-trained word embeddings for alignment. Finally, reconstruction module fuses the multiscale features output by the L.M.
4.1 Prompts Generation
Prompts serve as a straightforward and effective technique for enhancing the reasoning capabilities of LLMs (69. To harness the powerful capabilities of LLMs, we propose a comprehensive prompt generation module. First, the identity of LLM is established through system messages. Following this, specific task description is provided, clearly outlining the objectives for the L.M.. Next, the module describes the fMRT dataset using a user message. This description includes detailed information about the dataset, such the generation and dimension of fMRI time series data. Additionally, the user
3

---

message incorporates prior knowledge about the relationships across brain regions, derived from Pearson's correlation coefficient, helping the model focus on potentially relevant region pairs. Our prompt design is intended to provide the L.M with a structured representation of inter-regional brain relationships by explicitly encoding the Pearson correlation matrix, where each row and column corresponds to a specific brain region. To preserve consistent and interpretable mapping between brain regions and their associated fMRI ROI time series, numerical indexing is incorporated into the prompt along with corresponding explanations, ensuring the model can clearly associate each index with specific brain region. Examples of prompts and their mapping from Pearson correlation matrix to fMRI time series are provided in the Appendix A The LLM tokenizer then converts the prompts above into a sequence of tokens, enabling LLM to process and understand the prompt efficiently.
4.2 Multiscale Decomposition Mixing
fMRI time series exhibit distinct patterns at different temporal sampling scales. For instance, high- resolution fMRI data recorded per second (lower-level scale) provide higher spatial and temporal resolution, enabling the capture of subtle changes and local neural dynamics. In contrast, low- resolution fMRI data recorded hourly (higher-level scale) tend to reflect the overall or global activity patterns of brain regions. These observations naturally motivate the adoption of a multiscale analysis paradigm to disentangle the complex temporal structures embedded in fMRI signals. A multi-seale perspective enables the model to separate and capture diverse components of brain activity across different timescales, which is crucial for accurately modeling temporal variations.
Specifically, for short-term trends, where fine-grained temporal fluctuations such as rapid neural responses are more prominent, we adopt a bottom-up mixing strategy that emphasizes detailed local information. Conversely, for long-term trends, which emphasize broader patterns like sustained functional connectivity, we employ a top-down mixing strategy to incorporate high-level global structures.
Decomposition. The fMRI time series is initially normalized using reversible instance normalization 34 to achieve zero mean and unit standard deviation, addressing the issue of distribution shift in IMRI time series data. Different phenomena and patterns may emerge at various time scales, with fine scales capturing detailed patterns and coarse scales emphasizing broader changes [46]. By integrating information from multiple scales, LLMs can more accurately predict behaviors or trends, reducing errors in the process. Concretely, we employ downsampling to decompose the complex patterns [5]] resulting in M time series at different scales:
X AugPool K,)
M 17-1) 1},
where X; E BRIETIXN indicates the fMRI time series of i-th scale. The lowest level sequence X represents the input fMRI time series X, containing subtle temporal variations, while the highest level sequence -M--1 captures macroscopic variations.
[ the multiscale fMRI time series are then decomposed to short-term multiscale trends S {81 ..... SM} and long-term multiscale trends T (T....., TM using series
As illustrated in Figure
decomposition block [64]
T
TT
Subsequently, feedforward network is applied each trend independently. Due the distinet variations contained in the short-term and long-term multiscale fMRI time series, they require separate processing manage complex time variations.
Bottom-up Mixing, For short-term multiscale trends, lower-level detailed fMRI time series (c.g.Si) offers higher spatial and temporal resolution, capturing small changes and local dynamics in brain activity. In contrast, higher-level coarse fMRI time series (c.g.SM) integrate a broader range of brain activity, reflecting more holistic trends and patterns. Therefore, we adopt bottom-up approach from lower-level fine-scale fMRI time series upward to provide additional information for higher-level coarser-scale fMRI time series:
S. = + +
where (--) signifies MCdem1TN block (43. To better utilize temporal features and cross-brain connectivity in fMRI time series, we use a ModernTCN block to connect fMRI features across different scales, and the detailed framework of MCNemTTN block is illustrated in Figure 2

---

Unlike in computer vision, ModernTNN per- forms patch embedding 488] to preserve corre- lations among variables rather than embedding the N variables to dimension D. Denote patch length as P and stride as S. Through this patch- ing process, we obtain patches of length and the total number of patches N,- = - + 2. For the short-term fMRI time series trends at i-th scale, we obtain P. IR NNxx,> x. Then, one-dimensional convolution is applied to these patches, resulting in E, N Next, ModemTCN uses depthwise convolution to extract the temporal dependencies of short-term fMRI patches, convolutional feed forward network (ConyFFN) with group set to N to extract feature representations of different brain regions, and ConyFFN with group set to to extract correlations between brain regions. Among them, ConvFFN includes two pointwise convolutions with intermediate GELU activation function. Finally, these patches are projected and aligned with fMRI features at a different scale.
Parching
Embedking
DwCCoev
Com-7N
Greupex
ComFFN
Groupe
Projeeoiii
Inpux
Ouiptt
ModernTCN block
Figure 2: The architecture of Modern block.
Top-down Mixing. For long-term multiscale trends, unlike short-term multiscale trends, small changes can introduce noise when capturing macro trends in brain activity, while higher-level coarse fMRI time series (e.g. Tar) typically represent the overall activity of brain regions and are less affected by noise and short-term fluctuations. This relatively stable trend provides a solid foundation for lower-level fine-gra ane-grained fMRI time series (e.g. TT), making detailed analysis at lower levels more reliable. Therefore, a top-down approach is adopted:
T = T, + 2 (T+1).
We then obtained multiscale fMRI by summing the multiscale fMRI trends.
Cross Attention. order adaptively extract local semantic information, we first perform patching on multiscale fMRI time series data to aggregate similar tokens and preserve the localization of fMRI time series. However, since pre-trained LLM is trained on textual data and struggles to interpret fMRI time series data, we apply cross attention between the obtained multiscale fMRI patches and the pre-trained word embeddings to align two modalities. Here, the pre-trained word embeddings refer to vector representations of all the words in the corpus that are used to train LLM. Given the vast size of corpus, directly using all these vector representations would introduce a significant amount of task- iss-iireleevant information into the LLM. To address this issue, employ linear mapping matrix to reduce the vocabulary size, effectively merging related tokens. This transformation changes the dimension of the pre-trained word embeddings from S Dut (E) to S' Dim (F'), where S' is significantly smaller than S. Here, S represents the vocabulary size, and Ditm denotes the hidden dimensions of LLM model. Next, eross attention is applied and defined as:
e = H² W Kh Eh Vh -E"
e¹ KhrTT ))
(5)
t = dropout(so fmax
where O represents the fMRI patches at the i-th scale and the h-th head, H RNNN,
Whh E Dx & and WA. W E Piix f. By concatenating O across different heads, we derive O, This result is then aligned with the hidden dimensions of the LLM model using a linear mapping, producing the final output O E R NNN,x
This alignment of IMRI time series with natural language helps achieve more natural representations for the LLM. The resulting prompt embeddings as prefix, and multiscale IMRI embeddings are concatenated and serve as the input for the pre-trained L.MM, which is then fine-tuned using LoRA.
4.3 Multiscale Reconstruction Mixing
As shown on the right side of Figure [ we discard the prefix part to obtain multiple scales of fMRI time series data. Subsequently. linear mapping is performed to generate patches, followed by flattening and additional linear mapping to produce the predicted multiscale fMRI time series data X". We further split 2" and express it in concatenation form:
X' (2i, ..., X1e
(6)
5

---

where X' E RM To fully utilize the multiscale information, we first align time dimensions of fMRI time series data across multiple scales through linear mapping, and then summed them:
M-1
V = W. bias,, [I, M) Y'
V
where W, and bias; are learnable parameters. In the absence of ground truth in real [MRI datasets, employ an autoregressive model that leverages autocorrelation (effective connectivity) to predict fMRI data itself. This strategy enables the model to learn more representative and informative connectivity structures. This is implemented using self attention, where the attention weights serve as brain EC, and the output of the self attention corresponds to the reconstructed fMRI time series data Y RRTN derived from the brain effective connectivity E R N
4.4 Overall Objective Function
Our final loss consists of three components: reconstruction loss Lree sparsity loss Sspas directed acyclic loss Ldng and cross-scale contrastive loss Leses which are as follows:
C = Lree + O'spa Cusa + Oldgg Ldtg + Cresehesee
(8)
where O'spos Ogdag and Cceae are weights coefficient.
The reconstruction loss, quantified as Euclidean distance between the reconstructed and actual fMRI time series data, reflects the accuracy of reconstruction. fMRI signals exhibit rich temporal autocorrelation and dynamic interactions between brain regions, both of which are indicative of the EC network. By training the model to reconstruct the BOLD signal (i.e., to predict fMRI data from itself), the model is encouraged to learn the causal relationships (effective connectivity) among brain regions, even without manual labels. Thus, minimizing the reconstruction loss between the predicted and actual fMRI signals enables model to discover effective connectivity in an unsupervised manner.
The sparsity loss is LI regularization of Brain EC network A, which ensures brain EC derived from BrainEC-LLM is a sparse graph.
When brain EC network is directed acyelic graph (DAG), directed acyclic loss 777 is necessary to constrain the estimated brain EC. This loss is optional and depends on whether the ground truth is DAG:
Lang = tr(exp(A @ A)) - NN,
where 3 indicates Hadamard product.
To maintain consistency in the information across different scales of fMRI time series, we employ cross-scale contrastive loss. Considering that in multiscale decomposition mixing module, the (i + 1)- th scale is downsampled from i-th scale, maximizing their similarity (positive samples) ensures cross-scale consistency, while minimizing similarity with non-adjacent scales (negative samples) reduces irrelevant features. Prior to calculating cross-scale contrastive loss, a nonlinear projection head is required, which has proven be effective [10]. Let IR Bxx x represent the multiscale IMRI time series data obtained after passing through the projection head. Then the contrastive loss is defined as follows:
gZ Zk+1) = exp(psmmmmmmmccc ZZ.. 2i 1)/),
B M-1
log g(Z), z Z/t12 Ejjs 2s 1 (Z); Z
ci,+1
(10)
esc
2B(M
1)
6m 1m1
Lese ((,i++) + c(++1.1)
where Z stands for the 1MRI time series data at i-th scale of b-th sample, 7 represents temperature coefficient and sim denotes cosine similarity. Theorem 4.1. The maximum lower bound for Capt is
Cept 2log(M - (1(),,,, + I ZT)),
(1I)
where T(Z,, 2241) stands for the mutual information between 7 and 7b Lopt denotes the optimal contrastive loss.

---

The detailed proof is shown in Appendix B By proving the bounds of csse we can maximize lower bound on mutual information between different scale representations fMRI by minimizing
fose The algorithm description and pseudocode can be discovered in Appendix C
Experiments
5.1 Experimental Setups
The benchmark simulated datasets we use are Smith dataset [58] and Sanchez dataset [54] both generated by dynamic causal model. Specifically, Sanchez dataset offers higher temporal resolution and acquisition frequency compared to Smith dataset and minimally affects the non-Gaussianity of the BOLD signal, configuration commonly observed in real brain networks 45
Simulated fMRI Dataset.
Considering that real resting state fMRI dataset we use includes 7 ROIs and features cycles, the generated simulated dataset contains and nodes, named simulated complex network with node 5 (SCN5) and simulated complex network with node 7 (SCN7), respectively. Details on all benchmark simulated datasets used are provided in Appendix D.1
Furthermore, generate simulated dataset using CDRL [79.
Real Resting-state [MRI Dataset. To assess the performance of methods under real BOLD data conditions, we utilize high-resolution 7T human resting-state fMRI data [56] from medial temporal lobe. The real resting-state fMRI dataset comprises 23 healthy adults, each with an acquisition time of 1.0 seconds and session duration of 7 minutes per subject, yielding fMRI time series of 421 data points. We consider seven ROIs from the medial temporal lobe, specifically cornu ammonis (CAI), cornu ammonis 2, 3, and dentate gyrus (CA23DG), subiculum (SUB), entorhinal cortex (ERC), brodmann area 35 (BA35), brodmann area 36 (BA36), and parahippocampal Cortex (PHC). These regions assigned the numbers 1 through respectively. More details about the real resting-state dataset can be found in Appendix D.2
Post Process and Evaluation. The BrainECLLM directly outputs directed weighted graph representing the brain EC network. For evaluation, we apply a post processing step to generate a binary EC matrix that matches the ground truth in the simulated fMRI dataset. Similarly, for real fMRI data, we visualize the EC by indicating the presence or absence of directed edges between brain regions. This approach is consistent with common practices in the literature when true connectivity weights are unavailable 22 18]. The details of post processing can be found in Appendix D.3
To evaluate the effectiveness of methods, employ the following five commonly used evaluation metrics: Precision, Recall, F1, Accuracy, and Structural Hamming Distance (SHD). See Appendix D.4 for specific calculations.
Implementation Details. We Llama3-8B [6] the default backbone model unless otherwise specified. All our experiments are repeated three times, and we report the average results. All experiments are conducted on a single Nvidia L20-48GB GPU. For Smith dataset, training BrainEC- LLM takes approximately 5.5 hours. Model specific time complexity analysis and efficiency analysis experiments are shown in Appendix D.5
We train the model unsupervisedly in an autoregressive manner. Detailed training methods and hyperparameter settings can be found in Appendix [D6]and Appendix [D.7 respectively.
Baselines. Eight baseline methods are harnessed for comparison with the proposed method, and the baseline methods are as follows: spDCM (20), IsGC [15, ACOCTE 40), RL-EC [42), CR-VAE 36),
MetaCAE 300, MetaRLEC (75), CUTS+ Extra descriptions of the baseline methods can be
accessed in Appendix D.g.
5.2 Results on Simulated IMRI Dataset
We run BrainEC-LLM and the 8 baseline methods three times (group analysis) for all subjects in each simulated dataset and calculate the mean and variance for these runs. is important to note that some methods produce identical results across multiple runs, resulting in variance of O. Our main results are presented in Table where BrainEC-LLLM outperforms baselines most cases. Smith, Sanchez, and SCN5 all have EC networks with 5 nodes, but SCN5 features more edges, resulting in
7

---

Table 1: The mean and variance of the nine methods on the simulated fMRI datasets. The best and second- best values are highlighted and underlined.
Methods
Dutasets
Metries
spDCM 2014
CR-VAE
MeteAA 2024
MetaBLEC
CUTS
BrainEC-LLM
2017
2022
2022
2023
[022
2024
(Ours)
Precision
0.50=#0.00
0.45+0.P
0.55+0.100
0.38+0.55
0.33=0.09
0.57-0.06
Recall
0.47:-0.09
0.73::0225
Smith
0.44.00.0
0.5510:00
0.452:0.1
0.5610021
0.65-1012
0.661007
0.37=0.06
Accuracy
0.80100.0
0,5010:00
0.812:0.05
0.83:10.04
0.7610.11
0.8220.08
0.7240.03
0.72=0.04
0.84
SHD
5.00=00.00
5.00=0:00
4.67=0.25
4.33-40.94
6.00+22.33
4.26+20.93
7.00+0.56
7.00=0.78
Precision
0.57+0..0
0.76+0006
0.80
0.50-0004
0.50++0.07
0.80+0004
Rocall
0.57=20.00
0.86:0000
0.57
0,57400.7
0.7620013
0.29:10.11
0.5700.44
0.86=0.00
0.97 0.02
Sanchez
0.57=20.00
0.70
0 0.6
0.6 2..05
0.60:0007
0.37
067
0.74-0022
0.86 0.07
Accumacy
0.76-=0.000
0.00::000
03
0.72:0005
0.72- 04
084
010
0.91
SHD
6.00+00.00
5:0000000
4.33+047
7.0500822
6.89++0.85
400+00899
4.33-047
2.10
Precision
0.40:10.00
0.27:0000
0.50
0.67
0.40
0.36-10.02
0.50.00.1
0.45:0002
0.83-00.44
Recall
0.75
0.38.0000
0.13
0 0.25
0.67 10.66
0.58
0.000.004
0.63-00.3
0.63-10.05
SCNS
F1
0:52-=0.000
0.32+0.00
0 0.20--0044
D. 1.3-+0.04
0000-004
0 0.03
0.300003
0.53-000
0.71-0.04
0.04
Accumcy
0.56-20.00
0.48+00.00
0.68
0.72--0.12 12
0.570004
0.50-
000
0.:4000P
0.84 0.10
SHD
11.00=0.00
13.00+00.00
0.00
6.82-10.85
10.871994
11.67:00.77
8. 0.445
9.06=00.67
4.26-0..8 0.83
Presision
0.67=0000
0.30:0000
0.77
0.65
0.57:0000
0.08
0.52
0.34:0000
0.62-00002
Rocall
0.1320.000
0.60:0000
0.22==003
D. 0900
0.800014 14
047
0.5K-0002
0.61 0.05
SCN7
F1
0.22
0.4010.00
0.3 0.34
0.56.10.03
0.48
0.48.0.04
0.43-00:3
0.61-00.03
Accuracy
0.00
0.4520.00
0,7110.01
0.7510.01
0.52.0.05
0.M.00.05
0.69
0.52-00.02
0.76
SHD
11,0000.00
27.0010:00
12.67=0.47
11.67 10.47
23.6711.70 1.70
27.31
14.9
23.3310.94
11 06.60..91
(a) spDCM
(b) IsGC
(c) ACOCTE
(d) RL-EC
(c) CR-VAE
(f) MetaCAE
(g) MetaRI.FC
(h) CUTS+
(i) BrainECCLLLM
Figure 3: Brain EC estimated by nine methods on real resting-state IMRI dataset. The black lines represent correct connections, the green lines stand for spurious connections, and the red lines indicate missing connections.
more complex EC network. SCN7, on the other hand, has 7 nodes with complex EC network. We observe that most methods perform poorly on the Smith dataset, finding that aligns with existing literature [39]. Baseline methods such as IsGC, ACOCTE, and RL-EC perform relatively well on the first two datasets, but their performance declines as the network complexity increases. CR-VAL CREV shows more consistent performance across all four datasets. BrainEC-LLM excels on the first three datasets; however, its performance diminishes on the fourth dataset, likely due to the increased number of nodes. Overall, BrainEC-LLM frequently outperforms the baseline methods across the four different datasets.
To assess the significant differences between BrainEC-LLM and other baseline methods, t-test analysis and Wilcoxon test with Holm correction are performed, with results presented in Appendix The analysis reveals that, except for RL-EC, which shows no significant difference in the evaluation metric SHD, BrainEC-LLM is significantly different from the baseline methods in all other cases.
E.1
5.3 Results on real resting-state fMRI Dataset
Since no established brain EC network is available for existing real resting-state IMRI data, we evaluate methods by referencing [54]. We conduct experiments on the left and right hemispheres using BrainEC-LLM and other baseline methods. The resulting EC networks are displayed Figure In the left hemisphere, BrainEC- correctly identifies 11 connections, generates 6 spurious connections, and misses correct connections. Similarly, in the right hemisphere, BrainEC-LLM identifies 12 correct connections, produces 6 spurious connections, and misses 5 correct connections. In contrast, spDCM and IsGC generated relatively sparse graphs, detecting fewer edges and consequently missing many correct connections. Although CUTS+ misses fewer connections, its higher SHD indicates that BrainEC-LLLMM achieves superior overall accuracy EC estimation, as SHD penalizes both missing and spurious connections. Overall, BrainEC-LLM performs best in left hemisphere and ranks second
8

---

Table 2: Zero-shon learning results of BrainEC- LLM.
Datasets
Precisiont
Recalff
Flf
Accuracyt
SHD
Smith :SCNS
0.83
0.63
0.71
0.84
4.0
SCN5-+Sminh
0.68
0.51
0.58
0.76
5.0
Table 3: Classification results on ABIDE
Table 4: Classification results on ADHD.
Methods
Precision S3t
Recall(S
Accuracy(8)
Methods
Procision(551
Recall/Se1t
F1yS
Acouracy(5
spDCM
64.15+980
62.8-55100
63.51+7.733
64.74-4.31
69.15+8.60
63.84+5.31
38-6666
66.73-5.97
65.32+892
63.0-481
7087=5.62
609545.24
65:3+5542
ACOCTE
55.34=951
55.67=3.06
55.50=66.41
57.8223.76
ACOCTE
63.9::417
62.1342.91
62.7543.42
61.29=1141
RI-RC
55.76+9998
53.74-560
54.73+7.222
56.62-4.03
RI-BC
61.86-+8888
60424343
61.13++4.22
59.72-1775
CR-VAR
63.12-9.3
63.33-1336
63.22+41.9
64.35+3.75
CR-VAE
66.9+12.11
63,84+-4422
64.0645.611
646114484
MecaCAB
62.34=951
61.6=4311
62.00=5.93
18:439439
MetaCAE
64.5947.92
63,9544.14
64.20+45443
64.31=4997
MctRLEC
62.35:8.71 65.97-3388
64.1445.00
MctaLECC
63.7248.23
62.8014.82
CUTS+
67.82.8.34
65.41_-1133
67.52116880
69.111388
CUTS+
66.5017.52
65.6215.32
66.01
65.56.11.25
BminEC-LLM
69.20:1..88
81.11=8..19
92:31 71.11:33.00
BrainEC-LLM
67.516.22
72.3814.91
69.82.15.37
67.87_55.02
only to spDCM right hemisphere. A case study for visualization of LLM inputs is detailed in Appendix E.2
5.4 Zero-shot Learning
This task evaluates the transferability of BrainEC-LLM from source domain to target domain, specifically assessing how well the model performs dataset A (without any training data from A) when already trained on dataset B. Table demonstratee that our method performs well, even when it has never encountered another dataset. The diverse zero-shot performance reflects differences in network complexity between Smith and SCNS datasets. SCN5, with 5 nodes, ares, and 4 cycles introducing quadratic nonlinear causality, represents more complex structure than Smith (5 nodes, 5 arcs, O cycles). When transferring from Smith to SCN5, the model maintains stable Fl performance, as SCN5's complexity allows leveraging multiscale features learned from simpler Smith dynamics. Conversely, SCN5 + Smith transfer shows performance degradation, likely due to overfitting to SCN5's nonlinear patterns, making adaptation to Smith's 's simpler acyclic structure suboptimal. This indicates better model transfcrability to complex structures than simpler ones in zero-shot scenarios.
5.5 Downstream Tasks (Brain Disease Classification using EC networks)
To further validate the discriminative power of BrainEC-LLM for EC estimated on real fMRI dataset without ground truth, we apply it to the ABIDE I datase and ADHD datasef to estimate brain EC network. The resulting network then used as input to SVM classifier for downstream classification tasks. The 10-fold cross validation results of this analysis are presented in Table 3 and 4 which demonstrate that BrainEC-LLMM not only effectively estimates EC but also yields strong performance in downstream classification tasks.
Figure illustrates a comparative analysis of brain EC networks in healthy controls (HCs) versus individuals with autism spectrum disorder (ASD), based on AAL-90 atlas (90 regions) 13). To analyze the most critical effective connectivity between brain regions, we preserve the top 5% highest- scoring edges for visualization. Previous literature has consistently reported that individuals with ASD exhibit increased connectivity in right middle temporal gyrus (MTG) [7].. decreased connectivity in temporal pole (TPO) 23 and supplementary motor area (SMA)
63 and asymmetric connectivity alterations in the middle frontal gyrus (MFG), with reduced connectivity in the left hemisphere and increased connectivity in the right hemisphere [23 28 The connectivity patterns observed in figure align with these previous findings. More details can be found in Appendix E.3
5.6 Model Analysis
Ablation Study. We perform ablation studies on model backbone, model module, and the loss function, with the results presented in Table (5] "w/o LLM" refers to removing the pre-trained LLM entirely from the BrainEC-LLM framework. Our results indicate that including Lase provides a slight

---

Eppepesrent
(a) Influence of soft threshold hyperparameter 7/-
(a) HCs.
(b) HCs.
(c) HICs
(b) Influence of the dropout.
(d) ASD.
(e) ASD.
(1) ASD.
Figure 5; Hyperparameter analysis on the Smith dataset (the starred results are the best results).
Figure 4: Comparison of brain EC network be- tween healthy controls and ASD patients the ABIDE dataset Using BrainEC-LLM.
improvement over not including it. For other components, such as the prompts generation module and multiscale mixing module (includes both multiscale decomposition and mixing, without loss Lese), the improvements are more pronounced in BrainEC-LLLM Additionally, the results using Llama 3 as the backbone manifestly outperforms Mistral. More results be found in the Appendix E.4
Hyperparameter Analysis. We conduct experiments on Smith datasets to evaluate the param- eter sensitivity of BrainEC-LLM. Figures (a) and 55 (b) display the experimental results for soft threshold hyperparameters and dropout variations, respectively. When the soft threshold is set to 0.5, all metrics are optimal ex- cept for Recall, which remains sub-optimal, a pattern also observed with dropout. Consequently, the soft threshold is ultimately set 0.5, and dropout is 0,01. Refer to Appendix E.6 for further details.
Table 5: Ablations on Smith dataset.
Variant
Precisiont
Recallf
FI
Accuracyt
SHDI
Llama3-8B (Default) Mistral-7B
0.57=00.6
0.80
0.67::.111
0.84::0.06
4.02:20.84
0.29
0.40
0.33:0.13
0.68:00.88
7.65
w/o Prompis Generation w/o Multiscale Mixing
0.38 0.08
0.6010.15
0.4610.144
0.72.1..08
7.10:10.33
0,20::0.88
0.2120.12
0:20::0113
0.68:10.66
7.88
w/o Cross Attention
0,23:10112
0.32.10.13
0.27:10.12
0.53.1..06
11.80:10.93
w/o
0.33.0.11
0.4240.14
0.3540.14
0.724:0.08
7.14
w/o Lese
0.58.10.04
0.70.10.16
0.57.10.09
0.78.10.07
4.84 1.26
Conclusion
We present BrainEC-LLMM, first work to fine-tune pre-trained LLMs for estimating brain effective connectivity (EC) from fMRI data. BrainEC-LLM introduces multiscale decomposition mixing module that downsamples and decomposes fMRI time series data to capture short-term and long-term multiscale trends, mixing complex multiscale temporal variations in both bottom-up and top-down manner. These multiscale fMRI sequences are then aligned with natural language embeddings through attention using pre-trained word embeddings. Finally, the multiscale features generated by LLM are input into multiscale reconstruction mixing module to integrate the information across different scales. The brain EC network is then estimated using self attention. Our comprehensive empirical study demonstrates the effectiveness of BrainEC-LLM. One potential limitation that warrants further investigation is that existing LLMs primarily trained on textual data, whereas fMRI time series data are numerical. LLMs with enhanced numerical reasoning capabilities may yield better results in this context.
7 Acknowledgements
The authors thank anonymous reviewers for their valuable comments and helpful suggestions. This work was sponsored by Beijing Nova Program (20240484635), supported by National Natural Science Foundation of China (62106009, 62276010), and R&D Program of Beijing Municipal Education Commission (KMM002210000000, KZ202210005009,
10

---

References
[1] Daniel Aloalá-López, Jonathan Smallwood, Elizabeth Jefferies, Frank Van Overwalle, Kai Vogeley, Rogier B Mars, Bruce I Turetsky, Angela R Laird, Peter T Fox, Simon B Eickhoff, et al. Computing the social brain connectome across systems and states. Cerebral cortex, 28(7): 2207-2232. 2018.
[2] Neena Aloysius and M Geetha. A review deep convolutional neural networks. In 2017 international conference on communication and signal processing (ICCSP), pages 0588-0592. IEEE, 2017.
[3] Galia Avidan, Michal Tanzer, Fadila Hadj-Bouziane, Ning Liu, Leslie G Ungerleider, and Marlene Behrmann. Selective dissociation between core and extended regions of the face processing network congenital prosopagnosia. Cerebral cortex, 2466):1 2014.
[4] Sara Bagherzadeh, Mohsen Sadat Shahabi, and Ahmad Shalbaf. Detection of schizophrenia using hybrid of deep learning and brain effective connectivity image from electroencephalogram signal. Computers in Biology and Medicine, 146:105570, 2022.
[5] Taiyu Ban, Lyuzhou Chen, Derui Lyu, Xiangyu Wang, and Huanhuan Chen. Causal structure learning supervised by large language model. arXiv preprint arXiv:2311.1166 2023.
[6] Kamyar Barakati, Aleksander Molak, Chris Nelson, Xiaohang Zhang, Ichiro Takeuchi, and Sergei V Kalinin. Causal discovery from data assisted by large language models. Applied Physics Letlers, 127(12), 2025.
[7] Bastos. Dem for complex-value data: cross-spectra, coherence and phase-delays. Neuroim- age, 59(1):439 2012.
[8] Nitin Kumar Chauhan and Krishna Singh. A review on conventional machine learning vs deep learning. In 2018 International conference on computing, power and communication technologies (GUCON), pages 347-352. IEEE, 2018.
[9] Dongdong Chen and Lichi Zhang. Fe-stgnn: Spatio-temports graph neural network with functional and effective connectivity fusion for mci diagnosis. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 67-76. Springer, 2023.
[10] Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey Hinton. Big self-supervised models are strong semi-supervised learners. Advances in neural information 2020.
processing systems, 33:22243-22233
[11] Xiaoyu Chen, Changde Du, Che Liu, Yizhe Wang, and Huiguang He. Open- vocabulary auditory neural decoding using fmri mari-prompted 1lm. arXiv preprint arXiv:2405.07844. 2024.
[12] Yuxiao Cheng. Lianglong Li, Tingxiong Xiao, Zongren Li, Jinli Suo, Kunlun He, and Qionghai Dai. Cuts+: High-dimensional causal discovery from irregular time-series. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 11525-11533 2024.
[13] Zhengjia Dai, Chaogan Yan, Zhiqun Wang, Jinhui Wang, Mingrui Xia. Kuncheng Li, and Yong He. Discriminative analysis of early alzheimer's 's disease using modal imaging and multi-level characterization with multi-classine (m3). Neuroimage, 59(3):2187-2195 2012.
[14] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 4171-4186, 2019.
[15] Adora M DSouza, Anas Z Abidin, Lutz Leistritz, and Axel Wismüller. Exploring connectivity with large-scale granger causality on resting-state functional mri. Journal of neuroscience methods, 287:68-77 2017.
[16] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The llama 3 herd of models. arXiv preprint ar arXiv:2407.21788, 2024.

---

[17] Nico Eisenhauer, Matthew A Bowker, James B Grace, and Jeff R Powell. From patterns to causal understanding: structural equation modeling (sem) soil ecology. Pedobiologia, 58 (2-3):65-72, 2015.
[18] Sam Ereira, Sheena Waters, Adeel Razi, and Charles R Marshall. Early detection dementia with default-mode network effective connectivity. Nature Mental Health, 2(7):787-800, 2024.
[19] Ebrahim Feghhi, Nima Hadidi, Bryan Song, Idan A Blank, and Jonathan C Kao. What are large language models mapping in the brain? case against over-reliance on brain scores, arXiv preprint arXiv:2406.01558 2024.
[20] Karl J Friston, Joshua Kahan, Bharat Biswal, and Adeel Razi. A dem for resting state fmni. Neuroimage, 94:396-409 40066-4002222 2014.
[21] Le Gao, Shuang Qiao, Yigeng Zhang, Tao Zhang, Huibin Lu, and Xiaonan Guo. Parsing the heterogeneity of brain structure and function in male children with autism spectrum disorder: multimodal mri study. Brain Imaging and Behavior, pages 1-14, 2025.
[22] Matthieu Gilson, Ruben Moreno Bote, Adrián Ponce-Alvarez, Petra Ritter, and Gustavo Deco. Estimation of directed effective connectivity from fmri functional connectivity hints at asymme- tries of cortical connectome. PLoS computational biology, 1204)e1 2016.
[23] Enrico Glerean, Raj K Pan, Juha Salmi, Rainer Kujala, Juha M Lahnakoski, Ulrika Roine, Lauri Nummenmaa, Sami Leppämäki, Taina Nieminen-vor von Wendt, Pekka Tani, et al. Reorganization of functionally connected brain subnetworks in high-functioning autism. Human brain mapping, 1763):1066-110 1079, 2016.
[24] Nate Gruver, Mare Finzi, Shikai Qiu, and Andrew Wilson. Large language models are zero-shot time series forecasters. Advances in Neural Information Processing Systems, 36: 19622-19635. 2023.
[25] Youssef Hmamouche, Ismail Chihab, Lahoucine Kdouri, and Amal El Fallah Seghrouchni. A multimodal IIm for the non-invasive decoding of spoken text from brain recordings. arXiv preprint arXiv:2409.19710 2024.
[26] Amelia Rizky Idhartono, Nurul Hidayati, Alsyah Safarina Subekti, and Margareta Vernanda Moi. -strategi pembelajaran di tingkat sma untuk siswa dengan autism spectrum disorder (asd). Kanigara, 1399 2024.
[27] Takashi Itahashi, Takashi Yamada, Hiromi Watanabe, Motoaki Nakamura, Haruhisa Ohta, Chicko Kanai, Akira Iwanami, Nobumasa Kato, and Ryu-ichiro Hashimoto. Alterations of local spontaneous brain activity and connectivity in adults with figh-functioning autism spectrum disorder. Molecular autism, 6:1-14, 2015.
[28) Takashi Itahashi, Takashi Yamada, Hiromi Watanabe, Motoaki Nakamura, Haruhisa Ohta, Chicko Kanai, Akira Iwanami, Nobumasa Kato, and Ryu-ichiro Hashimoto. Alterations of local spontaneous brain activity and connectivity in adults with high-functioning autism spectrum disorder. Molecular autism, 6:14, 2015.
[29] Shruti Japee, Kelsey Holiday, Maureen D Satyshur, Ikuko Mukai, and Leslie G Ungerleider. A role of right middle frontal gyrus in reorienting of attention: a case study. Frontiers systems neuroscience, 9:23, 2015.
[30] Junzhong Ji, Zuozhen Zhang, Lu Han, and Jinduo Liu. Metacae: Causal autoencoder with meta-knowledge transfer for brain effective connectivity estimation. Computers in Biology and Medicine, page 107940, 2024.
[31] Yanchun Jiang, Yanbo Zhang, Liluo Nie, Huihua Liu, and Jinou Zheng. Identification and effective connections of core networks in patients with temporal lobe epilepsy and cognitive impairment: Granger causality analysis and multivariate pattern analysis. International Journal of Neuroscience, 133(9):935-90) 2023.
12

---

[32] Ming Jin, Yifan Zhang, Wei Chen, Kexin Zhang, Yuxuan Liang, Bin Yang, Jindong Wang, Shirui Pan, and Qingsong Wen. Position: What can large language models tell us about time series analysis. In Forty-first International Conference on Machine Learning, 2024.
[33] Katikapalli Subramanyam Kalyan. A survey of gpt-3 family large language models including chatgpt and gpt-4. Natural Language Processing Journal, page 100048, 2023.
[34] Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In 10th International Conference on Learning Representations, ICLR 2022, 2022.
[35] Tracey A Knaus, Claire Burns, Jodi Kamps, and Anne L Foundas. Atypical activation of action-semantic network in adolescents with autism spectrum disorder. Brain and cognition, 117:57-64, 2017.
[36] Hongming Li, Shujian Yu, and Jose Principe. Causal recurrent variational autoencoder for medical time series generation. In Proceedings of the AAAI conference artificial intelligence, volume 37, pages 8562-8570, 2023.
[37] Lei Li, Changchun He, Taorong Jian, Xiaonan Guo, Jinming Xiao, Ya Li, Heng Chen, Xiaodong Kang, Huafu Chen, and Xujun Duan. Attenuated link between the medial prefrontal cortex and the amygdala in children with autism spectrum disorder: Evidence from effective connectivity within the "social brain". Progress in Neuro-Psychopp Suurro-sycchpharmccooog and Biological Psychiatry, 111:110147,8 2021.
[38] Di Liang. Shengxiang Xia, Xianfu Zhang, and Weiwei Zhang. Analysis of brain functional connectivity neural circuits in children with autism based on persistent homology. Frontiers in Human Neuroscience, 15:745671, 2021.
[39] Jinduo Liu, Junzhong Ji, Guangxu Xun, Liuyi Yao, Mengdi Huai, and Aidong Zhang. Ec-gan: inferring brain effective connectivity via generative adversarial networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 4852 4859, 2020.
[40] Jinduo Liu, Junzhong Ji, Guangxu Xun, and Aidong Zhang. Inferring effective connectivity networks from fmri time series with a temporal entropy-score. IEEE Transactions on Neural Networks and Learning Systems, 33(10):5993-600 2022.
[41] Jinduo Liu, Lu Han, and Junzhong Ji. Mcan: multimodal causal adversarial networks for dynamic effective connectivity learning from fmri and eeg data. IEEE Transactions on Medical Imaging, (3(8):2913-2922) 2024.
[42] Yilin Lu, Jinduo Liu, Junzhong Ji, Han Lv, and Mengdi Huai. Brain effective connectivity learn ing with deep reinforcement learning. In 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 1664-1661 IEEE, 2022.
[43] Donghao Luo and Xue Wang. Modernten: modern pure convolution structure for general time series analysis. In The Twelfth International Conference on Learning Representations, 2024.
[44] Batta Mahesh. Machine learning algorithms-a review. International Journal of Science and Research (Inteeent,)),e)), 91):3311-388 2020.
[45] Falko Mecklenbrauck, Marius Gruber, Sophie Siestrup, Anoushiravan Zahedi, Dominik Grote- gerd, Marco Mauritz, Ima Trempler, Udo Dannlowski, and Ricarda I Schubotz. The significance of structural rich club hubs for the processing of hierarchical stimuli. Human Brain Mapping. 45(4):c26543, 2024.
[46] MC Mozer. Induction of multiscale temporal structure. In Neural Information Processing Systems, volume 4, pages 275-282. Morgan Kaufmann, 1992.
[47] Izzy Newsham, Luka Kovašević, Richard Moulange, Nan Rosemary Ke, and Sach Mukherjee. Large language models for zero-shot inference of causal structures biology. In ICLR 2025 Workshop on Machine Learning for Genomics Explorations, 2025.
13

---

[48] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Reprexentations, 2023.
[49] Alessandra M Pereira, Brunno M Campos, Ana C Coan, Luiz F Pegoraro, Thiago JR De Rezende, Ignacio Obeso, Paulo Dalgalarrondo, Jaderson C Da Costa, Jean-Claude Dreher, and Fernando Cendes. Differences in cortical structure and functional mri connectivity in high functioning autism. Frontiers in neurology, 9:539, 2018.
[50] David C Plaut and Marlene Behrmann. Response to susilo and duchaine: beyond neuropsy- chological dissociations in understanding face and word representations. Trends in cognitive sciences, 17(11):546, 2013.
[51] Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang. and Tim Januschowski. Deep state space models for time series forecasting, Advances in neural information processing systems, 31, 2018.
[52] Dipanjan Ray, Dmitry Bezmaternykh, Mikhail Mel'ni Karl J Friston, and Moumita Das. Altered effective connectivity in sensorimotor cortices is signature of severity and clinical course in depression. Proceedings of the National Academy of Sciences, 1 118(40):22105730118, 2021.
[53] Siti Hajar Mohd Roffeei, Noorhidawati Abdullah, and Siti Khairatul Razifah Basar. Seeking social support on facebook for children with autism spectrum disorders (asds). International journal of medical informatics, 84(5):375-385, 2015.
[54] Ruben Sanchez-Romeng Joseph D Ramsey, Kun Zhang. Madelyn RK Glymour, Biwei Huang, and Clark Glymour. Estimating feedforward and feedback effective connections from fmri time series: Assessments of statistical methods. Network Neuroscience, 3(2):274-306, 2019.
[55] Letten Saugstad. Infantile autism: a chronic psychosis since infancy due to synaptic pruning of the supplementary motor area. Nutrition and health, 20(3-2):111-122, 2011.
[56] Preya Shah, Danielle S Bassett, Laura EM Wisse, John A Detre. Joel M Stein, Paul A Yushkevich, Russell T Shinohara, John B Pluta, Elijah Valenciano, Molly Daffner, et al. Mapping the structural and functional network architecture of the medial temporal lobe using 7t mri. Human Brain Mapping, 39(2):851-865 2018
[57] Pramila P Shinde and Seema Shah. A review of machine learning and deep learning applications. In 2018 Fourth international conference on computing communication control and automation (ICCUBEA), pages 1-6. IEEE, 2018
[58] Stephen M Smith, Karla L Miller, Gholamreza Salimi-Khorshidi, Matthew Webster, Christian F Beckmann, Thomas E Nichols, Joseph D Ramsey, and Mark W Woolrich. Network modelling methods for fmri. Neuroimage, 54(2):875-891, 2011.
[59] Yuni Susanti and Michael Färber. Can Ilms leverage observational data? towards data-driven causal discovery with 1lms. arXiv preprint arXiv:2504. 10936, 2025.
[60] Mingtian Tan, Mike Merrill, Vinayak Gupta, Tim Althoff, and Tom Hartvigsen. Are language models actually useful for time series forecasting? Advances in Neural Information Processing Systems, 37:60122-60111 2024.
[61] Mohammad Mustafa Taye. Understanding of machine learning with deep learning: architectures, workflow, applications and future directions. Computers, 12(5):91, 2023.
[62] Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorek, Zhiliang Peng, Qiang Liu, Kriti Aggar- wal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, and Furu Wei. Image as a foreign language: BEÎT' pretraining for vision and vision-language tasks. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, 2023.
[63] Yonglu Wang, Lingxi Xu, Hui Fang, Fei Wang, Tianshu Gao, Qingyao Zhu, Gongkai Jiao, and Xiaoyan Ke. Social brain network of children with autism spectrum disorder: characterization of functional connectivity and potential association with stereotyped behavior. Brain Sciences, 13(2):280, 2023.
14

---

[64] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 34:22419-22430, 2021.
[65] Wen Xiong, Jinduo Liu, Junzhong Ji, and Fenglong Ma. Brain effective connectivity estimation via fourier spatiotemporal attention. In Proceedings of the 3 ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. I, pages 1657-1668, 2025.
[66] Jinping Xu, Chao Wang. Ziyun Xu, Tian Li, Fangfang Chen, Kai Chen, Jingjing Gao, Jiaojian Wang, and Qingmao Hu. Specific functional connectivity patterns of middle temporal gyrus subregions in children and adults with autism spectrum disorder. Autism Research, 13(3):
410-422, 2020.
[67] Shilin Xu, Xin Wang, Linling Shen, Xiaohui Yan, Guoyan Feng, and Fan Cao. Brain functional differences during irony comprehension adolescents with asd. Cerebral Cortex, 35(2): bhaf003, 2025.
[68] Ziyi Ye, Qingyao Ai, Yiqun Liu, Maarten de Rijke, Min Zhang, Christina Lioma, and Tuukka Ruotsalo. Generative language reconstruction from brain recordings. Communications Biology, 8(1):346, 2025.
[69] Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. A survey on multimodal large language models. National Science Review, (12)),, November 2024. ISSN 2053-7143 714X. doi: 10.10933/ss/wwae4003 URL http: //tp://xx.uu doi wwg.:.........10000 1093/ nsr nwwe
[70] Minqi Yu, Jinduo Liu, and Junzhong Ji. Causal invariance-aware augmentation for brain graph contrastive learning. In Forty-second International Conference on Machine Learning.
[71] Yaxu Yu, Xiaoqin Wang. Junyi Yang, and Jiang Qiu. The role of the mig in negative emotional processing in young adults with autistic-like traits: fmri task study. Journal of Affective Disorders, 276:890-897, 897, 2020.
[72] Khadija Zanna and Akane Sano. Fairnes-ddriven Iamed causal discovery with active learning and dynamic scoring. arXiv preprint 25035:2503.17569, 2025.
[73] Xinyu Zhang, Shanshan Feng, and Xutao Li. From text to time? rethinking the effectiveness of the large language model for time series forecasting. arXiv preprint 2504.02200.8881 2025.
[74] Zhengxin Zhang, Dan Zhao, Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Qing Li, Yong Jiang, and Zhihao Jia. Quantized side tuning: Fast and memory-efficient tuning of quantized large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1-17, 2024.
[75] Zuozhen Zhang, Junzhong Ji, and Jinduo Liu. Metarlee: Meta-reinforcement learning for discovery of brain effective connectivity. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 10261-10269, 2024.
[76] Xiaoxin Zhao, Shuyi Zhu, Yang Cao, Peipei Cheng, Yuxiong Lin, Zhixin Sun, Wenqing Jiang, and Yasong Du. Abnormalities of gray matter volume and its correlation with clinical symptoms in adolescents with sigh-functionin autism spectrum disorder. Neuropsychiatric Disease and Treatment, 18:717, 2022.
[77] Xun Zheng, Bryon Aragam, Pradeep K Ravikumar, and Eric P Xing. Dags with no tears: Continuous optimization for structure learning. Advances in neural information processing systems, 31, 2018
[78] Xiaofan Zhou, Liangjie Huang, Pinyang Cheng, Wenpen Yin, Rui Zhang, Wenrui Hao, and Lu Cheng. Accelerating causal network discovery of alzheimer disease biomarkers via scientific literature-based retrieval augmented generation. arXiv preprint arXiv:2504.0876. 2025.
[79] Shengyu Zhu, Ignavier Ng, and Zhitang Chen. Causal discovery with reinforcement learning.
In International Conference on Learning Representations, 2020.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Refer to the sentence "Considering the unique nature of... the first method to fine-tune LLMs for estimating brain EC from IMRI data" of the abstract and the last paragraph of section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Refer to Appendix
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well- sell-specification asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken low lighting. Or speech-to-texx system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't secnowleddger in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
16

---

Answer: [Yes]
Justification: Refer to Appendix
B
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Ys]]
Justification: Refer to "Implementation Details" in section 5.1 Appendix
C and Appendix
D.7
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
17

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
Refer to the last sentence of abstract, Appendix D.I and Appendix
D.2
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr cc/
public/
ssnniisssiinn soonisssscyypyp for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https?
des /CoesSoodSbbba bm ss i oonool icy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: Refer to "Post Process and Evaluation" and "Implementation Details" section 5.1 Appendix D.7
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Ye]]
Justification: Refer to Table I and Appendix
E.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
18

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are of range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Refer to "Implementation Details" section
5.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httpp::://aaarrr......ooolll///p
Answer: [Yes]
Justification: The datasets used in the paper are publicly available datasets, refer to "Simu-
lated IMRI Dataset" and "Real Resting-state IMRI Dataset" in section 5.1
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There are no privacy or security concerns and therefore no negative societal
impacts, refer to first paragraph of section [ for positive societal impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
19

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e-g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
We have properly cited all LLM-related papers whose models, code, or methods are used or referenced in our work, which can be found in section All datasets, pre-trained models, and libraries used in our experiments are publicly available and used in accordance with their respective licenses.
Justification:
The source of the data can be found at section 5.1
Guidelines:
The answer NA means that the paper does
existing assets. The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
20

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, haperssithcoo com/ catasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/codeemmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
21

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: Our work proposes the first method fine-tune LLMs for estimating brain effective connectivity from fMRI data in causal discovery field.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy httt::://eeuiips.cc//oofeee LLM) for what should or should not be described.
22

---

A Examples of Prompts
Our prompt design is intended to provide the LLM with structured representation of inter-regional brain relationships by explicitly encoding the Pearson correlation matrix, where each row and column corresponds to a specific brain region. To preserve consistent and interpretable mapping between brain regions and their associated fMRI ROI time series, numerical indexing is incorporated into the prompt along with corresponding explanations, ensuring the model can clearly associate each index with specific brain region. Examples of prompts are illustrated in Figure G which clearly illustrates how the LLM associates brain regions in the Pearson correlation matrix with those in the fMRI data. The purple, green, and pink prompts correspond to task description, dataset description, and prior knowledge, as depicted in Figure respectively.
"systt"""sste"' " "cortent" ("Yte are data analysis expert. MEE 1s recroinaging techntiqu this messures arata actiidiy by tracking changes In cesgrsstion.
ner the geites direct's pessrated, while 1922 date condists courere.
("rote": ''ayste", "content": "oa are data yni
suries that how praprccesoed scturra
oscert. Your losk is to help users andlyee IMRI Lire series dsta extract besin effective we conectivityy by rebwrk. ",
thees progrosssedd hel certas actiante trats effection
["rote Nheer" "conient"; Datasel descriptton; This throug
correctiaity (85), wuich returs the directal irrlunnce bebweey brain reglore, castaring
jutt
nodels, with dieenstons the batch stme, is the data points, and N the nuber of broin regions.
cotaset generated through oseance caosal eodils, vite soressions 8 stere 11 th batch stze, da the data points, and it the narban of bradn reglers.
The tollnggng 16 soters of Peomson concelation
I9et time sernes, with the number
donosing the brain regiinn"
regions
Wrior 0.00, 0.31,
"yrtor kooulndger
0.14,
0.06,
12 12.0 4.31, 4.16, Rol, 8.10
8.53, 0.05, 0.35, 6.12, 6.35
21 8.23 8.00 0.25, 0.12, 8:22
8.26 0.35, 0.00
0252, 0.27
0-34 4.35, 8,40, 8-32, 6:12
8.06 0.12, 0.37,
08, 53
5: 0.38,
0.15,
0.17,
0.33, 0.00")
B.Re")
(a) Prompt example of Smith datasct.
(b) Complex prompt example of Smith dataset.
Figure 6: Prompt example of Smith dataset.
B Proof of Theorem 4.1
For positive samples Z21.z they are drawn from the joint distribution Pi (z p(z), Z.41). In contrast, the negative samples (Z) # i,j # i + 1 are drawn from the marginal distribution pz(z), 23)) - p(Z)) Given a of samples, the probability of correctly identifying a positive sample is
p(1z), 2
1,., MF)
(Z] 1/+1) II [1iti,((+1)))e(l 2z)
Exsr Pi (Z) z II P2 (z))
p(Z", Z3 11 11+,1 pl (Z)))(zz)
Enpr Z,; 2% II j p(Z))p(Z))
pizz 2/:1n) 11 11idh,((+1)) P(Z p(Z))((Z))
(12)
Extt 1lZz z/.) 11 p(()))
p(Zi, 2 (+ P p(Z)))(Z)
T pl (2)))))())))))))))))) p (2)) Σ Exerr 2(Z)
p(2))
rlz Pimil
Σ Eddf p.Z?)p(Z')
28)
23

---

By comparing Eq.
12 and the definition of Ci++1
esc we find that the optimal function g (Z) 21+1)
P 4. p pZ
p 2.
pizi
proportional
Given that
conclude:
pl
plzz)
g (Z) (+1)
(13)
p(Z))
into the optimal loss objective Leee+ 1),opt to obtain
Now, we substitute Eq.
(t.(++))opt use
1
M-1
9 gopt (Z) Zi+1) Espigam (Z:; Z.j)
log
2B(M
got (Z,; Zi+z1)
log z
Σ j gopd (Zi Z2})
11211 p(2))
=-E log Z
p(Z;
Σ jtt m(?))
p(Z%)
p(Z)12))
log Z
Σ
p(zi
p(Z))
jf1((++1)
p(Z!)
(M - 2) (((2)1z))
E log
(14)
pZZ
m(Zz)
v(Z))
E log Z
p(Z ( (M -2)
p(Z%)
> E log
p(Z (M -1)
pzzz, 21+1)
E [log(A (M -1]]
JE log
p(Z)) (2)
log(M - 1))
p(Z z
E (2) 1 /ppp7777
log
p(Z (1(2)+1))
log(M 1) I (ZZ 2b+1)
Therefore, C( (pppt)).opt
2 log(M - 1) 1(z,, 2.41). Similarly, for other case, we also obtain
c(i+1,))op 2 log(M - 1) - 1(z Z Combining the above two equations with Copt c((
1(.t+)).opt l).opt
+ (1+1,,)),0 we have
Copt log(M 1) (1(2), 2iz1) 1(241122))
(15)
C Algorithm Description
The BrainEC-LLM algorithm comprises three main components; prompts generation (PG), multiscale decomposition mixing (MDM), and multiscale reconstruction mixing (MRM) modules. The detailed description of the algorithm can be found in Algorithm Initially, BrainEC-LLLM employs PG to produce relevant prior prompts. Next, MDM downsamples and decomposes fMRI time series data to obtain short-term and long-term multiscale trends, which mix the multiscale information in
24

---

Algorithm BrrainEC-LMM
Input: IMRI time series data
Parameters of prompts generation, multiscale decomposition mixing and multiscale reconstruction mixing modules: pg. V mdm: mrr: the training epochs E
Parameter:
Output: Brain effective connectivity network A
1: for epoch = 1 to E do
2: Prompts Generation: Generate task description, dataset description and prior knowledge
prompts to bootstrap LLM;
3: Multiscale Decomposition Mixing: Downsampling and decomposition of fMRI time series data are performed as described in (2) and Eq. (3);
4: Fine and coarse multiscale sequences are blended using bottom-up up and top-down approaches, respectively, as Eq. (4) and Eq. (5);
5: Perform cross attention on multiscale fMRI time series and pre-trained word embeddings;
6: Using prompts as prefixes and multiscale fMRI embeddings as input to the LLM,, fine-tune with LoRA;
7: Multiscale Reconstruction Mixing: Mixing multiscale IMRI time series and extracting brain
effective connectivity networks with self attention;
8: end for
9: Post-process;
10: return Brain effective connectivity A
bottom-up and top-down manner, respectively. These multiscale fMRI sequences then aligned with natural language expressions by performing cross attention with pre-trained word embeddings. The prompts are used as prefixes and concatenated with the multiscale fMRI embeddings, which then input into the LLM and fine-tuned using LoRA. Finally, the prefix part of the LLM output is discarded, and the reconstructed fMRI time series and brain effective connectivity network A are produced by inputting the data into self attention, fusing the information from multiple scales through the MRM.
D Experimental Settings
D.1 Simulated fMRI Dataset
The benchmark simulated datasets we use are Smith datase and Sanchez datase both generated by dynamic causal model. Specifically, Sanchez dataset offers higher temporal resolution and acquisition frequency compared to Smith dataset and minimally affects the non-Gaussianity the BOLD signal. Each session for both datasets lasted 10 minutes. Additionally, Smith dataset has repetition time (TR) of 3.00 seconds, while Sanchez dataset has TR 1.20 seconds, reflecting current acquisition protocols with higher temporal resolution. For Smith dataset, select Simulation 1 due to the fact that Simulation 1 is widely used as baseline for comparison with other datasets. In Sanchez dataset, we chose Simulation 2 because it features simple structure with two closed loops sharing a single node, This configuration is common in real brain networks and helps validate the model's performance in handling highly connected regions, known hubs 455
Fand
To further validate the model's accuracy, we generate simulated dataset using CDRL 779 Consid- ering that real resting-state fMRI dataset
includes 7 ROIs and features cycles. the simulated dataset contains 5 and 7 nodes, named simulated complex network with node 5 (SCN5) and simu- lated complex network with node 7 (SCN7), respectively. The generated simulated datasets exhibit quadratic nonlinear causality, and the corresponding EC network contains cycles and more complex relationships. Specifically, the ground truth in SCN5 contains nodes, while the one in SCN7 contains 7 nodes. Each dataset includes 50 samples. Details regarding the individual simulated datasets are presented in Table 6
"happ://gituub.ccm/ onmeecaacmmcemcccceeeettiiioevey huuprr/gtttaccoiaae.iaaaaoaaaaae
25

---

Table 6: Description of the benchmark simulation dataset.
Dataset
Subjects
Data Points
Nodes
Ares
Cycles
Smith
50
200
5
5
0
Sanchez
60
500
5
7
2
SCN5
50
200
5
8
4
SCN7
50
200
7
15
10
D.2 Real Resting-state fMRI Dataset
To assess the algorithm under real BOLD data conditions, we utilize high-resolution 7T human resting- state IMRI date from the medial temporal lobe. The real resting-state fMRI dataset was band-pass filter in the range of 0.008 to 0.08 Hz without spatial smoothing to prevent signal aliasing between neighboring regions. The resulting dataset comprises 23 healthy adults, each with an acquisition time of 1.0 seconds and a session duration of minutes per subject, yielding fMRI time series of 421 data points. We focus on seven regions of interest within each hemisphere of the medial temporal lobe: perirhinal cortex, divided into Brodmann areas 35 and 36 (BA35 and BA36); parahippocampal cortex (PHC); entorhinal cortex (ERC); subiculum (SUB); cornu ammonis (CAI); and region comprising CA2, CA3 and dentate gyrus together (CA23DG). Averaging the signals from CA2, CA3, and CA23DG into single regional signal helps mitigate potential issues in connectivity estimation caused by signal mixing between adjacent regions (Smith et al., 2011), 1), a challenge particularly pronounced regions that are difficult segment, such as CA2, CA3, and the dentate gyrus.
D.3
Post Process
For the obtained brain EC matrix A, a threshold 0 and maximum number of parent nodes Max Pa is required to convert A into a binary matrix. Given that different fMRI datasets may exhibit varying signal characteristies and noise levels, distinct thresholds need to be applied. Therefore, set the threshold 0 to be adaptive, and is calculated as follows:
= (| + x (mar([A)) min min()A))),
(16)
where 17 is a hyperparameter used to adjust thresholds. The maximum parent node proportion is established for the brain EC matrix, with the maximum number of parent nodes Maraa being determined by multiplying the number of brain regions N with the maximum parent node proportion MaxPa. For values exceeding the threshold in each row and column, the top M MarPa value is to 1, while all other values are set to O.
D.4 Evaluation Metrics
In order to assess the effectiveness of the mothods, we use the following evaluation metrics: Precision, Recall, F1, Accuracy and Structural Hamming distance (SHD). Among them, Recall and Precision are commonly used metrics in brain effective connectivity (EC) learning and other learning tasks. Fl is a harmonic mean that combines Precision and Recall, providing balanced assessment of both metrics. Accuracy refers to the proportion of samples in the prediction results that are correctly predicted. SHD represents the difference between the learned brain EC and ground-trui brain EC. In general, the higher the Precision, Recall, F1, Accuracy, and the lower the SHD, the better the performance of the method. The Precision, Recall, F1, Accuracy and SHD be calculated follows:
TP Precision
(17)
TP + FP
ТР
Recall TP + FN
(18)
2 X Precision X Recall
=
(19)
Precision + Recall
26

---

TP + +NN TP FP TN + FN
Accuracy
(20)
SHD - SHD=EA+ MA+ RA
(21)
where TP, FP, TN, and FN denote true positive, false positive, true negative, and false negative, respectively. EA, MA, RA signify extra arcs, missing arcs and reverse arcs, respectively.
D.5 Time Complexity Analysis
The model's time complexity is as follows:
Prompt generation module: O(N² T), where N and T denote the number of brain regions and the number of data points, respectively.
Multiscale decomposition mixing module: the time complexity of downsampling is M x ON x T), where M denotes the number of scales; the time complexity of both bottom- up and top-down mixing is M x O(T2), and the time complexity of cross-attention is M x O (dum x emodet)) ). where dum denotes the vector size of tokens the LLM embedding space and dmmodel represents the size of the model embedding space vector.
Multiscale reconstruction mixing: the time complexity of the linear layer is O(T2) and the time complexity of self attention is O(N2 x T)).
Therefore, the time complexity of the whole model is O(N T) TX M x Oddtm x &mmodee) + T + ulm --llmiinett iinetunin + dim x d d.modet). whereas LLM is mainly made by stacking the encoder or decoder of the transformer, the number of parameters is huge and the time complexity is also very high, so it can be seen that the time complexity of our proposed module is insignificant compared to the time complexity of LLM. We conducted experiments with and without LLM single Nvidia 1.20-48GB GPU, and their efficiency results are presented in Table 7 The results indicate that the method's time complexity is primarily driven by the LLM's time complexity.
+ M x O(T2) OON2 x T) M x O xT
Table 7: Efficiency analysis on Smith dataset.
Methods
w/o LLM
Default
Mem.
3584
24291
Speed(s/epoch)
1,563
89.324
D.6 Training Methods
Given the limited availability of fMRI data, segmentation would further reduce the dataset size and potentially compromise the reliability of results. Moreover, since EC lacks ground truth on real fMRI datasets, we opt not to segment the dataset but instead directly train model on the available fMRI data in an autoregressive manner, obtaining both the reconstructed fMRI data and brain ECC This approach leverages the inherent autocorrelation (i.c., the brain effective connectivity network) within the !MRI time series to predict the IMRI data itself, enabling the model to learn more representative and informative connectivity structures. As result, the model is trained by minimizing the difference between the predicted and actual fMRI signals. The EC network is generated as the outcome of this optimization process, and evaluation metrics are directly computed based on the predicted EC network without the need for a separate test set.
D.7 Hyperparameter Settings of BrainEC-LLM
The key parameter settings for BrainEC-LLM across different datasets are presented in Table g The Ilm layers differ due to the limitations of the machine's video memory capacity; they are set to the maximum number of layers that the system can support in that scenario. Although Sanchez dataset includes cycles, the relationships between the nodes are relatively simple. By incorporating acyclic constraints Ldags the complexity of BrainEC-LLLM can be reduced, which helps to minimize overfitting and improve the model's generalization ability.
27

---

The choice of 0.5 represents a balanced trade-off between generating overly sparse or dense effective connectivity networks, which we found work robustly across multiple fMRI datasets in our experiments. While 7 can indeed be tuned for specific applications, we adopted this universal value. And ensure fair comparison, we maintain consistent evaluation criteria across all methods by applying the same threshold parameter (1) 0.5) to convert all estimated EC matrices into binary networks.
Table 8: Hyperparameters settings of BrainEC-LLM in the aforementioned experiments.
Hyperparameters
Smith
Sanchez
SCNS
SCN7
Ilm layers
24
18
24
12
Oldag batch size
100
100
0
2
2
2
2
?7
0.5
0.5
0.5
0.5
dropout
0.01
0.01
0.01
0.01
model
32
32
32
32
D.8 Baseline Methods Parameter Setting
Eight baseline methods harnessed for comparison with the proposed method, which can be catego- rized into two groups: machine learning methods and deep learning methods. The baseline methods
follows: spDCM [20). IsGC [15], ACOCTE [40) RL-EC [42, CR- VAE [36], MetaCAE (30),
MetaRLEC [75,, CUTS+ 12 The parameter settings for these baseline methods are shown in
Table 9
Table 9; Parameter settings of seven baseline methods
Methods
Years
Parameters
nonlinear = 0. two state = 0, stochastic = 1,
spDCM
2014
centre = 1, induced = 1, maxit = 10
IsGC
2017
cmp 5, ARorder
normalize
ACOCTE
2022
a = 1.0, = 2.0, go = 0.98, 0.2
RL-EC
2022
nh 256, heads 16, stacks 6, nhdeeader 16
CR-VAE
2023
context = 20, = 0.1, = 0.05,nh = 64
nh 64, 0.05, 8 = 20.0. k = 3, d 4. Irg 0.02,
MetaCAE
2024
Ir2 0.02, Ir3 = 0.001, Irmain = 0.002
headSeneoder 8, blockSencoyler 3, droporrt = 0.1,
MetaRLEC
2024
Factor = 10 -reritic = 10
CUTS+
2024
layers gru = 1, Irstagel = 10-3 3 + 10-4 Irstage2 = 10-2 + 10-3
E Experiments
E.1 i-test Analysis
To assess the significant differences between BrainEC-LLM and other baseline methods, t-test analysis and Wilcoxon test with Holm correction are performed, with results presented in Table 10 and [T] p-values less than 0.05 indicates a significant difference the 95% confidence level. The analysis in f-test reveals that, except for RL-EC, which shows no significant difference in the evaluation metrie SHD, BrainEC-LLM is significantly different from the baseline methods in all other cases. The Wilcoxon test results are consistent with those of the (-test, confirming the absence of a significant difference in SHD between BrainEC-LLMM and RL-EC. In addition, the Wilcoxon test indicates significant difference in recall between PrainEC-LLM and CR-V VAE. However,
28

---

significant differences are observed in the other evaluation metrics, further demonstrating the superior performance of our method.
Table 10: p-values obtained from the t-test for BrainEC-LLLM and seven baseline methods. Underlined values indicate no significant difference at the 95% confidence level.
Methods
Precision
Recall
F1
Accuracy
SHD
spDCM
1.30
10
2.34 x 10 3
2.87 X 10-5 5
1.63 x 10
4.23 x 10 2
IsGC
2.48
10 10-5 5
4.85 10-3
1.27
10-5
6.88 10
2.75
10
ACOCTE
1.63
10
2.80 x 10-4
4.61 x 10-6
1.51 X 10-3
4.45 x 10
RL-EC
9.07
10
3.95 x 10-6
4.58 x 10
3.17 x 10
6.75 x 10
CR-VAE
1.62 10-3
1.34 x 10-3
5.16 x 10-6
9.52 x 10-1
4.17 x 10
MetaCAE
3.47 10-4
2.93 x 10-3
1.88 x 10-6
6.18 x 10-4
3.47 X 10-2
MetaRLEC
3.74 10-4
x 1.16x10" 10-3
8.19 x 10-7
4.77 x 10
3.44 X 10
CUTS+
1.40 10-4
1.46
10-3
8.85 X 10
2.84 10
3.07 10
Table v-vuue obtained from the Wilcoxon for BrainEC-LLM and seven baseline methods. Underlined values indicate no significant difference at the 95% confidence level.
Methods
Precision
Recall
F1
Accuracy
SHD
spDCM
1.61
10
6.84 x 10 10-3 3
4.88 x 10
2.44 x 10
4.88 x 10
IsGC
4.88
10
1.47
10 3
4.88 X 10
9.77 X 10
4.88 x 10
ACOCTE
3.13
10 10-2
4.88 x 10-4
9.77 x 10
1.61 X 10
6.84 x 10
RL-EC
1,20
10 10-2
4.88 x 10-4
3.42 x 10-3
5.62 X 10
3.42 x 10
CR-VAE
9.77
10
8.98 10-1
1.46 x 10-3
9.77 x 10
9.77 x 10-4
MetaCAE
3.42
10
4.65 x 10-3
3.42 x 10
1.95 x 10
3.42 x 10
MetaRLEC
4.88
10
4.88 x 10
4.88 x 10
4.88 x 10
4.88 x 10
CUTS+
4.88
10-1
3.42
10-3
4.88 x 10
4.88 x 10-1
4.88
10-1
E.2
Visualization of LLM Inputs
Figure 7 illustrates the evolution fMRI embeddings during training, demonstrating how BrainECLLM learns meaningful representations. Since prompts generation module is frozen, prompt tokens remain fixed throughout training, allowing us to focus solely on visualizing the fMRI multi- scale embeddings in LLM input. As shown in Figure 7 the early stages of training (e.g.. epoch 50), BrainEC-LLM has not yet fully learned, resulting in predominantly negative feature values. As training progresses and the number of epochs increases, the model gradually converges, and the feature representations shift towards more meaningful directions.
E.3 ABIDE Visualization
comparative analysis of brain effective connectivity (EC) networks in healthy controls (HCs) versus individuals with autism spectrum disorder (ASD), based on AAL-90 atlas (90 regions) 13 To analyze the most critical effective connectivity between brain regions, preserved top 5% of edges with the highest scores for subsequent analysis and visualization. The six figures illustrate the binarized brain EC networks, derived by averaging the original EC networks of healthy controls and ASD patients from the ABIDE I dataset, respectively. The nodes represent brain regions, while the edges indicate the effective connectivity. The node size reflects the centrality or importance of each region within the network. The other six heatmaps display three types of brain EC networks for both healthy controls and ASD patients respectively: the original brain EC networks, the top 5% of the strongest connections in the original brain EC networks, and the top 5% of the strongest connections in the binarized brain EC networks.
Figure illustrates
Previous literature has consistently reported that individuals with Autism Spectrum Disorder (ASD) exhibit increased connectivity in the right middle temporal gyrus (MTG) 71]. decreased connectivity the temporal pole (TPO) 23 and supplementary motor area (SMA) 63 and asymmetric connec-
tivity alterations in the middle frontal gyrus (MFG), with reduced connectivity in the left hemisphere
29

---

Epoch
Epoch 5D
Tokens 480 BU7301
500
1000
8 SEEES
Epoch 100
Epoch 150
1000
1000
0
-2000
2000
Feature dimension
Figure 7: Visualization of IMRI multiscale embedding in LLM inputs.
and increased connectivity in the right hemisphere 23 28). The connectivity patterns observed in the figure align with these previous findings, reinforcing the notion that these regions play critical roles in the neural underpinnings of ASD.
Middle Temporal Gyrus (MTG). The MTG is fundamentally involved in supporting a range of social and linguistic processes [67]. Evidence from facial recognition study demonstrated that individuals with congenital prosopagnosia showed diminished neural activation and weaker connectivity within the anterior temporal lobe, which highlights the region's 's critical role in facial information processing (3]. Additionally, research has indicated that the recogmition of both faces and words relies on high-resolution visual representations, shaped by the close functional coupling of visual and language-related brain regions and the neural tendency to preserve short inter-regional
pathways 50]
Temporal Pole (TPO). The TPO vital structure within the social brain network, is essential for the theory of mind (ToM), fundamental cognitive skill that enables individuals to comprehend and infer the mental states of others, as well as anticipate their actions [ 666. A substantial body of research has highlighted that disruptions in both the functional and structural integrity of the TPO are likely linked to the social difficulties observed in individuals with ASD 49), who frequently face considerable difficulties when engaging in ToM-related tasks.
Supplementary Motor Area (SMA). The SMA, located on the medial aspect of the frontal lobe anterior to the primary motor cortex, is primarily associated with motor planning and execution. In children with ASD, reduced functional connectivity between the SMA and social-related brain regions has been observed, which may contribute to deficits in social interaction and communication [55]. Compared healthy controls, individuals with ASD exhibit reduced functional connectivity in SMA neural circuits and diminished SMA activation 35 3
Such disruptions are potentially linked to abnormal semantic verb processing, which may hinder their ability to construct coherent discourse and develop adequate language proficiency, ultimately affecting behavioral patterns and the capacity for effective social engagement (26].
Middle Frontal Gyrus (MFG). Numerous neurocognitive and neuroimaging studies have demon- strated that MFG is associated with the pathophysiology of ASD
76 21 MFG integral executive functions, including working memory and attention control 29 Research has identified the MFG as one of the abnormal brain regions within the occipital pole network that associated with social and communication deficits in individuals with ASD [27]. The MFG is primarily involved in coor- dinating and integrating various types of information, playing key role in executive function and cognitive control [29 Prior studies have also reported that individuals with ASD often exhibit im- pairments in information integration and processing, which may contribute to difficulties effective communication, particularly in public socially demanding environments [53].
30

---

10g
(a) Healthy controls.
(b) Healthy controls.
(c) Healthy controls.
2012
2013
2005
X005
Peguoe
(d) Original brain EC network heatmap (healthy controls).
(c) Top 5% original brain EC
(f) Top 5% binarized brain EC
heatmap (healthy controls).
heatmap (healthy controls).
ASD.
(h) ASD.
(i) ASD.
A02
SOE
SOW
Original brain EC network
(k) Top 5% original brain BC
(l) Top 5% binarized brain EC heatmap (ASD).
heatmap (ASD).
heatmap (ASD).
Figure 8: Comparison of brain EC network between healthy controls and ASD patients the ABIDE I dataset Using Bra.
E.4 Ablation Study
We perform ablation studies on model backbone, model module, and the loss function, with the results presented Table [2] In the Table, "Transformer Decoder" indicates that the LLM is replaced with conventional transtormer decoder. "w/o LLM"" refers to completely removing the pre-trained LLM from the BrainEC-LLM framework. Since the cross attention is specifically designed for the LLM to reduce cross-modal discrepancies, it is also removed. "Spearman Correlation Matrix" and "Kendall Correlation Matrix" refer to replacing the Pearson Correlation Matrix the Prompt Generation module with the corresponding correlation matrix.
31

---

Table 12: Ablations on Smith and Sanchez dataset. The best and second-best values are highlighted and underlined.
Smith
Sanchez
Variant
Precision
Recallt
Fir
Accuracyt
SHD1
Precision"
Recallt
Fit
Accuracy!
SHD)
Lamma-888 (Default)
0.57:0006
0.010.15
066710.11
0.84
4.02:0.84
0,78:10.02
0.97:0002
0.86:0.07
09110005
2.10:10.75
Mismal-78
0.29+0007
0.0-0.100
0,35+0.13
0.68-0.08
7,5+1.36
01,48+0.05
0.43+0003
0.460.112
0,72+007
7.13+11224
Transfommer Decoder
0.44
0.78:0.11
0.56:00.12
0.74g0006
6.16:00.2
0.39:0.13
0.82:0.09
0.53+0.11
0.71=0007
6.35::0.88
Speasman Correlation Matris
0.43100:9
0.62:0.13
0.51
0.76100.08
6.14:10.91
0.39100077
0.67:10111
0.47
0.72.0006
6.45.0.88
Kendall Conrelation Matrix
0,51=0066
0,76=00112
0.1100.15
0:81=00088
1.23+1.06
00.53+0.05
0,78
0.33+0.166
0.80000777
5.12+1.10
w/o Prompes Generation
0.38:0004
0.60=0:15
0.46:00114
0.72=0008
7.10:0.93
0.52:00.0
0.57:10.04
0.54
0.72=0006
6.92:0.86
Multiscale Mixingino E
0.2010.08
0.2110.12
0.20.10113
0.6810.06
7.8810.95
0.11::0:05
0.29.10.01
0.3110.08
0.6810006
7.9111,19
w/o Cross Attention
0,23+0012
0,32+00.13
0.27+0.12
0:53=0006
11.80+-0933
01,43:0.04
0.35=0008
0.39:0.100
0,71=0077
7.24+0093
w/o LLM
0.33=0.11
0,42=0.14
0.35++014
0,72=0.08
7.14+1.200
0.50##0.07
0.55=#0.06
0.52+0.13
0.76=0006
7.04+1.16
Description
0.33+0007
0:25:0.13
0.28
0.76::0.04
6.32+0.97
0.355:0.06
0.27=00122
0.30
0.74:0005
6.15
woo Dotaset Description
0.1510.13
0,20:0.122
0,1710.18
0.62:0.07
10.3111:42
0.0810.14
0.28:10111
0.1210.17
0.7310005
8.9511/45
wio Peanson Correlation
0,43=0008
066-+0.09
0,51+0:13
0.76+0006
6.11+11.26
00322+0077
0.71:0.10
0.42+0:12
U85+0005
7.25++130
wla hops
0.53+0.06 062-+0.15
0,57+0.14
D81+0005
5.28+0.94
00.4++0.77
0.70+0.14
0.50-0.13
088+0004
6.15+0.90
w/o Like
0.34
0:22:0114
0.26
0.77=0.08
6.2211.23
0.28:0.11
0.15:0.13
0.18:0.15
0.8520007
1.35
wlo Love
0.58
0,70.10.16 0.57.00.09 0,7810.07 1.81
0.67.0.01
0.98.10.03 0.76:10.10 0.8210.07 1.34
Our results indicate that including Lese provides a slight improvement over not including it. For other components, such as the prompts generation module and multiscale mixing module (includes both multiscale decomposition and mixing, without loss Lese), the improvements are more pronounced in BrainBC-LLMM. Additionally, the results using Llama as the backbone are manifestly outperforms Mistral.
E.5 Prompt Complexity Analysis
We conduct experiments to examine the effect of prompt complexity, using both simplified and complex prompts. The results are presented in Table [3] and examples of complex prompt is shown in Figure (b). These results demonstrate that all components of the prompt are crucial, with the absence the task description and dataset description leading to a significant decline in performance metrics.
Table 13: Sensitivity analysis experiment on the simplicity or complexity of prompts.
Variant
Smith
Sunchez
Precision
Rocallf
Fi
Accuracy!
SHD
Precision
Recallf
Accuracyt
SHD1
Llama3-88 (Default)
0.57=006
0,0010.15
0.6710.11
0.8400.66
4.02:0084
0,78-0002
0.9710,02
0.86100077
0.91:0005
2,10:00.5
wis Tisk Description
0:33-0007
0.25-40113
0.28++.111
0,76-0.44
6.32+0.97
0.35-006
0.27+0.12
0.30+0.10
0.74-0.5
6.15+1.02
weo Danaset Description
0.15=0.13 0:20::.122
0.1710.18
0.62=0.77
10.31
0.08
0.2860.11
0.1240.17
0.73=0006 1.95
wio Pearson Correlation
0.43-0088
0.63+0.09
0.51++0.13
0,76-0066
6.11++1.26
0.32=0007
0.7140.10
0,42+0.12
0.85-005
7.25++130
Complex Prompt
0.62:0010 0.6110.12 0.6140.14 0.83-0.06
4.23:60.2
0.83-00066
0.71110.06
0.76.00.11
0.88-0007
3.1720.95
E.6 Hyperparameter Analysis
We conduct experiments on Smith datasets to evaluate the parameter sensitivity of BrainEC-LLM. Figures 9 (a) and (b) display the experimental results for soft threshold hyperparameters and dropout variations, respectively. When the soft threshold is set to 0.5, all metrics are optimal except for Recall, which remains sub-optimal, a pattern also observed with dropout. Consequently, the soft threshold is ultimately set to 0.5, and dropout is set to 0.01.
As shown in Figure (c), (d), and (c), we also conduct sensitivity analysis for the loss weight coefficients. Given the different magnitudes of each loss term, we carefully selected distinct ranges for the hyperparameter settings in experiments. For instance, we set aspa to 10.1, 1, 2. 10, 100) and Oddgg to (1, 10, 100, 500, 1000). For aspa, performance peaks at 2, while the best results for Odag and aese achieved at 100 and 10, respectively. Both overly small and large values degrade performance, highlighting the need for balanced regularization. We therefore set aspa = 2, O'dag 100, and Cresc = 10 in all experiments.
We also conduct additional experiments with a refined search around the optimal region of soft threshold hyperparameter 17. Since the best performance occurs at 17 0.5 in Figure (a), we perform
32

---

0.8
0.6
Prectsion
Recall
Accuracy
0.05
880
SHD
0.75
0.2
0.70
03 04
03
56
07
03 04 05 66 87 hyperparameter
53 04 05 hyperparameter
05
87
04
03 06 07
003 04 05 06 BY hyperparameter ?
nypperarrmeeee
hyperparameter
Influence of the soft threshold hyperparameter 7).
0.60
0.50
0.9
0.50
Recall
SHD
0.00
0.45
0.40
0.75
0.2
0.061..01
0
0.301..01
0.201031
dropout
dropout
dropout
dropout
dropout
(b) Influence of the dropout.
0.8
0.8
Accuracy
Preciiiion
2.4
Recall
0.6
0.8
SHD
0.4
0.
0.2
10
100
10
100
0.1
100
0.3
100
agpa
Agga
0gg
0gps
(c) Influence of the loss weights coefficient Osspa
0.9
Precision
Recall
0.5
Accuracy
0.8
0.4
0.4
0.2
100
0.2
100
100 Choap
500
1800
100
0gop
ddag
(d) Influence of the loss weights coefficient Cldag-
0.5
10
Precision
0.5
0.6
Accuracy
Recall
SHD
0.4
82
0.3
0.4
0.2
0.2
100
Gcsc
Dlesc
Influence of the loss weights coefficient dleac-
Figure Hyperparameter analysis on the Smith dataset, where the starred results are the best results.
Figure 10: Influence of the soft threshold hyperparameter 7) with fine-grained step.
fine- fine-grained search from 0.4 to 77 = 0.6 with step 0.02. The results are presented in Figure TO The refined search shows that optimal performance is consistently achieved at E E(0.48,0.50.0.52 across all metrics. The choice of y = 0.5 particularly intuitive given threshold formulation: mnin[|A)) When 7/ 0.5, the threshold is set to the midpoint between the minimum and maximum absolute values in the adjacency matrix, which represents natural balance point for binary classification.
0 = minn A)) + 7) (max()A)
33

---

Limitation
One potential limitation that warrants further investigation is that existing LLMs are primarily pre- trained on massive textual corpora and therefore inherently optimized for processing and reasoning over natural language. contrast, fMRI time series data are continuous, high-dimensional numerical signals that exhibit complex temporal and spatial dependencies. This modality gap introduces challenges in effectively aligning the statistical properties of fMRI signals with the natural language representations used in LLMs. As result, current LLMs may not fully capture the fine-grained temporal dynamics and -ppeif- patterns embedded in fMRI data.
34


Reviews:
[{"review_id": "GQlibbFn5H", "reviewer": "Reviewer_PhSv", "summary": "This paper proposes BrainEC-LLM, a novel approach that fine-tunes large language models (LLMs) for estimating effective connectivity (EC) from fMRI data. The authors introduce a multiscale decomposition mixing module and employ cross-modal alignment via pre-trained word embeddings, aiming to address the challenges posed by complex causal structures and the modality gap between fMRI signals and natural language.", "questions": "1.The authors should clarify how the prior knowledge used in the method is obtained. Is it extracted from external resources, domain-specific rules, or learned during training?(page 3, section 4.1)\n2. The procedure of the “Mixing” step depicted in Figure 1 lacks sufficient explanation. Please elaborate on how the mixing operation is performed, including the rationale, inputs, and expected outcomes.\n3. The distinction between bottom-up and top-down mixing is not clearly defined. (page 4, section 4.2)\n4. In Figure 3, the authors evaluate model performance based on predicting correct, spurious, and missing connections. However, the directionality and weight distribution of connections are not considered, which could be critical in many tasks. This omission weakens the validity of the evaluation. The authors are encouraged to provide an explanation for this and consider incorporating direction and weight analysis in future work.\n5. In the ablation study presented in Table 4, the proposed method does not outperform the baseline (LLaMA3-8B). The authors should provide further analysis or justification to demonstrate the practical advantages of their method, especially when the performance gains are marginal or absent.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "While the research topic is timely and the application to EC is novel, the manuscript would benefit from more detailed explanations of key components, a clearer articulation of theoretical contributions, and further analysis to justify its claimed advantages.", "was_revised": true, "final_justification": "After reading the authors’ rebuttal in detail, I acknowledge that several of my original concerns have been addressed. I have therefore increased my score from 3 to 4.", "rebuttal": "We truly appreciate the time and effort you have dedicated to reviewing our work. Your expert insights are invaluable to us, and we have taken your feedback seriously in crafting our responses.\n\n**Strengths And Weaknesses:**\n\nWe appreciate your recognition of the timeliness and novelty of our work. In the **revised version** of the manuscript, we will provide a **clearer statement of the motivation** behind our method, along with **more detailed explanations of key components**, particularly the **bottom-up and top-down mixing mechanisms** in the multiscale decomposition mixing module.\n\n**Questions**\n\n**Q1** \n\nThank you for your insightful comment. We apologize for any confusion caused by the wording. The \"prior knowledge\" mentioned in our method refers to the **Pearson correlation coefficient matrix computed directly from the input fMRI time series data.** It is not obtained from external resources or domain-specific rules, nor is it learned during training.\n\nInstead, it serves as a **data-driven prior**, reflecting **instantaneous statistical dependencies** among brain regions. This correlation matrix is calculated as a preprocessing step before effective connectivity estimation and is used to guide or regularize the learning of directed connections, helping the model focus on potentially relevant region pairs. This approach aligns with practices in previous literature where undirected correlation-based connectivity is used to assist the learning of more complex directional or causal models.\n\n**We will revise the manuscript to make the source and role of this prior knowledge clearer.**\n\n**Q2**\n\nThe \"mixing\" step in BrainEC-LLM refers to the integration of multiscale temporal trends from fMRI data, implemented in the **Multiscale Decomposition Mixing (MDM) module** to disentangles complex fMRI multiscale temporal variations from a new perspective. Here is a detailed breakdown:\n\n- **Rationale**: \n  \n  fMRI time series exhibit distinct patterns at different temporal sampling scales. For instance, high-resolution fMRI data **recorded per second (lower-level scale) provide higher spatial and temporal resolution**, enabling the capture of subtle changes and local neural dynamics. In contrast, **low-resolution fMRI data recorded hourly (higher-level scale) tend to reflect the overall or global activity patterns of brain regions.**\n  \n  These observations naturally motivate the adoption of a multiscale analysis paradigm to disentangle the complex temporal structures embedded in fMRI signals. A multi-scale perspective enables the model to separate and capture diverse components of brain activity across different timescales, which is crucial for accurately modeling temporal variations.\n  \n  Specifically, for **short-term trends, where fine-grained temporal fluctuations such as rapid neural responses are more prominent, we adopt a bottom-up mixing strategy that emphasizes detailed local information**. Conversely, for **long-term trends, which emphasize broader patterns like sustained functional connectivity, we employ a top-down mixing strategy to incorporate high-level global structures.**\n\n- **Inputs**: The module takes decomposed short-term multiscale trends $\\mathcal{S} = \\{\\mathcal{S}_1, ..., S_M\\}$ and long-term multiscale trends $\\mathcal{T} = \\{\\mathcal{T}_1, ..., \\mathcal{T}_M\\}$ as inputs. These trends are derived from downsampled fMRI time series at M scale, where lower scales $(\\mathcal{S}_1, \\mathcal{T}_1)$ capture high-resolution temporal details and higher scales $(\\mathcal{S}_M, \\mathcal{T}_M)$ reflect macroscopic patterns.\n\n- **Expected Outcomes**: The mixing step produces fused multiscale features that retain both short-term dynamics and long-term stability. \n\n**Q3**\n\n- **Bottom-up mixing** (for short-term multiscale trends): \n  \n  For short-term trends, where fine-grained temporal fluctuations such as rapid neural responses are more prominent, it is important to preserve and propagate detailed temporal information across scales. In this setting, fine-grained features from lower-level scales (high-resolution, e.g., $\\mathcal{S}_1$) are passed to higher-level scales (coarser, e.g., $\\mathcal{S}_M$) to enhance their representations with detailed temporal dynamics. This process is formulated as $\\mathcal{S} _i=\\mathcal{S} _i+\\Phi(\\mathcal{S} _{i-1})$ where $\\Phi(\\cdot)$ denotes a ModernTCN block that can better utilize temporal features and cross-brain connectivity in fMRI time series.\n  \n  designed to effectively leverage temporal dependencies and cross-region interactions in fMRI time series.\n\n- **Top-down mixing** (for long-term multiscale trends): \n  \n  For long-term trends, which emphasize broader patterns such as sustained functional connectivity, it is essential to guide lower-level representations with high-level global information. In this approach, stable patterns from higher-level scale (coarser, e.g., $\\mathcal{T}_M$) are propagated downward to constrain lower-level scale details (e.g., $\\mathcal{T}_1$), helping to suppress noise and maintain consistency across temporal resolutions. This process is formulated as $\\mathcal{T}_i = \\mathcal{T} _i + \\Phi(\\mathcal{T} _{i+1})$, where high-level information enhances lower-level representations, thereby ensuring consistency between global trends and local variations.\n\n**Q4** \n\nThank you for your insightful comment. We would like to first clarify an important point: **the brain effective connectivity (EC) networks directly output by our method are directed and weighted graphs. After post-processing, we obtain binarized EC matrices that still preserve directionality**, with the edge weights discretized to 0 or 1 to indicate the presence or absence of a directed connection.\n\nFor evaluation purposes, we binarize the EC matrices to match the format of the **ground truth in the simulated datasets**, which are provided as **binary but still directed** adjacency matrices. Similarly, for real fMRI data, we visualize the EC network by showing whether a directed edge exists between brain regions, which aligns with common practices in the literature where true underlying weights are not available. Details of this post-processing procedure are provided in **Appendix D.3. Below is an excerpt from the appendix for your reference:**\n\nFor the obtained brain EC matrix $A$, a threshold $\\theta$ and maximum number of parent nodes $MaxPa$ is required to convert $A$ into a binary matrix. Given that different fMRI datasets may exhibit varying signal characteristics and noise levels, distinct thresholds need to be applied. Therefore, we set the threshold $\\theta$ to be adaptive, and it is calculated as follows:\n\n$\\theta=min(|A|)+ \\eta\\times(max(|A|)-min(|A|))$\n\nwhere $\\eta$ is a hyperparameter used to adjust thresholds. The maximum parent node  proportion is established for the brain EC matrix, with the maximum number of parent nodes $MaxPa$ being determined by multiplying the number of brain regions $N$ with the maximum parent node proportion $MaxPr$. For values exceeding the threshold in each row and column, the top $MaxPa$ value is set to 1, while all other values are set to 0.\n\n**We will include the above-mentioned rationale for the post-processing step in the revised version of the manuscript.**\n\n**Q5**\n\nWe apologize for the confusion caused by the wording in Table 4. To clarify, the **\"LLaMA3-8B\" row in the ablation study refers to our proposed method using LLaMA3-8B as the backbone LLM**. This is not a baseline method, but rather serves as the **default setting** of BrainEC-LLM.\n\n\n**Thank you for your constructive feedback. We will re-examine our work and incorporate the above content and experiments into the new version of the paper. We hope our comprehensive response adequately addresses your concerns. If you have any additional questions or require further clarification on any aspect of our work, we would be more than happy to provide additional details and engage in further discussion.**"}, {"review_id": "fPUZBHkFUK", "reviewer": "Reviewer_VkW4", "summary": "The manuscript proposes BrainEC-LLM for fine-tuning large language models to estimate brain effective connectivity from fMRI time series data. It introduces a multiscale decomposition mixing module to capture both short- and long-term temporal trends and a cross-attention mechanism to align fMRI features with natural language embeddings. Experiments on both simulated and real fMRI datasets show that BrainEC-LLM outperforms state-of-the-art baselines in EC estimation and demonstrates strong generalization to new datasets and downstream tasks.", "questions": "1. How sensitive is the model to the generated prompts if you make them simpler or more complex (with additional background information that would explain some terminology)?\n2. The authors have to improve visualization and discussion when comparing brain networks. See weaknesses.", "limitations": "The authors adequately addressed the limitations.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Comprehensive evaluation across six diverse datasets, including both simulated and real-world fMRI data.\n2. Comprehensive ablations: testing different backbone models, removing the prompt generation, multiscale mixing, cross-attention, LLM module, and cross-scale contrastive loss, thresholds, and dropout.\n\nWeaknesses:\n1. Table 1, Table 2, Table 3, Table 4, Table 10 do not have statistical testing between the best model and others (e.g., Wilcoxon). Where there are multiple comparisons, specific correction should be applied (e.g., Holm).\n2. Tables do not have AUROC. It is important because it is not a threshold-based metric.\n3. Figure 4 is too small. It is very difficult to draw any conclusions from it. You'll need to make them better for visualization. Although it is larger in Figure 8, the authors should compare and visualize the differences, rather than just showing Healthy and ASD as examples.", "was_revised": true, "final_justification": "After a careful review of the paper, the author's rebuttal, and further discussion, I am maintaining my original overall assessment of borderline accept. \n\nResolved concerns:\n- The authors addressed my concerns regarding statistical testing by providing additional Wilcoxon test results (with Holm correction) and clarifying significance findings for Table 1.\n- AUROC has now been added to the results. \n- The authors acknowledged and committed to improving the clarity and interpretability of the visualizations in Figures 4 and 8.\n- The prompt sensitivity and ablation results were further clarified and supported with additional experiments, as requested.\n\nThe manuscript would also benefit from more polished and better-annotated visualizations, as highlighted by me and several other reviewers. Currently, these improvements have only been promised and are not yet reflected in the submission. Since updated visualizations could potentially impact the interpretation of results and the discussion section, I am not willing to raise my score at this time.", "rebuttal": "Your careful examination and valuable critique of our paper are highly appreciated. We are grateful for your guidance and have addressed your points in detail below.\n\n**Strengths And Weaknesses:**\n\n**W1**\n\nWe **have already performed t-test significance** experiments on the data presented in Table 1 of the original manuscript. For detailed results, **please refer to Appendix E.1**. Additionally, in response to your suggestion, we conducted a Wilcoxon test with Holm correction, and the updated results are shown in Table 1 below. **These new findings are consistent with our previous t-test results**, indicating no significant difference in SHD between BrainEC-LLM and RL-EC. Moreover, the Wilcoxon test reveals no significant difference in recall between BrainEC-LLM and CR-VAE. However, significant differences are observed in other evaluation metrics, further highlighting the superiority of our method. We appreciate your feedback, which has contributed to a more rigorous analysis of the model comparison.\n\n**Table 1: p-values obtained from the Wilcoxon for BrainEC-LLM and seven baseline methods. Underlined values indicate no significant difference with Holm correction.**\n|Method|Precision|Recall|F1|Accuracy|SHD|AUROC|\n|-|-|-|-|-|-|-|\n|spDCM|$1.61\\times10^{-2}$|$6.84\\times10^{-3}$|$4.88\\times10^{-4}$|$2.44\\times10^{-3}$|$4.88\\times10^{-4}$|$4.88\\times10^{-4}$|\n|lsGC|$4.88\\times10^{-4}$|$1.47\\times10^{-3}$|$4.88\\times10^{-4}$|$9.77\\times10^{-4}$|$4.88\\times10^{-4}$|$4.88\\times10^{-4}$|\n|ACOCTE|$3.13\\times10^{-2}$|$4.88\\times10^{-4}$|$9.77\\times10^{-4}$|$1.61\\times10^{-2}$|$6.84\\times10^{-3}$|$9.77\\times10^{-4}$|\n|RL-EC|$1.20\\times10^{-2}$|$4.88\\times10^{-4}$|$3.42\\times10^{-3}$|$5.62\\times10^{-3}$|$\\underline{3.42\\times10^{-2}}$|$6.84\\times10^{-3}$|\n|CR-VAE|$9.77\\times10^{-4}$|$\\underline{8.98\\times10^{-1}}$|$1.46\\times10^{-3}$|$9.77\\times10^{-4}$|$9.77\\times10^{-4}$|$9.77\\times10^{-4}$|\n|MetaCAE|$3.42\\times10^{-3}$|$4.65\\times10^{-3}$|$3.42\\times10^{-3}$|$1.95\\times10^{-3}$|$3.42\\times10^{-3}$|$2.93\\times10^{-3}$|\n|MetaRLEC|$4.88\\times10^{-3}$|$4.88\\times10^{-4}$|$4.88\\times10^{-4}$|$4.88\\times10^{-4}$|$4.88\\times10^{-4}$|$4.88\\times10^{-4}$|\n|CUTS+|$4.88\\times10^{-4}$|$3.42\\times10^{-3}$|$4.88\\times10^{-4}$|$4.88\\times10^{-4}$|$4.88\\times10^{-4}$|$4.88\\times10^{-4}$|\n\n**W2**\n\nThank you for this valuable suggestion. We completely agree that AUROC is a crucial evaluation metric, as it is threshold-independent and provides a more comprehensive assessment of model performance across different decision thresholds.\n\nFollowing your recommendation, we have incorporated AUROC results into our experimental evaluation, as shown in Table 2. The results demonstrate that BrainEC-LLM achieves superior performance on the AUROC metric as well, further validating the effectiveness and robustness of our proposed approach.\n\nThe inclusion of AUROC makes our evaluation more comprehensive and objective. We greatly appreciate the reviewer's guidance in helping us refine our experimental assessment framework, and **we will incorporate this evaluation metric in the revised version of our manuscript**.\n\n**Table 2: The mean and variance of the nine methods on the simulated fMRI datasets. The best and second-best values are **highlighted** and $\\underline{underlined}$.**\n|Datasets|Metrics|spDCM2014|lsGC2017|ACOCTE2022|RL-EC2022|CR-VAE2023|MetaCAE2024|MetaRLEC2024|CUTS+2024|BrainEC-LLM(Ours)|\n|-|-|-|-|-|-|-|-|-|-|-|\n|Smith|Precision↑|0.50±0.00|0.50±0.00|0.53±0.17|**0.58±0.12**|0.45±0.19|0.55±0.10|0.38±0.05|0.33±0.09|$\\underline{0.57±0.06}$|\n|      |Recall↑|0.40±0.00|0.60±0.00|0.40±0.16|0.47±0.09|0.73±0.25|$\\underline{0.79±0.15}$|0.60±0.12|0.42±0.12|**0.80±0.15**|\n|      |F1↑|0.44±0.00|0.55±0.00|0.45±0.17|0.52±0.10|0.56±0.21|$\\underline{0.65±0.12}$|0.46±0.07|0.37±0.06|**0.67±0.11**|\n|      |Accuracy↑|0.80±0.00|0.80±0.00|0.81±0.05|$\\underline{0.83±0.04}$|0.76±0.11|0.82±0.08|0.72±0.03|0.72±0.04|**0.84±0.06**|\n|      |SHD↓|5.00±0.00|5.00±0.00|4.67±0.25|4.33±0.94|6.00±2.83|$\\underline{4.26±0.93}$|7.00±0.86|7.00±0.78|**4.02±0.84**|\n|      |AUROC↑|0.63±0.00|0.70±0.00|0.64±0.10|0.68±0.06|0.71±0.17|$\\underline{0.77±0.12}$|0.63±0.08|0.57±0.09|**0.80±0.08**|\n|Sanchez|Precision↑|0.57±0.00|0.60±0.00|0.76±0.06|**0.80±0.02**|0.50±0.04|0.50±0.07|0.80±0.04|0.64±0.03|$\\underline{0.78±0.02}$|\n|       |Recall↑|0.57±0.00|$\\underline{0.86±0.00}$|0.57±0.00|0.57±0.07|0.76±0.13|0.29±0.11|0.57±0.14|0.86±0.00|**0.97±0.02**|\n|       |F1↑|0.57±0.00|0.71±0.00|0.65±0.02|0.67±0.05|0.60±0.07|0.37±0.07|0.67±0.06|$\\underline{0.74±0.02}$|**0.86±0.07**|\n|       |Accuracy↑|0.76±0.00|0.80±0.00|0.83±0.02|$\\underline{0.84±0.02}$|0.72±0.03|0.72±0.04|0.84±0.03|0.83±0.02|**0.91±0.05**|\n|       |SHD↓|6.00±0.00|5.00±0.00|4.33±0.47|$\\underline{3.95±0.47}$|7.05±0.82|6.89±0.85|4.00±0.89|4.33±0.47|**2.10±0.75**|\n|       |AUROC↑|0.67±0.00|0.77±0.00|0.73±0.02|0.75±0.05|0.68±0.06|0.57±0.06|0.75±0.04|$\\underline{0.80±0.05}$|**0.92±0.04**|\n|SCN5|Precision↑|0.40±0.00|0.27±0.00|0.50±0.04|$\\underline{0.67±0.05}$|0.40±0.03|0.36±0.02|0.50±0.01|0.45±0.02|**0.83±0.04**|\n|    |Recall↑|**0.75±0.00**|0.38±0.00|0.13±0.02|0.25±0.08|$\\underline{0.67±0.06}$|0.58±0.06|0.25±0.04|0.63±0.03|0.63±0.05|\n|    |F1↑|0.52±0.00|0.32±0.00|0.20±0.04|0.36±0.04|0.50±0.04|0.44±0.03|0.33±0.03|$\\underline{0.53±0.03}$|**0.71±0.04**|\n|    |Accuracy↑|0.56±0.00|0.48±0.00|$\\underline{0.68±0.05}$|0.72±0.12|0.57±0.04|0.53±0.02|0.68±0.07|0.64±0.09|**0.84±0.10**|\n|    |SHD↓|11.00±0.00|13.00±0.00|8.00±0.43|$\\underline{6.82±0.85}$|10.67±0.94|11.67±0.47|8.24±0.45|9.06±0.67|**4.26±0.83**|\n|    |AUROC↑|0.50±0.00|0.35±0.00|0.52±0.04|0.58±0.03|0.50±0.05|0.44±0.03|0.54±0.01|$\\underline{0.56±0.03}$|**0.77±0.04**|\n|SCN7|Precision↑|$\\underline{0.67±0.00}$|0.30±0.00|**0.77±0.02**|0.65±0.01|0.37±0.00|0.34±0.03|0.52±0.02|0.34±0.00|0.62±0.02|\n|    |Recall↑|0.13±0.00|0.60±0.00|0.22±0.03|0.49±0.03|$\\underline{0.80±0.14}$|**0.84±0.08**|0.47±0.13|0.58±0.02|0.61±0.05|\n|    |F1↑|0.22±0.00|0.40±0.00|0.34±0.04|$\\underline{0.56±0.03}$|0.50±0.03|0.48±0.05|0.48±0.04|0.43±0.03|**0.61±0.03**|\n|    |Accuracy↑|0.71±0.00|0.45±0.00|0.74±0.01|$\\underline{0.75±0.01}$|0.52±0.03|0.44±0.05|0.69±0.05|0.52±0.02|**0.76±0.05**|\n|    |SHD↓|14.00±0.00|27.00±0.00|12.67±0.47|**11.67±0.47**|23.67±1.70|27.33±2.49|14.94±0.82|23.33±0.94|$\\underline{11.86±0.91}$|\n|    |AUROC↑|0.55±0.00|0.41±0.00|0.59±0.02|$\\underline{0.67±0.02}$|0.52±0.01|0.46±0.06|0.60±0.02|0.47±0.04|**0.68±0.03**|\n\n\n**W3**\n\nThank you for your constructive suggestion. We acknowledge that the visualization in Figure 4 is overly complex and lacks clear emphasis, which makes it difficult for readers to extract meaningful conclusions. **In the revised manuscript, we will update the visualizations and highlight the key brain regions** to enhance clarity and facilitate the interpretation of the results.\n\n**Questions:**\n\n**Q1**\n\n**Regarding the simplification of the prompt, we have already conducted detailed ablation experiments on each component of the prompt, which are presented in Appendix E.4. The results are summarized in Table 3 below** (w/o Task Description, w/o Dataset Description, w/o Pearson Correlation). Additionally, we **conducted experiments involving more complex prompts** that introduce background information on fMRI and EC. The experimental results are also shown in Table 3. These results demonstrate that all components of the prompt are crucial, with the absence of the Task Description and Dataset Description leading to a significant decline in performance metrics.\n\nThe prompt used for introducing the background information is as follows:\n> fMRI is a neuroimaging technique that measures brain activity by tracking changes in blood oxygenation. \n    If we use simulated fMRI data, the time series are directly generated, while real fMRI data consists of time series that have been preprocessed by external sources. \n    We use these preprocessed fMRI time series to estimate brain effective connectivity (EC), which refers to the directed influence between brain regions, capturing causal interactions rather than just correlations.\n\n**Table 3: Sensitivity analysis experiment on the simplicity or complexity of prompts.**\n| Variant| **Smith**|||||**Sanchez**|||||\n|-|-|-|-|-|-|-|-|-|-|-|\n||Precision↑|Recall↑|F1↑|Accuracy↑|SHD↓|Precision↑|Recall↑|F1↑|Accuracy↑|SHD↓|\n|**Llama3-8B(Default)**|0.57±0.06|**0.80±0.15**|**0.67±0.11**|**0.84±0.06**|**4.02±0.84**|0.78±0.02|**0.97±0.02**|**0.86±0.07**|**0.91±0.05**|**2.10±0.75**|\n|w/oTaskDescription|0.33±0.07|0.25±0.13|0.28±0.11|0.76±0.04|6.32±0.97|0.35±0.06|0.27±0.12|0.30±0.10|0.74±0.05|6.15±1.02|\n|w/oDatasetDescription|0.15±0.13|0.20±0.12|0.17±0.18|0.62±0.07|10.31±1.42|0.08±0.14|0.28±0.11|0.12±0.17|0.73±0.06|8.95±1.45|\n|w/oPearsonCorrelation|0.43±0.08|0.63±0.09|0.51±0.13|0.76±0.06|6.11±1.26|0.32±0.07|0.71±0.10|0.42±0.12|0.85±0.05|7.25±1.30|\n|ComplexPrompt|**0.62±0.10**|0.61±0.12|0.61±0.14|0.83±0.6|4.23±0.92|**0.83±0.06**|0.71±0.06|0.76±0.11|0.88±0.07|3.17±0.95|\n\n**Q2** Please see W3 for more details.\n\n**We sincerely appreciate your thoughtful insights. Your suggested detailed aspects and experimental approaches will be integrated into the revised manuscript. We hope our responses have adequately addressed all concerns, and we would be most grateful if you could consider increasing the Rating score accordingly.**"}, {"review_id": "OCotYyVi6B", "reviewer": "Reviewer_g8f7", "summary": "This paper addresses the problem of estimating brain effective connectivity (EC) from fMRI time series data by proposing BrainEC-LLM. This method fine-tunes LLMs for brain EC estimation to overcome the limitations of prior approaches that relied on inference-only or pairwise querying, and addresses the shortcomings of traditional machine learning or deep learning methods, which lack the ability to capture multiscale temporal dynamics and require task-specific retraining.\nBrainEC-LLM addresses these issues through a structured architecture comprising five key components: 1) a prompt generation module to provide the LLM with contextualized knowledge about the task, fMRI dataset characteristics, and prior inter-regional correlations; 2) multiscale decomposition mixing to isolate temporal trends at various scales; 3) cross-attention with pre-trained word embeddings to bridge the gap between fMRI signals and language models; 4) multiscale reconstruction mixing for feature fusion; and 5) a self-attention module for estimating directed EC graphs. Experiments were conducted on simulated datasets (Smith, Sanchez, SCN5/7), high-resolution real fMRI data, and the ABIDE I dataset for ASD classification (downstream task). The model outperformed eight baselines on different metrics and showed zero-shot generalization. Also, the proposed method achieved the highest classification accuracy (71.1%) for ASD vs. control. Ablation studies confirmed the importance of each aforementioned module, particularly the multiscale and cross-attention mechanisms. The time complexity analysis showed that most of the computational cost stems from the LLM component, while the added modules contribute minimally.", "questions": "* What is the underlying intuition for the proposed multiscale decomposition and downsampling strategy?\n\n* Could a wavelet transformation have been used instead of multiscale decomposition and downsampling strategy, and what benefits does the proposed approach offer compared to wavelet-based alternatives?\n\n* How exactly were the datasets used for training and evaluation? The current explanations in the appendix are vague...", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nOverall, this paper addresses an interesting problem with a well-motivated approach. The proposed method is both innovative and practically applicable and offers new insights into the intersection of fMRI analysis for brain EC estimation and language models. The paper is clearly written, thoroughly covering different aspects of the problem and methodology. Additionally, the authors provide extensive implementation details in the appendix, which supports reproducibility for further research.\n\nWeaknesses:\n\nAlthough the method itself is clearly described, one area that could be improved is explaining the intuition behind the proposed multiscale decomposition and downsampling strategy. Could wavelet transformation have been used instead, and what are the advantages of the proposed method over wavelet-based approaches? Also, the framework figure would benefit from clearer dimensional annotations to help readers better understand the data flow and transformations across modules. Finally, more detailed and transparent information about how the datasets were used for training is necessary. The descriptions in the appendix are vague, and it is difficult to fully assess the experimental setup.", "was_revised": true, "final_justification": "After considering the rebuttal and the clarifications provided by the authors, I maintain my score of 4.\n\n- The authors adequately addressed my main concerns regarding the intuition behind the multiscale decomposition and the downsampling strategy. Their explanation clarified the role of bottom-up and top-down mixing across temporal scales.\n\n- I suggested the potential use of wavelet transforms as an alternative. While the authors highlighted differences, I believe wavelets could still offer a principled and comparable approach worth exploring in future work. This remains an open point but does not affect the current evaluation significantly.\n\n- Other concerns, such as the need for clearer dimensional annotations in the framework figure and more detailed information on dataset usage, were acknowledged and are expected to be improved in the revised manuscript.\n\nOverall, the work presents a new and interesting application of LLMs to brain effective connectivity estimation, which I believe has value to the community.", "rebuttal": "Thank you for your thorough review and valuable insights. We deeply appreciate the feedback provided and have responded to your comments below.\n\n**Strengths And Weaknesses:**\n\n***W1** Although the method itself is clearly described, one area that could be improved is explaining the intuition behind the proposed multiscale decomposition and downsampling strategy.*\n\nThank you for your helpful suggestion. We agree that further clarification of the intuition behind our multiscale decomposition and downsampling strategy would improve the manuscript.\n\nThe **core motivation** behind the downsampling and decomposition components **is to support the bottom-up and top-down paradigm** proposed in our method. This paradigm requires the model to operate across multiple temporal scales and to apply different processing strategies to distinct multiscale trends, such as short-term and long-term dynamics.\n\nSpecifically, the **downsampling** process is implemented using average pooling to generate fMRI time series at multiple temporal resolutions. Lower-level scales preserve higher spatial and temporal granularity, enabling the capture of subtle fluctuations and localized neural activity. In contrast, higher-level scales reflect broader or global activity patterns across brain regions.\n\nThe **decomposition** step further separates the multiscale fMRI time series into trend components with different temporal characteristics, distinguishing between **short-term multiscale trends** and **long-term multiscale trends**. This structure allows the model to apply bottom-up mixing to the short-term trends, while applying top-down mixing to the long-term trends.\n\n**We will revise the manuscript to include this explanation** and improve the clarity of the  motivation in the relevant section.\n\n***W2** Could wavelet transformation have been used instead, and what are the advantages of the proposed method over wavelet-based approaches?*\n\nWhile wavelet transform provides a mathematically grounded approach for multiscale signal analysis, it is not suitable for replacing the average pooling-based downsampling and series decomposition used in our method. This is because **wavelet transform produces a set of frequency-localized coefficients without explicitly separating coarse-grained temporal trends or enabling directional information fusion across scales.** In contrast, **our proposed framework is explicitly designed to support a bottom-up and top-down mixing** paradigm by generating multiscale representations that are both task-adaptive and structurally integrable with LLMs. Moreover, our downsampling and decomposition components are lightweight, offering better flexibility, interpretability, and performance in the context of effective connectivity estimation.\n\n***W3** Also, the framework figure would benefit from clearer dimensional annotations to help readers better understand the data flow and transformations across modules.*\n\nThank you for the suggestion. We acknowledge that the current framework figure lacks detailed dimensional annotations, which may hinder the reader’s understanding of the data flow and transformations. In the revised version, we will update the figure to include clear dimensional labels for each module to better illustrate the processing pipeline.\n\n***W4** Finally, more detailed and transparent information about how the datasets were used for training is necessary. The descriptions in the appendix are vague, and it is difficult to fully assess the experimental setup.*\n\nThank you for pointing this out. We would like to clarify that **our method does not involve explicit training and test set splitting because the model is designed in an autoregressive manner**. Specifically, due to the lack of ground truth for brain effective connectivity (EC), we adopt an **unsupervised approach** that leverages the inherent autocorrelation (i.e., the brain effective connectivity network) within the fMRI time series to predict the fMRI signals themselves. As a result, the model is trained by minimizing the difference between the predicted and actual fMRI signals. **The EC network is generated as the outcome of this optimization process, and evaluation metrics are directly computed based on the predicted EC network** without the need for a separate test set.\n\n**Additional details regarding the autoregressive design can be found in Appendix D.6. We will also include the above explanation in the revised version of Appendix D.6** to provide greater clarity.\n\n**Questions:**\n\n***Q1*** \n\nSee W1.\n\n***Q2*** \n\nSee W2.\n\n***Q3*** \n\nSee W4.\n\n**Thank you for providing such valuable and constructive suggestions. We will incorporate your suggestions into the new version to make the paper clearer and easier to understand. We hope our responses have adequately addressed all concerns, and we would be most grateful if you could consider increasing the Rating score accordingly.**"}, {"review_id": "LswUfFf7nX", "reviewer": "Reviewer_Qbm5", "summary": "This work proposed BrainEC-LLM to borrow the power of large-scale LLM for effective connectivity (EC) prediction. EC as a causal discovery task can naturally utilize the knowledge base of LLM to advance the performance. BrainEC-LLM is framed with multi-scale fMRI timeseries decomposition and reconstruction to address complex causal structures in brain EC task. BrainEC-LLM has shown superior performance in 4 stimulated datasets and competitive performance in one real dataset. Zero-shot learning experiments demonstrated the high compatibility of the text embedding by LLM.", "questions": "1. Why zero-shot learning has not test for real dataset? This will demonstrate the potential of this work for real-world applications.\n\n2. BOLD signal reconstruction is shown in the method figure. Why this loss is needed, and how is this loss affecting performance? \n\n3. Does the ablation study of w/o multiscale mixing lead to the model using pure LLM features? How is this implemented?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Superior performance in 4 stimulated datasets and competitive performance in one real dataset.\n\n2. Enable zero-shot learning capability for brain EC task.\n\nWeaknesses:\n1. The motivation is not clear. The authors bring a new approach to LLM based brain EC prediction due to the limited capacity of LLM to capture complex causal structures. The complexity of causal structures of EC, however, is not defined or described across the manuscript. The motivation of the second component multiscale decomposition mixing (MDM) is also hard to find. \n\n2. Figure 3 shows CUTS+ has the best performance (32 correct links), while the proposed BrainEC-LLM has fewer, 22 correct links.\n\n3. Zero-shot learning in Table 2 shows a diverse performance without explain, where SCN5 keeps the same F1 score but Smith dropped a lot.\n\n4. Table 4 shows one LLM model has manifestly outperformed another one. Performance by the bad one is the worst in comparison with baselines in Table 1, and it is less accurate than w/o LLM. This demonstrates the performance of BrainEC-LLM is highly relied on text embedding from LLM.\n\n5. Downstream application is limited in one dataset. Performance evaluation is limited with only a linear classifier, SVM.\n\n6. LLM is known as a large-scle model, and thus it holds a high computational cost. However, the computational costs of BrainEC-LLM are missed.\n\n7. Performance is unstable when alterring the hyperparameters as shown in Fig. 5. Although the best hyperparameter was found with step 0.1, the high variance of performance implict there are better hyperparameters if tuning with smaller steps.", "was_revised": true, "final_justification": "Rebuttal has addressed my concerns and i'm sticking with the positive rating.", "rebuttal": "Thank you very much for your feedback and for providing many professional suggestions that helped us improve the quality of the paper.\n\n**Strengths And Weaknesses:**\n\n**W1**\n- **The complexity of causal structures of EC**: Effective connectivity (EC) can be conceptualized as a directed graph that illustrates the causal relationships between brain regions. In this graph, each node corresponds to a brain region, and each directed edge represents the causal influence exerted by one region on another.\n\n  **EC is complex, primarily because it lacks a ground truth for validation.** The lack of a reference standard means that the **causal relationships between brain regions must be inferred indirectly**, relying on models that can capture the dynamic and context-dependent interactions between regions. \n\n- **multiscale decomposition mixing (MDM)**: Directly combining LLMs with fMRI time series data can compromises performance due to modality differences, and **LLMs struggle with capturing complex multiscale temporal variations**. To address this limitation, our method introduces a multiScale decomposition mixing component for effective multiscale temporal dynamics capture. fMRI time series exhibit distinct patterns at different scales: high-resolution data (e.g., per second) capture subtle local neural dynamics, while low-resolution data (e.g., pre hour) reflect global brain activity patterns. **This naturally motivates multiscale analysis to disentangle complex temporal structures in fMRI signals**. \n\n  Specifically, for short-term trends, where fine-grained temporal fluctuations such as rapid neural responses are more prominent, we adopt a bottom-up mixing strategy that emphasizes detailed local information. Conversely, for long-term trends, which emphasize broader patterns like sustained functional connectivity, we employ a top-down top-down strategy to incorporate high-level global structures.\n\n**W2** \n\nWhile CUTS+ identifies more correct connections (32 vs. 22), this doesn't imply better performance when accounting for spurious and missing connections. For instance, predicting all possible connections would identify all true connections but introduce many false positives.\n\nIn Figure 3, **CUTS+ identifies 12 spurious connections and misses 1 connection in both left and right hemispheres, while our method finds only 6 spurious connections in each hemisphere, though it misses 6 and 5 connections, respectively. This indicates that our method strikes a better balance** between detecting true connections and avoiding false positives, leading to more reliable EC estimation.\n\n**W3**\n\nThe diverse zero-shot performance in Table 2 reflects differences in network complexity between Smith and SCN5 datasets. SCN5, with 5 nodes, 8 arcs, and 4 cycles introducing quadratic nonlinear causality, represents a more complex structure than Smith (5 nodes, 5 arcs, 0 cycles). When transferring from Smith to SCN5, the model maintains stable F1 performance, leveraging multiscale features learned from Smith. Conversely, SCN5 → Smith transfer shows performance degradation, likely due to overfitting to SCN5's nonlinear patterns, making adaptation to Smith's simpler acyclic structure suboptimal.\n\n**W4**\n\nThe performance difference between Mistral-7B and Llama3-8B reflects varying capabilities in encoding neuroscience related concepts, with Llama3-8B offering better semantic representations for brain EC tasks. Mistral-7B's embeddings appear less aligned with fMRI data, leading to suboptimal cross-modal fusion. However, Mistral-7B's performance is only slightly worse than w/o LLM, showing that while LLM quality matters, our framework remains robust with less suitable text encoders. This highlights both the importance of appropriate LLM selection and our model's adaptability across different language models.\n\n**W5**\n\nWe added the XGBoost classifier, and the results for ABIDE I in Table 1 show that BrainEC-LLM outperforms all other methods. We also conducted experiments with SVM and XGBoost on ADHD dataset (https://fcon_1000.projects.nitrc.org/indi/adhd200/), with results in Tables 2 and 3. Table 3 will be provided later. Table 2 shows that while lsGC achieves higher Precision, BrainEC-LLM achieves the best overall performance.\n\n**Table1: Classification results on ABIDE I dataset using XGBoost classifier.**\n|Methods|Precision(%)↑|Recall(%)↑|F1(%)↑|Accuracy(%)↑|\n|-|-|-|-|-|\n|spDCM|59.67±9.24|54.79±5.48|57.13±7.43|65.24±5.75|\n|lsGC|53.71±8.34|48.82±5.70|51.04±6.69|55.01±5.37|\n|ACOCTE|49.62±8.48|47.71±4.59|48.65±6.25|50.73±4.62|\n|RL-EC|61.94±9.75|63.63±5.26|62.77±6.29|61.42±4.83|\n|CR-VAE|59.73±7.88|58.45±5.62|59.08±4.80|61.42±4.56|\n|MetaCAE|63.31±5.26|$\\underline{64.64±5.86}$|63.97±5.36|64.24±3.84|\n|MetaRLEC|57.62±8.30|61.49±4.06|59.49±5.89|60.87±3.89|\n|CUTS+|$\\underline{68.42±7.26}$|63.61±4.83|$\\underline{65.92±4.91}$|$\\underline{65.47±5.51}$|\n|BrainEC-LLM|**72.41±8.80**|**67.43±5.11**|**66.60±5.18**|**67.56±5.47**|\n\n**Table2: Classification results on ADHD dataset using SVM classifier.**\n|Methods|Precision(%)↑|Recall(%)↑|F1(%)↑|Accuracy(%)↑|\n|-|-|-|-|-|\n|spDCM|69.15±8.60|63.84±5.31|$\\underline{66.38±6.56}$|$\\underline{66.73±5.97}$|\n|lsGC|**70.87±5.62**|60.95±5.24|65.53±5.42|64.23±5.33|\n|ACOCTE|63.94±4.17|62.13±2.91|62.75±3.42|61.29±3.41|\n|RL-EC|61.86±4.88|60.42±3.43|61.13±4.02|59.72±3.78|\n|CR-VAE|66.39±7.71|63.84±4.42|64.04±5.61|64.61±4.84|\n|MetaCAE|64.59±7.92|63.93±4.14|64.20±5.43|64.31±4.97|\n|MetaRLEC|63.72±8.23|62.80±4.82|63.21±5.98|64.39±4.83|\n|CUTS+|66.50±7.52|$\\underline{65.62±5.32}$|66.01±4.76|65.56±4.25|\n|BrainEC-LLM|$\\underline{67.45±6.23}$|**72.38±4.91**|**69.82±5.57**|**67.87±5.02**|\n\n**W6**\n\nWe have already included a detailed analysis of the computational costs of BrainEC-LLM in **Appendix D.5**, where both the theoretical time complexity and experimental results are provided.\n\n**W7**\n\nFollowing your suggestion, we conducted additional experiments with a refined search around the optimal region. Since the best performance occurs at $η$=0.5 in Figure 5, we performed a fine-grained search from **$η$=0.4 to $η$=0.6 with step 0.02**. The  results are presented in Table 4.\n\nThe refined search shows that optimal performance is consistently achieved at $\\eta\\in${0.48,0.50,0.52} across all metrics. **The choice of $η$=0.5 is particularly intuitive given the threshold formulation: $\\theta=min(|A|)+ \\eta\\times(max(|A|)-min(|A|))$** (more details in Appendix D.3). **When $η$=0.5, the threshold is set to the midpoint between the minimum and maximum absolute values in the adjacency matrix, which represents a natural balance point for binary classification.**\n\n**Table 4: Soft threshold hyperparameter analysis on Smith dataset.**\n|$\\eta$|0.40|0.42|0.44|0.46|0.48|0.50|0.52|0.54|0.56|0.58|0.60|\n|-|-|-|-|-|-|-|-|-|-|-|-|\n|Precision↑|0.51±0.05|0.51±0.05|0.51±0.05|0.51±0.05|**0.57±0.06**|**0.57±0.06**|**0.57±0.06**|0.52±0.05|0.43±0.03|0.43±0.03|0.43±0.03|\n|Recall↑|**0.80±0.15**|**0.80±0.15**|**0.80±0.15**|**0.80±0.15**|**0.80±0.15**|**0.80±0.15**|**0.80±0.15**|0.60±0.12|0.41±0.15|0.41±0.15|0.38±0.15|\n|F1↑|0.62±0.12|0.62±0.12|0.62±0.12|0.62±0.12|**0.67±0.11**|**0.67±0.11**|**0.67±0.11**|0.55±0.12|0.40±0.15|0.40±0.15|0.40±0.15|\n|Accuracy↑|0.78±0.05|0.78±0.05|0.78±0.05|0.78±0.05|**0.84±0.06**|**0.84±0.06**|**0.84±0.06**|0.80±0.05|0.76±0.06|0.76±0.06|0.76±0.06|\n|SHD↓|5.00±1.23|5.00±1.23|5.00±1.23|5.00±1.23|**4.02±0.84**|**4.02±0.84**|**4.02±0.84**|5.00±1.23|6.22±0.91|6.22±0.91|6.22±0.91|\n\n**Questions:**\n\n**Q1**\n\nIn addition to the LLM, our model incorporates attention modules that require the input data to have consistent dimensions. However, in our datasets, both the number of time steps and the number of brain regions differ across datasets. Consequently, only the Smith and SCN5 datasets, which have the same dimensions, can be used for zero-shot experiments. We will consider addressing this limitation in future work to better demonstrate the potential of our approach for real-world applications.\n\n**Q2**\n\nSince **there is no ground truth labels for EC in real-world fMRI data**, supervised learning is difficult to apply. To address this, **we adopt an unsupervised approach that leverages the inherent autocorrelation (i.e., the brain effective connectivity network) in fMRI time series, to predict the fMRI data itself**. For relevant information on unsupervised learning, please refer to **Appendix D.6**. \n\nfMRI signals exhibit rich temporal autocorrelation and dynamic interactions between brain regions, both of which are indicative of the EC network. By training the model to reconstruct the BOLD signal (i.e., to predict fMRI data from itself), the model is encouraged to learn the causal relationships (effective connectivity) among brain regions, even without manual labels. Thus, minimizing the reconstruction loss between the predicted and actual fMRI signals enables model to discover effective connectivity in an unsupervised manner.\n\nWe appreciate your suggestion and **will include a more detailed explanation in both the loss function section and Appendix D.6 of the revised manuscript**.\n\n**Q3**\n\nAs long as the LLM module remains in the architecture, the model inherently utilizes LLM features. However, in \"w/o multiscale mixing\" ablation, the LLM output does not undergo multiscale decomposition, thus lacking LLM multiscale representations but still with LLM features. Typically, LLM features are processed through Multiscale Reconstruction Mixing module to generate both the brain EC network and predicted fMRI data. W/o multiscale mixing removes the decomposition and mixing components from the multiscale processing pipeline while preserving the instance normalization and cross attention. Consequently, due to the absence of multiscale features, the $L_{csc}$ loss term is also eliminated from the training objective.\n\n**We deeply appreciative your guidance. The specific details and experimental strategies you proposed will be included in our upcoming revision.**"}]

Author rebuttal:

Meta-review: The paper presents BrainEC-LLM, a method for estimating brain effective connectivity (EC) from fMRI data using fine-tuned large language models (LLMs). Unlike prior approaches that rely on fixed queries without model adaptation, BrainEC-LLM introduces a multiscale decomposition mixing (MDM) module to capture both short- and long-term temporal patterns in fMRI signals. It also uses cross-attention with pre-trained word embeddings to align neural time series with language representations, helping bridge the cross-modal gap. The model is evaluated on six datasets, including simulated and real fMRI data, and shows strong performance in EC estimation and downstream tasks like ASD classification. Ablation studies confirm the importance of key components, and the authors address reviewer concerns in the rebuttal. I would recommend it for acceptance.
Decision: Accept (poster)