Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents a graph neural network architecture for analyzing dynamic brain imaging data. The proposed approach uses both structural connectivity (via DWI) and functional connectivity (via fMRI) to setup the graph edge information, while the fMRI signals are used to define node features. The proposed network alternates between analyzing temporal information via gated temporal convolutional network layers and spatial information via GNN layers. A multi-resolution soft cluster smoothing is applied as the pooling operation in the GNN. Attribution of structural and functional imaging inputs is also explored using integrated gradients interpretation method. The proposed method is tested on a multi-task fMRI dataset and demonstrated better performance in both ablation studies and comparison to state-of-the-art GNN methods.
main_review: Strengths:

- The proposed method combines structural and functional brain information, which are often assessed separately.

- The proposed network includes components for both temporal and spatial processing.

- The proposed network utilizes an individualized graph structure (connections) for each sample, rather than defining one adjacency matrix for all like most standard approaches. The network can handle this in part due to the soft clustering and keeping the same number of nodes at each GNN level.

- The paper is fairly clearly presented.

Weaknesses:

- The "strided non-causal TCN" I feel is a bit of a misleading name - when I think temporal convolutional network I imagine it needs to have the causal aspect, as it's trying to ensure appropriate analysis of temporal information. The "non-causal" TCN is just a strided convolution on 1D data that happens to be in time dimension. Also, it would be helpful if the authors briefly include the information/equation for how the gating mechanism works.

- For the experiments, the partition of the dataset appears to be by scan, not by subject. Since there are 1940 scans, 6 different scan tasks and only 56 subjects, my understanding is each subject may have undergone multiple session of the same task. Thus, it does not seem fair to split by scan, but rather should perform a patient-wise split for the validation method to get a better estimate of generalization performance.

- While the total number of scans is reasonably large for neuroscience study, given the smaller number of individual subjects, a single split validation study does not seem appropriate - a cross-validation framework would be more convincing. Furthermore, as there is only a single split, there is no variance information so it is difficult to assess if there are true differences between methods that perform similarly.

- The extremely large gap in empirical performance between the standard GNN models and the proposed method seems rather jarring (up to ~45% difference in accuracy). This makes me wonder whether the other models are not being appropriately tuned, or are given appropriate depth? I did not find any details about the baseline models, e.g., how many layers are used? For the proposed method it seems the authors would have used 4 layers, did they match the same number for baseline models? Also, how are the graphs constructed for the baseline model - are the same connections used or did the authors vary connections per sample like in their approach? And are the connections based on structural or functional data? In general, how would the authors explain the large difference in performance? It could also be interesting to compare to other GNN methods that were developed specifically for functional brain data analysis, e.g. Li et al (as cited in the paper), or Li et al, Braingnn: Interpretable brain graph neural network for fmri analysis, bioarxiv 2020.

- The authors compare to GNN-only baselines, but not TCN-only baseline. I think it would make sense to compare to TCN models given (over) half of their network is based on TCN, and also given the large gap in performance using GNN only models; this may point to the feature learning in the TCN layers as being very important compared to the GNN component for classification learning.

- How is the cluster number c chosen? The authors note that c decreases with deeper layers, but there is no mention as to the values used or how they are determined. Considering the authors claim this cluster smoothing is essential, it is important to explain how related parameter c is set. Does changing c then greatly affect the results?

Minor comments:
- For the variable for structural connectivity SC, the use of 2 letters for 1 variable is confusing when reading the equations - considering changing to 1 letter.
- More detail on the number of each type of task scan and how many scans per subject would be welcome (eg added to appendix).


summary_of_the_review: My recommendation is based on the new proposed network for combining functional/structural brain data analysis in a meaningful way for processing temporal and spatial information, and allowing individualized graph structures to be learned/used within a GNN model. While the results appear good, they appear somehow incredulously higher than standard GNN methods, and important missing information about experimental setup and empirical comparisons further dampen my enthusiasm for the work.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a deep learning method for temporal data on graph nodes, specifically designed for brain imaging data. It can be deployed for classification of data where the time-varying data and graphs are individual specific, and pinpoint subgraph structure where group-specific changes occur. 
main_review: Strength: 
- Clearly written and easy to follow.
- Practically useful framework for fMRI data analysis. 
- Extensive experiment that validates the proposed framework. 

Weakness:
- Novelty of the paper is quite unclear, especially in the introduction. Why is analyzing time varying signals on graphs challenging? What are the problems that recent methods are facing, and how are the authors are solving them?
- Why is a normalized adjacency matrix used? Any technical / clinical motivation?
- It is not clear why having individual adjacency matrix implies a latent graph cannot be learnt. A_adp is a functional graph derived from node signals; why can't each adjacency matrix can be deployed here? This is important as the authors use both AHW and A_adpHW later in the framework. 
- From anatomical perspective, relationships in local (structural) connections make more sense than those among far ROIs. What is the rational that it can be beneficial (rather than simply referencing Ying 2018)?
- In the experiment, did the authors divide the given dataset in to a single set of partitions? Was cross-validation adopted?
- Temporal data varies in their length per subject. How was this problem dealt? 
- During the method comparisons, I am not sure if it is valid to set the same epoch limit to derive evaluation measures as each model has different complexity. 
- Is the outcome properly validated? There are several labels according to memory, attention, dot prove and so on... are the label-specific variations detected from this framework really associated with those tasks? 
summary_of_the_review: This paper tackles an important problem in neurosience. 
Clinical motivation and the overall pipeline make sense, and extensive efforts were put on evaluation of the ideas and framework. 
The paper is mostly clear, but I do have some concerns as mentioned in the Main Review above which can be very critical. 
I am willing to change my score once I go through the rebuttal and other reviews if needed.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a Graph Neural Network model to estimate latent dynamics in the human brain using functional Magnetic Resonance Imaging (fMRI) and Diffusion Weighted Imaging (DWI) while performing a classification task. The model consists of four parts 1) estimating each sample's adjacency matrix by learning a shared projection matrix 2) gated temporal convolutional neural network for learning information from time-series data 3) graph neural network to get embeddings of temporal and spatial information combined by 4) multi-resolution inner cluster smoothing. The authors use integrated gradients for estimating important features for the classification task helping with model interpretation. The authors show that the proposed method can perform better in the prediction task than GCN, GAT V2, etc. On the real dataset, it is shown that the method can capture temporal and spatial heterogeneity and provide regions important for the presented classification task.
main_review: I found the approach introduced in the paper interesting, and the motivation is clear. I enjoyed that the proposed approach aims to extract latent dynamics information, thus capturing temporal heterogeneity, which can be very useful in neuroscience as well as disease prediction research. While this contribution shows that the proposed method can generate better prediction results, I see several issues with it as it stands. The rationale for combining various existing approaches as proposed in the paper should be clearly justified emphasizing the novelty. The presentation of the manuscript can be improved, and a few details seem to be missing. I have some concerns, which are listed below.

1) Existing approaches to estimate dynamic connectivity and approaches estimating spatiotemporal patterns in the fMRI data [1,2,3,4] are not mentioned. Authors could comment on what existing approaches (based on linear modeling) lack, relevance/irrelevance of the approaches in the given context, and compare the results.
2) The model is not straightforward to understand in one go. Paragraphs in Section 2.2 could be rewritten such that they are more connected and give a better picture of the model. 
3) To validate the method and illustrate the motivation, please consider including some simulation studies on synthetic datasets. Since the method is intended to capture latent dynamics, the authors could borrow simulation settings from the related literature.
4) It is mentioned that the F1 metric is used because of the imbalanced dataset, but the information about the imbalanced dataset is missing for understanding the data. Also, the number of frames for each task is missing. 
5) It seems that the experiments are performed only for one set of train, validation, and test set. Why are multiple runs of k-fold cross validation not used here? 
6) Not sure what to make of the qualitative analysis in Figures 6, 12, and 13. It would be helpful to discuss the importance of "important" nodes in the context of these tasks rather than just stating the region name. Also, the authors should describe the regions mentioned in the main text. The claim of the paper is that the proposed method can find good representations but the authors have not explained the importance of these subnetworks in the context of the task presented. Even a small discussion would be helpful.
7) We notice that signal-important ROIs are not necessarily the same as connection-important ROIs:? What does it mean from the point of view of biological relevance? How to use these remarks as biomarkers?
8) How reproducible are these important regions extracted? Since the complete problem is non-convex, depending on initialization, will the end result vary? Since these regions would be used in further downstream analysis, they would need to be highly reproducible.   
9) Authors mention, "We also find using adaptive adjacency matrices, and inner cluster smoothing can stabilize training, making the model less prone to overfitting and achieving close-to-best performance over a larger range of hyperparameters." Can you show results to support the claim made?
10) Authors mention, "Important brain regions obtained from $ATTR_A$ mostly comply with the previous literature." What are the important brain regions? Can you cite literature to back up this claim?
11) Is there a reason that $A_{iadp}$ is sparse for each sample? Is this expected biologically or due to the way in which the model is defined? Would it be helpful to use sparsity constraints on $A$? This might be helpful in controlling the sparsity and in the interpretability.
12) Can you comment on the biological relevance of the brain regions found in $ATTR_X$ 
13) How are $h_{adp}$ in eq 1 and $K$ in eq 2 set?
14) X-axis and Y-axis ticks size can be increased to improve the visibility in Fig 3, Fig 5, Fig 10 and Fig 11.
15) What is $\tilde{D}$ and $v$ in section 2.1? Is $\tilde{D}$ a diagonal matrix where $v,v$ entry is equal to the given sum?
16) Published versions of the referenced arxiv papers should be cited.

I am happy to raise my score if the above concerns are addressed.

[1] Li, Lingge, et al. "Modeling dynamic functional connectivity with latent factor Gaussian processes." Advances in neural information processing systems 32 (2019): 8263-8273.

[2] Taghia, Jalil, et al. "Bayesian switching factor analysis for estimating time-varying functional connectivity in fMRI." Neuroimage 155 (2017): 271-290.

[3] Zhang, Gemeng, et al. "Estimating dynamic functional brain connectivity with a sparse hidden Markov model." IEEE transactions on medical imaging 39.2 (2019): 488-498.

[4] Bhinge, Suchita, et al. "Extraction of time-varying spatiotemporal networks using parameter-tuned constrained IVA." IEEE transactions on medical imaging 38.7 (2019): 1715-1725.

summary_of_the_review: Based on the above assessments, I suggest rejection of this paper. The current experimental evaluations and discussion need to be stronger to be more convincing, and the presentation can be improved.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a method of graph neural network to jointly models dynamic functional signals and structural connectivities, to learn a good deep representation of brain dynamics. In this article, the author proposes some improved methods for graph neural networks, such as learning sample-level latent graph structures, temporal convolutional network and multi-resolution inner cluster smoothing methods to better learn insightful interpretations of brain dynamics. In the experiment, the author verified the effectiveness of the method in this paper on the fMRI signal data.
main_review: Pros:
1. The method in this article has made many improvements to graph neural networks. These improvements are to capture the spatial information and temporal information in brain dynamics. These improvements have also been verified in experiments to learn better temporal and spatial importance sanity.

2. The author uses the method of graph neural network to learn spatial information and temporal information. This new proposal may be extended to different fields. Not only does it have the effect of improving the brain image, but it may also be useful for general computer vision in the future. . So the author's innovation in the method is quite meaningful.

 Cons:
1. In the experiment, the author compared with other different graph neural network methods. In table 1, the accuracy of this method is about 20% higher than other methods. These results are quite confusing to me, and the GCN method only has 41% accuracy. I think the baseline method compared by the author may be too poor. It is more appropriate to compare it with other methods that are not graph neural networks. The ones in Table 1 The result is still unable to prove the effectiveness of the method.

2. The improvement of several methods on graph neural network in this paper is mainly based on some previous work. But the main contribution is to make some methodological adjustments for brain imaging, which greatly reduces the innovation point of this article. Therefore, the overall innovation of the article still needs to be improved.
summary_of_the_review: This article improves the network method of graph neural network to learn a better representation for brain dynamics in the application of brain images. I prefer to reject this article because this article has made some small improvements on GNN, which is more like to combine them in different ways to improve the overall effect. In addition, the baseline effects compared by the author in the experiment are relatively weak, and the effectiveness of the method cannot be demonstrated.
review_title: null

Meta-review: This paper proposes a Graph Neural Network model to estimate latent dynamics in the human brain using functional Magnetic Resonance Imaging (fMRI) and Diffusion Weighted Imaging (DWI). The representation is tested on a classification task. While reviewers acknowledge the importance of this application, various concerns have been raised and partially addressed. The work focuses on graph deep learning and offers limited evidence of its superiority over more traditional ML or non graph based deep learning. Besides the methodological novelty is unclearly argued, which is not ideal for the audience of a conference like ICLR.

For all these reasons, this work cannot be endorsed for publication at ICLR 2022.
Decision: Reject