  Objective: Multi-modal functional magnetic resonance imaging (fMRI) can be
used to make predictions about individual behavioral and cognitive traits based
on brain connectivity networks. Methods: To take advantage of complementary
information from multi-modal fMRI, we propose an interpretable multi-modal
graph convolutional network (MGCN) model, incorporating the fMRI time series
and the functional connectivity (FC) between each pair of brain regions.
Specifically, our model learns a graph embedding from individual brain networks
derived from multi-modal data. A manifold-based regularization term is then
enforced to consider the relationships of subjects both within and between
modalities. Furthermore, we propose the gradient-weighted regression activation
mapping (Grad-RAM) and the edge mask learning to interpret the model, which is
used to identify significant cognition-related biomarkers. Results: We validate
our MGCN model on the Philadelphia Neurodevelopmental Cohort to predict
individual wide range achievement test (WRAT) score. Our model obtains superior
predictive performance over GCN with a single modality and other competing
approaches. The identified biomarkers are cross-validated from different
approaches. Conclusion and Significance: This paper develops a new
interpretable graph deep learning framework for cognitive ability prediction,
with the potential to overcome the limitations of several current data-fusion
models. The results demonstrate the power of MGCN in analyzing multi-modal fMRI
and discovering significant biomarkers for human brain studies.
