Using neural networks for a functional connectivity classification of fMRI data | by Gelana Tostaeva | Towards Data Science

In this tutorial you will:
explore a neuroimaging dataset from nilearn;
perform a canonical independent component analysis in nilearn;
extract functional connectivity coefficients between regions determined by the component analysis;
use the coefficients for a classification multilayer perceptron model using Keras.
All the necessary code will be provided in this guide. For the full version, please see this GitHub page.
Prerequisites: This tutorial uses Python, Nilearn, and Keras. You do not need to have used Nilearn before, but you should have some experience with using Python and Keras. If you have never made a neural network before, this tutorial might be difficult to follow towards the end when we build a model, but feel free to stick around for the Nilearn part.
We will explore a neural network approach to analyzing functional connectivity-based data on attention deficit hyperactivity disorder (ADHD). Functional connectivity shows how brain regions connect with one another and make up functional networks. As such, it might hold insights into how the brain communicates
Our approach does not directly rely on neuroimaging scans and, instead, makes use of vectorized functional connectivity measures. This provides a computationally light way of getting practice with using machine learning for functional connectivity analysis.
Similar approaches using more complicated neural network models have been used for classification, in the context of brain disorders in particular. However, these have proven to be challenging due to the numerous features in the data and validation difficulty (see Du et al., 2018 for a review). Our own analysis also shows less than optimal model accuracy, but the purpose of this tutorial is to provide a guide on how to get started. You are encouraged to try out your own neural network in the end!
Nilearn, which is a Python module for neuroimaging data we will be using, has a variety of preprocessed datasets you can easily download with a built-in function:
from nilearn import datasetsnum = 40adhd_data = datasets.fetch_adhd(n_subjects=num)
We will use the ADHD-200 publicly accessible dataset composed of resting-state fMRI and anatomical data collected from multiple research centers. Nilearn has data for only 40 subjects, so we load all of them.
We can inspect the dataset by looking at the .keys() — we see that there are 4 types of information. “func” features the paths to the rs-fMRI data images; “confounds” are the CSV files containing the nuisance variables (confounds) we want to be aware of as not to affect our analysis; “phenotypic” provides explanations for the preprocessing steps; “description” is, well, description of the dataset.
The data were collected to increase understanding of the neural correlates of ADHD. If you want, you can learn more about the original dataset here. For the purposes of this tutorial, you only need to know that the dataset features both typically developing individuals (“controls”) and those with diagnosed ADHD (“treatments”).
We will extract the functional connectivity coefficients from this dataset to classify whether a given subject is a “control” or “treatment”. We will do so by first using an independent component analysis from nilearn and then use it to extract functional connectivity coefficients. Finally, we will build a neural network using these coefficients to discriminate “controls” from “treatments”.
Independent component analysis (ICA) is commonly used to assess functional connectivity. Nilearn has a method for group-level ICA (CanICA) which allows for control over single subject variability, especially given that we are interested in functional networks.
We use Nilearn’s built-in function and get nice visualizations of what we are working with. We choose to use a 20-component decomposition based on the standards provided in the Nilearn documentation for the chosen dataset. We get the independent components using masker_.inverse_transform which we then plot using the Nilearn’s plotting options for both the statistical and probabilistic atlas maps. We need the statistical one to plot the default mode network (DMN) specifically — the plot_stat_map function allows plotting cuts of this region of interest; the probabilistic one simply uses all the components yielded from the decomposition and layers them on top of the default anatomical brain image.
from nilearn import decompositioncanica = decomposition.CanICA(n_components=20, mask_strategy=’background’)canica.fit(func)#Retrieving the componentscomponents = canica.components_#Using a masker to project into the 3D spacecomponents_img = canica.masker_.inverse_transform(components)#Plotting the default mode network (DMN) without region extractionplotting.plot_stat_map(image.index_img(components_img, 9), title='DMN')plotting.show()#Plotting all the componentsplotting.plot_prob_atlas(components_img, title='All ICA components')plotting.show()
Our component decomposition is hard to interpret conclusively (essentially, we are seeing different brain regions in the fMRI data), but we can use it as a filter to extract the regions we are interested in. We do so calling the NiftiMapsMasker function from Nilearn to “summarize” the brain signals we obtained using ICA. Once we have that, we transform the extracted data to time-series by using the fit_transform method.
We then use everything we know about the dataset (“func”, “confounds”, and “phenotypic” files) to get the information we need, including whether the subject is “treatment” or “control” and their associated data collection location (site).
#Using a filter to extract the regions time series from nilearn import input_datamasker = input_data.NiftiMapsMasker(components_img, smoothing_fwhm=6, standardize=False, detrend=True, t_r=2.5, low_pass=0.1, high_pass=0.01)#Computing the regions signals and extracting the phenotypic information of interestsubjects = []adhds = []sites = []labels = []for func_file, confound_file, phenotypic in zip( adhd_data.func, adhd_data.confounds, adhd_data.phenotypic): time_series = masker.fit_transform(func_file, confounds=confound_file) subjects.append(time_series) is_adhd = phenotypic[‘adhd’] if is_adhd == 1: adhds.append(time_series)  sites.append(phenotypic[‘site’]) labels.append(phenotypic[‘adhd’])
Thus far, we used a CanICA to get components that we needed to determine the regions of interest. The last thing to do before we can build our neural network model is to get functional connectivity coefficients. For this, we need to look at the functional connectivity between the regions of interest we extracted. We considered three different kinds of functional connectivity and determined correlation to be the most accurate. You can find how we did this in the full code.
Correlation simply determines the marginal connectivity between pairwise regions of interest. Nilearn has a built-in method for computing the correlation matrices, the ConnectivityMeasure function. We only need to specify the kind of functional connectivity we are interested in and then fit the time-series data we extracted in the previous step.
from nilearn.connectome import ConnectivityMeasurecorrelation_measure = ConnectivityMeasure(kind=’correlation’)correlation_matrices = correlation_measure.fit_transform(subjects)for i in range(40): plt.figure(figsize=(8,6)) plt.imshow(correlation_matrices[i], vmax=.20, vmin=-.20, cmap=’RdBu_r’) plt.colorbar() plt.title(‘Connectivity matrix of subject {} with label {}’.format(i, labels[i]))
We now have our connectivity matrices for all subjects, but let’s see what the average connectivity across all looks like. We split the matrices into those of treatment versus control subjects for comparison. This comparison is what our neural network model will use for classification.
#Separating the correlation matrices between treatment and control subjectsadhd_correlations = []control_correlations = []for i in range(40):    if labels[i] == 1:        adhd_correlations.append(correlation_matrices[i])    else:        control_correlations.append(correlation_matrices[i])#Getting the mean correlation matrix across all treatment subjectsmean_correlations_adhd = np.mean(adhd_correlations, axis=0).reshape(time_series.shape[-1],                                                          time_series.shape[-1])#Getting the mean correlation matrix across all control subjectsmean_correlations_control = np.mean(control_correlations, axis=0).reshape(time_series.shape[-1],                                                          time_series.shape[-1])#Visualizing the mean correlationplotting.plot_matrix(mean_correlations_adhd, vmax=1, vmin=-1,                               colorbar=True, title='Correlation between 20 regions for ADHD')plotting.plot_matrix(mean_correlations_control, vmax=1, vmin=-1,                               colorbar=True, title='Correlation between 20 regions for controls')
We can see that the connections are not particularly strong for either group (the diagonal line can be ignored as it shows correlation with itself and, thus, always equals to 1). To better visualize the connections and the differences, we can project these back onto the brain.
#Getting the center coordinates from the component decomposition to use as atlas labelscoords = plotting.find_probabilistic_atlas_cut_coords(components_img)#Plotting the connectome with 80% edge strength in the connectivityplotting.plot_connectome(mean_correlations_adhd, coords,                         edge_threshold="80%", title='Correlation between 20 regions for ADHD')plotting.plot_connectome(mean_correlations_control, coords,                         edge_threshold="80%", title='Correlation between 20 regions for controls')plotting.show()
This gives us a nice connectome, the brain map of all the connections for 20 regions we are looking at.
The resulting ADHD connections do not seem as dense, compared to the control ones, which might be related to the notion of reduced functional connectivity associated with ADHD (Yang et al., 2011). In line with some of the previous research (Tomasi & Volkow, 2012), for ADHA subjects, we notice fewer connections in the superior parietal cortex (the upper right part in the first coronal graph) which is thought to be implicated in attention. There also seem to be fewer connections in the DMN, which we visualized before, a network active during rest and associated with the “self” — one that is suggested to be altered in ADHD (Mowinckel et al., 2017). These, albeit small differences, suggest that it should not be impossible to classify between treatments and controls using correlation matrices in our neural network model.
If you would like to see an interactive visualization of the connectome, you can run the line below. Otherwise, we are ready to move on to the modeling!
#Creating the interactive visualizationview = plotting.view_connectome(mean_correlations, coords, edge_threshold='80%')#To display in the cell belowview#To display in a different tabview.open_in_browser()
Now that we have our correlation matrices providing a vectorized measure of functional connectivity, we can use these as the input data for our neural network.
Before we build our model, we should split the data for training (70%) and testing (30%):
from sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split(correlation_matrices, labels, test_size=0.3)
Our neural network can be anything from one layer to five. After playing with a few different architectures, we settled on a Sequential model made up of four Dense layers. This tutorial assumes you know what these mean, so we will not go into all the details and give a brief overview of the overall architecture instead. If you need a refresher on the topic, this blog is a good start.
import kerasfrom keras.models import Sequentialfrom keras.layers import Densefrom keras.optimizers import Adamclassifier = Sequential()#First Hidden Layerclassifier.add(Dense(32, activation=’tanh’, kernel_initializer=’random_normal’, input_shape=connectivity_biomarkers[‘correlation’].shape[1:]))#Second Hidden Layerclassifier.add(Dense(16, activation=’relu’, kernel_initializer=’random_normal’))#Third Hidden Layerclassifier.add(Dense(16, activation=’relu’, kernel_initializer=’random_normal’))#Output Layerclassifier.add(Dense(1, activation=’sigmoid’, kernel_initializer=’random_normal’))#Compiling the modelclassifier.compile(optimizer = Adam(lr =.0001),loss='binary_crossentropy', metrics =['accuracy'])#Fitting the modelclassifier.fit(np.array(X_train),np.array(y_train), batch_size=32, epochs=100)
And this is it! We built a simple neural network. Let’s recap the architecture of our model:
We use the Sequential model so that we can simply build layers on top of one another;
We choose Dense layers which are simple layers in a neural network — you can think of these as linear models taking multiple inputs and producing single output. We use 4 of these as we are dealing with functions that are not necessarily linearly separable, but, since we are essentially only interested in a binary classification, you don’t have to use 4, and might get away with fewer — this is what seemed to work best in terms of model’s predictive power. For each layer, we specific different numbers of nodes (32, 16, 16, 1) which is also mostly trivial in this example model. The only rule of thumb is to start with the number smaller or equal to the length of your input; in our case, we have 40 matrices, so 32 seems appropriate. The final output layer should be related to the number of output categories. Since this is a binary classification problem, we need to use 1.
For activation functions (which basically do the input-output signal conversion), we use Tanh, ReLu, and Sigmoid. Tanh and Sigmoid are used for the first and last layers, respectively, and not in the hidden ones. This is because both can be characterized by the vanishing gradient problem meaning they would output a zero gradient if the input is higher (which we are not sure of, so better be safe). We could have just used ReLu functions which overcome this problem due to its simplicity (it gives x as output if x is positive and 0 if otherwise), but these are better avoided in non-hidden layers because they might product no gradient, or “dead” neurons. The current architecture (Tanh, ReLu, ReLu, Sigmoid) seemed to, again, result in most optimal evaluation metrics like accuracy;
Finally, we use an Adam optimizer and a binary cross-entropy loss function. Adam is considered to be a good option, especially for noisy data. Binary cross-entropy is a usual choice for classification problems since it is independent for each class and ensures that one output vector is unaffected by other components.
Now, there’s only one thing left to do — see how our model performs. We will use accuracy as our main metric as it represents the proportion of correct classifications.
Let’s start with the training set:
eval_model=classifier.evaluate(np.array(X_train), np.array(y_train))eval_model
Hooray, the accuracy on our training data is 1. Onto the scary part of testing data.
y_pred=classifier.predict(X_test,batch_size=32)y_pred =(y_pred>0.5)from sklearn.metrics import confusion_matrix, classification_reportcm = confusion_matrix(y_test, y_pred)print(cm)cr = classification_report(y_test, y_pred)print(cr)
The overall accuracy of our classification is 75% which is not terrible but could be better. We also only got 2 false negatives and 1 false positive. It’s now up to you to play with the simple model you build in this step-by-step guide! A disclaimer that the model in the corresponding Jupyter notebook has a much lower accuracy than reported here likely due to random processes — all the more reason to experiment and come up with a better model.
This tutorial explored how functional connectivity data can be used to screen for ADHD. We created a neural network based on functional connectivity coefficients for different regions to do so. As we have seen through the component analysis and connectome plots, the differences between treatment and control samples are not prominent which could explain the lower accuracy of our screening. More work with larger datasets should be done to see if there is a more consistent pattern within functional connectivity in ADHD. It could be also interesting to examine a higher number of regions and components in future analyses.
Hopefully, this has given you some idea of how you can use machine learning for functional connectivity analysis of fMRI data. We (conveniently) used preprocessed data, but if you want to learn more about the preprocessing step of fMRI data analysis, check out my other tutorial here. Until next time!
Abraham, A., Pedregosa, F., Eickenberg, M., Gervais, P., Mueller, A., Kossaifi, J., ... & Varoquaux, G. (2014). Machine learning for neuroimaging with scikit-learn. Frontiers in neuroinformatics, 8, 14.
Du, Y., Fu, Z., & Calhoun, V. D. (2018). Classification and prediction of brain disorders using functional connectivity: promising but challenging. Frontiers in neuroscience, 12.
Mowinckel, A. M., Alnæs, D., Pedersen, M. L., Ziegler, S., Fredriksen, M., Kaufmann, T., ... & Biele, G. (2017). Increased default-mode variability is related to reduced task-performance and is evident in adults with ADHD. NeuroImage: Clinical, 16, 369–382.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., ... & Vanderplas, J. (2011). Scikit-learn: Machine learning in Python. Journal of machine learning research, 12(Oct), 2825–2830.
Tomasi, D., & Volkow, N. D. (2012). Abnormal functional connectivity in children with attention-deficit/hyperactivity disorder. Biological psychiatry, 71(5), 443–450.
Varoquaux, G., Sadaghiani, S., Pinel, P., Kleinschmidt, A., Poline, J. B., & Thirion, B. (2010). A group model for stable multi-subject ICA on fMRI datasets. Neuroimage, 51(1), 288–299.
Yang, H., Wu, Q. Z., Guo, L. T., Li, Q. Q., Long, X. Y., Huang, X. Q., ... & Gong, Q. Y. (2011). Abnormal spontaneous brain activity in medication-naive ADHD children: a resting state fMRI study. Neuroscience letters, 502(2), 89–93.