Paper:
Learning Neural Representations of Human Cognition across Many fMRI Studies
Cognitive neuroscience is enjoying rapid increase in extensive public brain-imaging datasets. It opens the door to large-scale statistical models. Finding a unified perspective for all available data calls for scalable and automated solutions to an old challenge: how to aggregate heterogeneous information on brain function into a universal cognitive system that relates mental operations/cognitive processes/psychological tasks to brain networks? We cast this challenge in a machine-learning approach to predict conditions from statistical brain maps across different studies. For this, we leverage multi-task learning and multi-scale dimension reduction to learn low-dimensional representations of brain images that carry cognitive information and can be robustly associated with psychological stimuli. Our multi-dataset classification model achieves the best prediction performance on several large reference datasets, compared to models without cognitive-aware low-dimension representations; it brings a substantial performance boost to the analysis of small datasets, and can be introspected to identify universal template cognitive concepts.
Due to the advent of functional brain-imaging technologies, cognitive neuroscience is accumulating quantitative maps of neural activity responses to specific tasks or stimuli. A rapidly increasing number of neuroimaging studies are publicly shared (e.g., the human connectome project, HCP [1]), opening the door to applying large-scale statistical approaches [2]. Yet, it remains a major challenge to formally extract structured knowledge from heterogeneous neuroscience repositories. As stressed in [3], aggregating knowledge across cognitive neuroscience experiments is intrinsically difficult due to the diverse nature of the hypotheses and conclusions of the investigators. Cognitive neuroscience experiments aim at isolating brain effects underlying specific psychological processes: they yield statistical maps of brain activity that measure the neural responses to carefully designed stimulus. Unfortunately, neither regional brain responses nor experimental stimuli can be considered to be atomic: a given experimental stimulus recruits a spatially distributed set of brain regions [4], while each brain region is observed to react to diverse stimuli. Taking advantage of the resulting data richness to build formal models describing psychological processes requires to describe each cognitive
conclusion on a common basis for brain response and experimental study design. Uncovering atomic basis functions that capture the neural building blocks underlying cognitive processes is therefore a primary goal of neuroscience [5], for which we propose a new data-driven approach.
Several statistical approaches have been proposed to tackle the problem of knowledge aggregation in functional imaging. A first set of approaches relies on coordinate-based meta-analysis to define robust neural correlates of cognitive processes: those are extracted from the descriptions of experimentsbased on categories defined by text mining [6] or expert [7]-and correlated with brain coordinates related to these experiments. Although quantitative meta-analysis techniques provide useful summaries of the existing literature, they are hindered by label noise in the experiment descriptions, and weak information on brain activation as the maps are reduced to a few coordinates [8]. A second, more recent set of approaches models directly brain maps across studies, either focusing on studies on similar cognitive processes [9], or tackling the entire scope of cognition [10,11]. Decoding, i.e. predicting the cognitive process from brain activity, across many different studies touching different cognitive questions is a key goal for cognitive neuroimaging as it provides a principled answer to reverse inference [12]. However, a major roadblock to scaling this approach is the necessity to label cognitive tasks across studies in a rich but consistent way, e.g., building an ontology [13].
We follow a more automated approach and cast dataset accumulation into a multi-task learning problem: our model is trained to decode simultaneously different datasets, using a shared architecture. Machine-learning techniques can indeed learn universal representations of inputs that give good performance in multiple supervised problems [14,15]. They have been successful, especially with the development of deep neural network [see, e.g., 16], in sharing representations and transferring knowledge from one dataset prediction model to another (e.g., in computer-vision [17] and audioprocessing [18]). A popular approach is to simultaneously learn to represent the inputs of the different datasets in a low-dimensional space and to predict the outputs from the low-dimensional representatives. Using very deep model architectures in functional MRI is currently thwarted by the signal-to-noise ratio of the available recordings and the relative little size of datasets [19] compared to computer vision and text corpora. Yet, we show that multi-dataset representation learning is a fertile ground for identifying cognitive systems with predictive power for mental operations.
Contribution. We introduce a new model architecture dedicated to multi-dataset classification, that performs two successive linear dimension reductions of the input statistical brain images and predicts psychological conditions from a learned low-dimensional representation of these images, linked to cognitive processes. In contrast to previous ontology-based approaches, imposing a structure across different cognitive experiments is not needed in our model: the representation of brain images is learned using the raw set of experimental conditions for each dataset. To our knowledge, this work is the first to propose knowledge aggregation and transfer learning in between functional MRI studies with such modest level of supervision. We demonstrate the performance of our model on several openly accessible and rich reference datasets in the brain-imaging domain. The different aspects of its architecture bring a substantial increase in out-of-sample accuracy compared to models that forgo learning a cognitive-aware low-dimensional representation of brain maps. Our model remains simple enough to be interpretable: it can be collapsed into a collection of classification maps, while the space of low-dimensional representatives can be explored to uncover a set of meaningful latent components.

Review:
This paper introduces a framework to analyze heterogeneous fMRI datasets combining multi-task supervised learning with multi-scale dictionary learning. The result shows that classification of task conditions in a target dataset can be improved by integrating the large-scale database of humman connectome project (HCP) using their method.
The proposed entire framework looks fairly reasonable to me. However, the method contains many different technical ideas and each of them is not necessarily well validated. The technical ideas include at least the following: 1) use of resting state data instead of task data for training the first stage; 2) use of multi-scale spatial dictionary combining multiple runs of sparse dictionary learning; 3) using Dropout for regularization; and 4) adding HCP data to improve classification on another target dataset. In my view, current results only support point 4. The following comments are related to the other points.
In the experiment (Sec2.2), the comparison between single and multi-scale dictionaries (methods 2 and 3) seems to be made for different dictionary sizes (256 and 336). However, I think they should be balanced for a fair comarison (for instance, learn 336 atoms at once for the non-multi-scale dictionary), since currently the effects of having different dimensionalities and different spatial scales cannot be dissociated. I would also suggest comparing with simply using PCA to reduce to the same dimensionality, which may clarify the advantage of using localized features.
Moreover, the methods 1-3 use L2-regularization but 4-5 uses Dropout, which may also not be a fair comparison. What happens if you use L2-regualarization for methods 4-5, or otherwise Dropout for 1-3?
I also wondered what happens if the spatial dictionary is trained based on the task data rather than resting-state data. The use of rest data for training sounds reasonable for the goal of "uncovering atomic bases" (Introduction) but it would be necessary to compare the two cases in terms of both test accuracy and interpretability. In particular, can similar spatial maps in Figs4,5 be obtained even without resting-state decomposition?
Other minor comments:
- Eq1,3: what does the objective actually mean, when taking expectation with respect to M? Is it correct that the objective is minimized rather than maximized? - Eq3: "log" seems to be dropped.
- Fig2: what are the chance levels for each dataset? No result by "full multinomial" for LA5c.
- Line235: what does "base condition" mean?