Paper:
Integrated accounts of behavioral and neuroimaging data using flexible recurrent neural network models
Neuroscience studies of human decision-making abilities commonly involve subjects completing a decision-making task while BOLD signals are recorded using fMRI. Hypotheses are tested about which brain regions mediate the effect of past experience, such as rewards, on future actions. One standard approach to this is model-based fMRI data analysis, in which a model is fitted to the behavioral data, i.e., a subject's choices, and then the neural data are parsed to find brain regions whose BOLD signals are related to the model's internal signals. However, the internal mechanics of such purely behavioral models are not constrained by the neural data, and therefore might miss or mischaracterize aspects of the brain. To address this limitation, we introduce a new method using recurrent neural network models that are flexible enough to be jointly fitted to the behavioral and neural data. We trained a model so that its internal states were suitably related to neural activity during the task, while at the same time its output predicted the next action a subject would execute. We then used the fitted model to create a novel visualization of the relationship between the activity in brain regions at different times following a reward and the choices the subject subsequently made. Finally, we validated our method using a previously published dataset. We found that the model was able to recover the underlying neural substrates that were discovered by explicit model engineering in the previous work, and also derived new results regarding the temporal pattern of brain activity.

Review:
In this paper the authors use a recurrent neural network model to study reward processing in human fMRI experiments. The main novelty of this approach is that it takes into account both the neural data (measured by fMRI) and the behavioral outcomes. In contrast the standard approach is to fit a computational model on the behavioral data, then use that as a regressor in a classical GLM analysis, which may find a good solution for the behavioral data but one that is different from the solution implemented in the brain. In this respect, this application stands out. Most other neural network models that are used to model fMRI data (e.g. Guclu and van Gerven, J Neuroscience 2015) do not use the fMRI data to train the neural network directly.  The paper is well-written and clearly explained. The network topology the authors use is simple, elegant and appropriate for the proposed application. The implementation is fairly careful and the proposed method to 'unroll' the temporal dynamics to find the contribution of each timepoint and brain region to the reward signal (eq 11) is a nice way to visualise the underlying dynamics of the model, although not strictly novel because I am aware of other methods that use the derivatives of the predictive model for localising the discriminative signal.   As such, I find this work to be highly original, very clearly written and of a sufficient methodological quality to be published at NIPS, contingent on a satisfactory response to the questions outlined below. I also think that the method could be adapted for other problems relatively easily so does make a significant contribution. My comments for improvement mainly relate to clarity:  I would suggest to split the figures 3 and S1 and try to include all of them in the main manuscript (rather than splitting across main text and supplementary material). Switching from the different components in panels c and d of both respectively was initially quite confusing and I had to read the descriptions of both in the main text several times before I understood them. This could definitely be improved.   The procedure for parameter optimisation part was not really clear (e.g. it is not really clear how Figure S2 is involved in parameter optimisation), nor is it clear which different settings for lambda (eq. 7) were evaluated. Was this done via a grid-search? How was leave-one-out cross-validation performed (e.g. subject level?). The authors state 'see the supplementary material for details of the optimization methods' but this only seems to refer to the optimizers chosen, not parameter optimization per se.  Also it appears that a single cross-validation loop was employed, then the same parameter setting was used for all subjects. Is that correct? Please provide more details so that the reader can determine whether proper, unbiased parameter optimization techniques were employed.