Peer review of: Unconscious reinforcement learning of hidden brain states supported by confidence (2020)



<b>REVIEWER COMMENTS</B> 

Reviewer #1 (Remarks to the Author): 

Review of "Learning to use latent brain representations with confidence" 

In this revised manuscript the authors have toned down claims about the causal role of meta-
cognition in the ability to learn latent state representations from brain states. The manuscript is 
considerably improved in the light of the revisions made. 
A few outstanding issues are noted: 

(1) The division of confidence ratings into different high vs low categories for session 1 and 2 (1 = 
low, 2-4 = high) vs session 3 (1 to 3 = low, 4 = high) shown in Fig 5B seems arbitrary and difficult to 
justify other than by the fact that dividing the data differently in this way shows a significant 
differential effect between PFC and VC. The authors should either use a consistent criterion across the 
3 sessions if they wish to make claims about common effects across sessions, or they need to justify 
why they have elected to use different criteria in the different sessions. Session 3 is a different task 
altogether from session 1 and 2, so there may be a justification for doing so, but if they want to claim 
common mechanisms, using the same analysis criteria across the two different tasks would seem 
prudent. 
(2) The neural substrates results presented on page 14 refer to supplementary figure s7 in which it is 
claimed that the RPE signal describes a global search in sessions 1 and 2 which eventually converges 
to the basal ganglia by session 3. However, this convergence may not be anything to do with a 
convergence of a global search – instead session 3 is a different task to sessions 1 and 2, and so any 
difference in RPE correlates between sessions 1 and 2 vs session 3 could simply reflect the difference 
in the task properties, and be nothing to do with a convergence in global search. If they find evidence 
for such convergence WITHIN sessions 1 and 2 without considering session 3, then this claim could be 
made more strongly perhaps. As is, it should not be made without severe qualification, because it is 
impossible to tell why session 3 is different to sessions 1 and 2, the most parsimonious explanation 
being it is due to the difference in the task. 

Reviewer #2 (Remarks to the Author): 

I continue to feel that this study is an interesting one that will command considerable interest. In 
brief, what the Cortese and colleagues have done is very unusual. They have shown that people can 
learn to take one of two different actions conditional on the existence of one of two states. Such 
conditional learning is well known but what makes this report exciting is the nature of the states being 
considered – they are multivariate patterns of activity distributed across voxels in either prefrontal 
cortex or visual cortex. Just the observation that human subjects are able to do this will be of interest 
to many readers. In their response to reviewer 1 point 2 the authors have summarized very clearly 
why this is an unusually interesting result to have obtained; it is remarkable that the participants were 
able to identify that these internal brain states defined the conditions that determined which action to 
take. 

Second, the authors have shown that there is a relationship between the learning of this process and a 
measure of subjective confidence that the participants provide. Some of the other reviewers have 
focussed on the direction of cause and effect underlying this correlation. This is indeed an interesting 
question but it is secondary to the fact that the correlation exists. In response to the reviewers, the 
authors have toned down their description of this aspect of the results so that the correlational 

findings are presented clearly. Nevertheless, there are interesting reasons for thinking that the 
authors maybe correct to speculate that better metacognition leads to better learning. Even if the 
authors’ interpretation of the direction of cause and effect proves ultimately to be incorrect the 
observation of the correlation will remain interesting and is likely to drive future investigation. As it 
stands it is reported in a measured manner. 

Throughout the report the authors have illustrated the neural data on which learning is based. 
Reviewer 1 has argued that the neuroimaging data are of little value but, of course, the neural data 
are critical here. They serve in the place of stimuli that might be learned about. The neuroimaging 
data are then used in quite a different manner in figure 6 which summarizes a very different set of 
neural activity patterns and their evolution during learning. These are not the patterns that the 
participants were learning about but rather they are patterns related to the learning process itself: 
patterns related subjective confidence in decisions and patterns related to the prediction errors that 
drive learning. 

Finally, it might be worth noting that the authors have scanned a reasonably sized group of human 
participants but they have scanned each person three times so that they are working with a data set 
that is much larger than the average behavioural of neuroimaging data set. 

Reviewer #3 (Remarks to the Author): 

I am not quite satisfied by the authors’ response concerning the problem of sample size (issue 2). The 
fact that scanning is time consuming and expensive is not recevable as a scientific argument. They 
start with 22 subjects and exclude 4 subjects (3 for technical reasons, 1 because the decoder was not 
working). This means that they excluded 18% of the initial subjects. The final sample size of 18 is far 
below the currently accepted sample size. A recent meta-analysis (Lebreton et al. 2019) indicates that 
the median sample size for model-based fMRI studies in the past years is 27, meaning that their 
sample size represents the 66% of the average accepted sample size. If they do not want to run a 
replication experiment, they cold at least increase the sample size to match the median one. I also ask 
them to perform retrospective power calculation to determine what is the power of detecting the 
reported effects with their sample size.