Perceptual Learning Inceptedby Decoded fMRI Neurofeedback
Without Stimulus Presentation
Kazuhisa Shibata,* Takeo Watanabe,*† Yuka Sasaki,‡ Mitsuo Kawato
It is controversial whether the adult primate early visual cortex is sufficiently plastic to cause visual
perceptual learning (VPL). The controversy occurs partially because most VPL studies have examinedcorrelations between behavioral and neural activity changes rather than cause-and-effect relationships.
With an online-feedback method that uses decoded functional magnetic resonance imaging (fMRI)
signals, we induced activity patterns only in early visual cortex corresponding to an orientation without
stimulus presentation or participants’ awareness of what was to be learned. The induced activation
caused VPL specific to the orientation. These results suggest that early visual areas are so plastic thatmere inductions of activity patterns are sufficient to cause VPL. This technique can induce plasticity
in a highly selective manner, potentially leading to powerful training and rehabilitative protocols.

hether adult primate visual cortex has
sufficient plasticity to allow for behavioral and/or sensitivity changes remains
a point of great controversy. Many studies haveexamined how activity changes in the brain are
correlated with performance improvements on avisual task resulting from repetitive training, known
as visual perceptual learning (VPL). However,such a correlational approach has not conclusively settled the adult plasticity debate. Althoughsome studies have found correlations between
performance increase and changes in early visual areas (1–5), other studies found correlations
in higher visual and/or decision areas (6–8). Noneof these studies directly addresses the questionof whether early visual areas are sufficiently plastic to cause VPL. Changes in early visual areas
observed in correlation with VPL do not exclude
the possibility that the changes are, in reality, areflection of the influences of changes in otherbrain areas. On the other hand, changes in higher
brain areas in some conditions in correlation withsome types of VPL do not rule out the possibility
that early areas are sufficiently plastic to cause
VPL in other conditions.
To addresses the question of whether earlyvisual areas are that plastic, we developed a functional magnetic resonance imaging (f MRI) onlinefeedback method, by which activation patterns
corresponding to the pattern evoked by the presentation of a real and specific target orientation
stimulus were repeatedly induced without theparticipants’ knowledge of what is being learnedW

and without external stimulus presentation [seesupporting online materials (SOM) and methods].
The mere induction of the activation patterns
resulted in significant behavioral performance improvement on the target stimulus orientation, but
not on other orientations.
The complete experiment consisted of fourstages: (i) pre-test (1 day), (ii) fMRI decoder construction (1 day), (iii) induction (decoded fMRI
neurofeedback, 5 days for four participants and
10 days for six participants), and (iv) post-test(1 day) stages (Fig. 1A).
In the pre- and post-test stages, we measured
participants’ performance in an orientation discrimination task (10 participants). In each trial(Fig. 1B), participants were asked to report whichof three orientations (10°, 70°, or 130°) (Fig. 1C)
had been presented in a Gabor patch (see SOM).
Next, we tried to obtain fMRI activity patterns in V1/V2 that are induced by the presentation of each of the three tested orientations inGabor patches for each participant (fMRI decoder construction stage). Participants were asked
to perform a task designed to maintain their attention to the Gabor patches while fMRI signals
in V1/V2 were measured (Fig. 1D and SOM).
Based on the fMRI signals, we constructed amultinomial sparse logistic regression decoder (9)to classify a pattern of the measured fMRI signals into one of the three orientations (fig. S1).
Once the decoder was constructed, each participant took part in a 5- or 10-day induction
stage during which he or she learned to induce
activation patterns in V1/V2 that corresponded
to the target orientation, one of the three orientations, that was randomly assigned to each participant. During each trial (Fig. 1E and SOM),
participants were asked to “somehow regulate
activity in the posterior part of the brain to makethe solid green disc that was presented 6 s lateras large as possible (the maximum possible size
corresponds to the outer green circle).” The sizeof the disc presented in the feedback period
corresponded to the decoder output for the target
orientation, which represented the likelihood of
the blood oxygen level–dependent signal patternin V1/V2 obtained in the preceding induction
period being classified into the target orientation
for which the performance is aimed to be improved. It roughly represented how similar theactivation pattern obtained in the induction period in the absence of visual stimulation is to thepattern evoked by the real Gabor stimulus of the

Downloaded from www.sciencemag.org on December 8, 2011

REPORTS

Advanced Telecommunications Research Institute International
Computational Neuroscience Laboratories, 2-2-2 Hikaridai,
Keihanna Science City, Kyoto 619-0288, Japan.
*Present address: Department of Psychology, Boston University, 64 Cummington Street, Boston, MA 02215, USA.
†To whom correspondence should be addressed. E-mail:takeo@bu.edu‡Present address: Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General
Hospital, 149 Thirteenth Street, Charlestown, MA 02129, USA;and Department of Radiology, Harvard Medical School, 25
Shattuck Street, Boston, MA 02115, USA.

Fig. 1. Procedure of the main experiment. (A) Stages of the experiment. (B) Procedure of a trial of the teststages. (C) Gabor patches of the three orientations used in this experiment. (D) Example of a sequence ofstimuli in one trial of the decoder construction stage. (E) Procedure of a trial in the induction stage.

www.sciencemag.org

SCIENCEVOL 334

9 DECEMBER 20111413

target orientation presented during the decoder
construction stage. We call the likelihood (similarity) target-orientation likelihood. However,participants were not informed of what the size
represented. They were told that they would receive a payment bonus proportional to the mean
size of the feedback disc, but all other information, including the target orientation, the purposeof the neurofeedback, and the meaning of thedisc size, was withheld from the participants.
The main purpose of our study was to test
whether early visual cortical areas are sufficiently plastic to cause VPL of a specific orientation
as a result of mere repetitive inductions of activity
patterns corresponding to that orientation. However, before testing this hypothesis, it was necessary to examine whether participants could learn
to induce significantly high target-orientation
likelihood; that is, a neural activity pattern in
V1/V2 that is similar to an activity pattern evokedby the actual presentation of the target orientation. Thus, we first examined whether outputs ofthe decoder could be biased toward the selected
target orientation compared with the other two

Fig. 2. Results of the induction stage. The mean
(TSE; represented by error bars) likelihoods of threeorientations assessed by a decoder based on V1/V2
activity patterns. For the first 5 days, combined datafrom 5-day (4 participants) and 10-day (6 participants)
neurofeedback sessions is shown. For the last 5 days,
data from 10-day neurofeedback session is shown.

orientations that were rotated T60° from the target orientation. Figure 2 shows that the overallmean target-orientation likelihood in V1/V2 wassignificantly higher than chance across the participants, on average, during the induction stage[t(9) = 3.34, P < 10−2]. The mean (across theparticipants) target-orientation likelihood in
V1/V2 for the first 30 trials of the first neurofeedback day was around chance level (fig. S2).
Thus, there was no significant orientation bias
for the target orientation before neurofeedback,and participants quickly learned to induce significantly high target-orientation likelihood, evenduring the first neurofeedback day (see performance for day 1 in Fig. 2). We also applied thesame analysis to the overall activity pattern thatis the mean across trials and confirmed the sametendency (fig. S3).
Were participants aware of the purpose of
the induction stage? After the post-test stage, participants were asked about what they thought
the size of the feedback disc represented, but
none of their responses was even remotely related to the true workings of the experiment (see“After post-test stage” section of SOM). After being told that the disc size represented the likelihood of one of three orientations, subjects wereasked to report the orientation they thought they
had been trained on by picking one of the threeorientations. The percentage of the choice of thetarget orientation in fig. S4 was statistically undistinguishable from what would be expected fromchance ( c2 = 0.20, P = 0.90).
The purpose of the induction stage was tohave participants learn and then continue to induce activity patterns with significantly high targetorientation likelihood in V1/V2. This learning in
the induction stage should not be confused withthe plasticity or VPL that refers to improvements
on visual tasks. The main purpose of the present
study was to examine whether the mere repetitive
induction of specific activation patterns in V1/V2
causes VPL reflected as performance improvement.
We compared participants’ performance inthe pre- and post-test stages (Fig. 3, A to C).
Three-way [test stage × orientation × signal-tonoise (S/N) ratio] analysis of variance with repeated measures indicated significant main effectof S/N ratio (F3,27 = 683.17, P < 10−4) and sig-

Fig. 3. Results of the pre- and post-tests. (A to C) Discrimination accuracies for –60°
(A), target orientation (B), and +60° (C). Error bars indicate SE. (D) Sensitivity values(d′) in the pre-test subtracted from those in the post-test for three orientations at

1414

9 DECEMBER 2011

VOL 334

nificant effect of interaction between test stage,orientation, and S/N ratio (F6,54 = 2.68, P = 0.02).
Post-hoc t test between accuracies in pre- andpost-tests revealed that discrimination performance for the target orientation significantly improved at the 6% S/N ratio [t(9) = 5.76, P < 10−2
with Bonferroni correction]. Sensitivity (d′) in the
pre-test subtracted from that of the post-test wassignificantly greater than zero for the target orientation at the 6% S/N ratio [t(9) = 5.60, P < 10−3
with Bonferroni correction] (Fig. 3D).
What is the relation between the targetorientation likelihood in V1/V2 and d′ changes?
The sensitivity changes for the participants with10 days training (induction) were larger than for
those with 5 days training (fig. S5). This observation was consistent with the general tendency
that the magnitude of VPL is larger with longer
training until it reaches an asymptote. Thus, we
computed the summation of the target-orientation
likelihoods in V1/V2 for all trials for each participant and plotted the sensitivity change against
the summation. The correlation was even stronger
for the likelihood summation (r = 0.87, P = 10−3)(Fig. 3E) than for the average likelihood (r = 0.74,
P = 0.01).
To test whether the VPL observed in the main
experiment resulted simply from participants’ involvement in the test stages, we conducted acontrol experiment with six new participants inwhich only the pre- and post-test stages wereconducted (SOM). No significant performance
improvement was observed (fig. S6).
Participants can be trained to control theoverall mean activation of an entire brain region or the activation in one region relative tothat in another region (10–13). One might wonder if the participants in the present study simply
learned to regulate the overall activity of V1/V2.
However, two lines of evidence argue against
this possibility. First, the multinomial sparse logistic regression decoder used in this study computed the linear weighted sum of voxel activities,
and weights of the decoder were almost symmetrically distributed around 0 (fig. S7). Second,some voxels in V1/V2 were activated positively
and others negatively, rather than uniform positive or negative activation during the induction
stage (fig. S8).

6% S/N ratio. Error bars indicate SE. (E) Correlation between the summation oftarget-orientation likelihoods in V1/V2 in the induction stage and sensitivity
changes for the target orientation at 6% S/N ratio.
SCIENCEwww.sciencemag.org

Downloaded from www.sciencemag.org on December 8, 2011

REPORTSREPORTS
Our results indicate that the adult early visual cortex is so plastic that mere repetition of theactivity pattern corresponding to a specific featurein the cortex is sufficient to cause VPL of a specific orientation, even without stimulus presentation, conscious awareness of the meaning of theneural patterns that participants induced, or knowledge of the intention of the experiment. How is thepresent research on VPL distinguished from previous approaches? Unit recording and brain
imaging studies have successfully revealed thecorrelation between VPL and neural activity
changes (1–8). However, these correlation studies
cannot clarify cause-and-effect relationships. The
studies that examined the effect of a lesion (15) ortranscranial magnetic stimulation (TMS) (16, 17)
to a brain region on VPL have shown whether theexamined region plays some role in VPL. However, these studies cannot clarify how particular
activity patterns in the region are related to VPL.
In contrast, the present decoded fMRI neurofeedback method allowed us to induce specific neuralactivity patterns in V1/V2, which caused VPL.
The present decoded fMRI neurofeedback
method can be used to clarify cause-and-effectrelationships in many functions in system neuroscience (18, 19). Although previous fMRI onlinefeedback training is a promising technique for
influencing human behaviors (10–13), as in lesion or TMS studies, it could at best reveal influences of the entire extent of an area/region
on learning/memory, which is a certain limitation
for neuroscientific research (20). In contrast, thepresent decoded fMRI neurofeedback method
induces highly selective activity patterns withina brain region, thus allowing the investigator to
influence specific functions. It can “incept” a person to acquire new learning, skills, or memory, or
possibly to restore skills or knowledge that has
been damaged through accident, disease, or aging,
without a person’s awareness of what is learned
or memorized.

References and Notes
1. A. Schoups, R. Vogels, N. Qian, G. Orban, Nature 412,
549 (2001).
2. Y. Yotsumoto, T. Watanabe, Y. Sasaki, Neuron 57, 827(2008).
3. T. Hua et al., Curr. Biol. 20, 887 (2010).
4. N. Censor, Y. Bonneh, A. Arieli, D. Sagi, J. Vision 9,
1 (2009).
5. A. Karni, D. Sagi, Nature 365, 250 (1993).
6. C. T. Law, J. I. Gold, Nat. Neurosci. 11, 505 (2008).
7. T. Yang, J. H. Maunsell, J. Neurosci. 24, 1617 (2004).
8. C. M. Lewis, A. Baldassarre, G. Committeri, G. L. Romani,
M. Corbetta, Proc. Natl. Acad. Sci. U.S.A. 106, 17558(2009).
9. O. Yamashita, M. A. Sato, T. Yoshioka, F. Tong, Y. Kamitani,
Neuroimage 42, 1414 (2008).
10. S. Bray, S. Shimojo, J. P. O’Doherty, J. Neurosci. 27,
7498 (2007).
11. A. Caria et al., Neuroimage 35, 1238 (2007).
12. R. C. deCharms et al., Neuroimage 21, 436 (2004).
13. N. Weiskopf et al., Neuroimage 19, 577 (2003).
14. A. Toda, H. Imamizu, M. Kawato, M. A. Sato, Neuroimage
54, 892 (2011).
15. K. R. Huxlin et al., J. Neurosci. 29, 3981 (2009).
16. E. Corthout, B. Uttl, V. Walsh, M. Hallet, A. Cowey,
Neuroreport 11, 1565 (2000).
17. F. Giovannelli et al., Neuropsychologia 48, 1807 (2010).
18. H. R. Dinse, P. Ragert, B. Pleger, P. Schwenkreis,
M. Tegenthoff, Science 301, 91 (2003).
19. Y. Miyashita, Science 306, 435 (2004).
20. M. Kawato, Philos. Trans. R. Soc. London Ser. B 363,
2201 (2008).
Acknowledgments: This work was conducted in “Brain
Machine Interface Development” under the Strategic Research
Program for Brain Sciences by the Ministry of Education,
Culture, Sports, Science and Technology of Japan. T.W. waspartially supported by NIH grants R01 AG031941 and R01
EY015980 and Y.S. by grants R01 MH091801 and NSF
0964776. We thank J. Dobres, M. Fukuda, G. Ganesh,
H. Imamizu, A. R. Seitz, and K. Tanaka for their comments on a
draft of this manuscript and M. Fukuda, Y. Furukawa, S. Hirose,
M. Sato, and ATR BAIC for technical assistances.

Supporting Online Materialwww.sciencemag.org/cgi/content/full/334/6061/1413/DC1
Materials and Methods
Figs. S1 to S9References (21–31)
1 August 2011; accepted 26 October 201110.1126/science.1212003

Downloaded from www.sciencemag.org on December 8, 2011

The neurofeedback provided to participants
was based on activation patterns only in V1/V2.
However, this procedure might have induced
neural activities in areas other than V1/V2, whichmight also contribute to VPL. To test whether
other regions quantitatively contributed to VPL,
we conducted two offline tests with other areas(such as V3, V4, the intraparietal sulcus, and thelateral prefrontal cortex) that have been implicated in VPL (6–8).
If the orientation-specific activation patternsin V1/V2 during the induction stage induced similar orientation-specific brain activities in otherareas, the activation patterns in those areas should
predict the target-orientation likelihood in V1/V2
on a trial-by-trial basis. In the first offline test,
we employed a sparse linear regression method(14) to predict the target-orientation likelihoodsin V1/V2 from activation patterns in those higher areas in each trial during the induction stage(SOM). Goodness of prediction for the targetorientation likelihood in V1/V2 by other areas,or prediction accuracies of the sparse linear regression, was evaluated by coefficients of determination, all of which were less than 5% (fig. S9A).
We conducted a second offline test to examine the possibility that the decoder simply performed poorly in higher brain areas. We examined
whether accurate orientation information can beread out from each brain area when real orientation stimuli are presented in the decoder construction stage. As was done for V1/V2 during
the fMRI decoder construction stage, we built amultinomial sparse logistic regression decoder toclassify activation patterns into each of the threeorientations (SOM). Decoding accuracies weresignificantly higher than chance level in all ofthese areas (fig. S9B, also compare fig. S9, A
and B). The results of these two offline tests
indicate that influences of the neurofeedback on
VPL were largely confined to early visual areas
such as V1/V2.

(HP) network subserving this type
Entorhinal Cortex Layer III Input to the hippocampus
of association have remained unknown. The EC
provides inputs to the HP via two major pro(Fig. 1A): the trisynaptic pathway (TSP)
Hippocampus Is Crucial for Temporal jections
originating from EC layer II and the monosynaptic pathway (MSP) originating from EC layer
Association Memory
III (ECIII). Studies on genetically engineered
Junghyup Suh,1 Alexander J. Rivest,1 Toshiaki Nakashiba,1 Takashi Tominaga,2 Susumu Tonegawa1*
Associating temporally discontinuous elements is crucial for the formation of episodic andworking memories that depend on the hippocampal-entorhinal network. However, the neuralcircuits subserving these associations have remained unknown. The layer III inputs of theentorhinal cortex to the hippocampus may contribute to this process. To test this hypothesis,
we generated a transgenic mouse in which these inputs are specifically inhibited. The
mutant mice displayed significant impairments in spatial working-memory tasks and in theencoding phase of trace fear-conditioning. These results indicate a critical role of the entorhinal
cortex layer III inputs to the hippocampus in temporal association memory.

A

critical feature of episodic memory
shared by some forms of working memory is the ability to associate tempo-

rally discontinuous elements, called temporal
association memory (1–3). However, the neural circuits within the entorhinal cortex (EC)–www.sciencemag.org

SCIENCEVOL 334

mice (4–7) and lesioned rats (8–11) have demonstrated crucial roles of the TSP in several featuresof episodic-memory processing, such as patterncompletion (5–8) and separation (4, 8). In contrast, the MSP contributions to episodic-memory
processing remain poorly known. We tested the1
The Picower Institute for Learning and Memory, RIKEN–MIT
Center for Neural Circuit Genetics, Department of Biology andDepartment of Brain and Cognitive Sciences, Massachusetts
Institute of Technology, Cambridge, MA 02139, USA. 2Department of Neurophysiology, Faculty of Pharmaceutical
Sciences, Tokushima Bunri University, Kagawa, Japan.

*To whom correspondence should be addressed. E-mail:tonegawa@mit.edu

9 DECEMBER 20111415