Summary

Introduction
Reading letters and words consists of multiple steps including orthographic, phonological, and lexico-seman- tic processing, which are usually acquired at school age but can still be mastered in adulthood. Previous imaging studies for adult subjects have reported that the left fusiform gyrus (FG) showed more activation for visually presented consonant letter strings than for pseudofonts or faces (Price et al., 1996; Puce et al., 1996; Tarkiainen et al., 2002), indicating that this region is selectively involved in letter recognition. Moreover, recent func- tional imaging studies have clarified that the left FG showed larger activation for both pronounceable words and pseudowords than for unpronounceable consonant letter strings (Fiez et al., 1999; Brunswick et al., 1999; Cohen et al., 2002), suggesting that the left FG discrimi- nates, on the basis of orthographic regularities, between legal and illegal strings. Several studies have also re- ported that the activation for letter strings extended into the left posterior inferior temporal gyrus (PITG) (Paulesu et al., 2000; Xu et al., 2001; Mechelli et al., 2003). While the left FG/PITG is thus referred to as the “visual word form area" (McCandliss et al., 2003), other researchers have clarified that this region is also activated during various multimodal tasks that do not engage visual word form processing (Price and Devlin, 2003). It is thus cru- cial to determine which cognitive factors are responsible for activating the left FG/PITG. A fundamental question that has not been addressed previously is whether the left FG/PITG or other regions show plasticity for learning to associate letters with speech sounds, which is one critical subprocess of reading. It also remains unclear whether forming a new link between orthography and other linguistic informa- tion is necessary for the cortical plasticity, if such plas- ticity is indeed observable, in those regions. It is still possible that mere visual exposure to unfamiliar letters or simple crossmodal matching between visual symbols and sounds is sufficient for plasticity. It is thus necessary to directly compare cortical activations under various conditions of symbol-sound matching with the same task requirements. In the present study, we used func- tional magnetic resonance imaging (fMRI) to examine cortical activations while the adult subjects were trained with Hangul letters for the first time. The subjects were Japanese who had learned kana letters by the age of seven, but they had no knowledge of Hangul letters before the experiments. Kana and Hangul letters are exclusively used in the Japanese and Korean languages, respectively. For both kana and Hangul letters used in the present study, there was a one-to-one correspon- dence between each letter and a speech sound. Here we focused on prelexical processing and used nonsense words for all the linguistic stimuli. The subjects partici- pated in fMRI sessions over two consecutive days (Days 1 and 2, approximately 24 hr apart). The scanning ses- sions served as both testing and repeated exposures to the Hangul letters, and there was no separate training session except the initial exposure to the letters (see Experimental Procedures). The fMRI sessions consisted of four audiovisual matching tasks characterized as the following condi- tions: kana and speech (KS), Hangul and speech (HS), Hangul and nonspeech (HN), and nonletter and non- speech (NN) (Figure 1A). Under the KS condition, the subjects were presented with a string of three kana letters for each trial of 4 s, and they were asked to judge whether or not it matched a string of three-syllable speech sounds. Individual letters were serially pre- sented to ensure that the subjects attended equally to each letter. Because each letter was immediately masked by the next letter or a masking stimulus for the last letter, the subjects had to recognize each individual letter within the brief presentation time (250 ms). Under the HS condition, the subjects were asked to judge whether a string of three Hangul letters matched a string of three-syllable speech sounds. Under both the KS and HS conditions, we used the same set of nine syllables as speech sounds, each of which was unambiguously associated with one of nine kana letters and one of nine Figure 1.  Audiovisual Matching Tasks and Stimuli (A) Experimental conditions for audiovisual matching tasks. We em- ployed the following four conditions: kana and speech (KS), Hangul and speech (HS), Hangul and nonspeech (HN), and nonletter and nonspeech (NN). Under one of these conditions, a string of three visual stimuli (250 ms each) was presented at the center of the screen for each trial of 4 s. The subjects were asked to judge whether or not the visual string matched a string of auditory stimuli (total of 550 ms) that were presented either before or after the visual stimuli. A random dot pattern was presented between visual strings. A red cross for fixation was always shown at the center of the screen, but it was omitted from the figure. For the KS and NN examples, the visual and auditory strings matched, and they did not match for the HS and HN examples. (B) All visual stimuli used in the four conditions. Each visual stimulus and its associated sound are shown. The nine letters in the leftmost column were used under the KS condition. The subjects were al- ready familiar with kana letters, which are used in the Japanese writing system. The nine letters in the middle column were used under the HS condition. The subjects learned how to read each Hangul letter at one letter per second immediately before the experi- ments. Because each letter was masked by the next stimulus, accu- racy data critically reflected the subjects' ability to recognize each letter within the brief presentation time of 250 ms. Note that these kana and Hangul letters are associated with the same set of speech sounds. In the rightmost column, the upper eight Hangul letters were used under the HN condition, and the lower four geometric patterns were used under the NN condition. Under the HN condition, each Hangul letter was associated with one of the following non- speech sounds: low-frequency pure tone (T-L), high-frequency pure tone (T-H), low-frequency white noise (N-L), and high-frequency white noise (N-H). The subjects learned these associations immedi- ately before the experiments, but they had no knowledge about proper reading of these letters in the Korean language. Each Hangul letter used under the HS condition is a combination of two elements: one element (〇, 1, or □) for a consonant, the other element (F, I, or –1) for a vowel. Each Hangul letter used in the HN condition is also a combination of two elements, and each element was associated with either a sound type (pure tone or white noise) or a sound height (low or high). Note that some of these elements, such as circles and squares, are shared among the Hangul letters under both conditions. The complexity of letters is thus basically equivalent between the HS and HN conditions. Hangul letters (Figure 1B). It should be noted that all of the linguistic stimuli used under the KS and HS condi- tions are phonotactically legal strings. Under the HN condition, we used another set of eight Hangul letters, and the subjects were trained to associate these com- plex Hangul letters with nonspeech sounds. This condi- tion lacks phonology; thus, no link between orthography and phonology is trained throughout the experiments. Under the NN condition, in order to control for general cognitive factors related to low-level crossmodal match- ing and association, we further replaced letters with geometric patterns that are associated with nonspeech sounds. We targeted HS as a major learning condition of inter- est for matching speech sounds with newly learned let- ters, whereas KS and HN served as control conditions for speech sounds and Hangul letters, respectively. In functional mapping analyses, we performed a two-way analysis of variance (ANOVA) for task and day (Days 1 and 2), focusing on the HS-selective brain activations. Following further analyses of task comparisons and functional connectivity, HS-selective activations re- vealed the cortical plasticity for forming a new link be- tween orthography and phonology.
Results
Behavioral Data
We performed two separate experiments, each of which consisted of two-day sessions (Days 1 and 2): Experi- ment I under the KS i, HS i, and NN i conditions, and Exper- iment II under the KS ll , HN ll , and NN ll conditions (the subscripts denote Experiment I or II). The HS i and HN i conditions were conducted in separate experiments, so that the subjects could concentrate on a single condition for learning Hangul letters per experiment. The learning curves obtained from the individual scanning sessions are shown in Figure 2. For each task, we compared the accuracy data of the first half of the number of sessions (1–5 or 11–15) with those of the latter half of the number of sessions (6–10 or 16–20) separately for Days 1 and 2. Only HN ll on Day 1 showed a significant difference (paired t test, p < 0.05), whereas there were no signifi- cant differences for all other tasks on Day 1 or Day 2 (p > 0.05). In Experiment I, the averaged accuracy data on Days 1 and 2 were as follows (mean ± standard error, n = 12): KS t, 97.5% ± 0.6% and 97.8% ± 0.9%; HS t, 68.2% ± 2.1% and 76.4% ± 2.0%; and NN i , 93.7% ± 1.8% and 96.2% ± 1.2%. Both HS 1 and NN 1 resulted in significant accuracy improvements over the two days (paired t test, p < 0.05), without a significant difference across days in KS ı (p > 0.5). In Experiment II, the averaged accuracy data on Days 1 and 2 were as follows: KS n , 97.9%  ± 0.6% and 97.9% ± 0.6%; HN r, 90.8% ± 1.0% and 97.6% ± 0.6%; and NN ll , 93.6% ± 1.8% and 96.1% ± 0.6%. Both HN』and NN』resulted in significant accuracy improvements (p < 0.05), whereas there was no signifi- cant difference across days in KS ll (p > 0.5). These results indicate that learning effects of HS, HN, and NN on Day 1 were clearly manifested on Day 2. Next, we separately performed a two-way ANOVA on the accuracy data for each experiment and found signifi- cant main effects of task and day, as well as the interac- tion of task by day (p < 0.001), in both experiments. A post hoc test (Fisher's protected least significant differ- ence) revealed significant differences between KS 1 and HS  as well as between HS  and NN  (p < 0.0001) in Experiment I and between KS n and HN (p < 0.005) in Experiment II. To compare learning effects between the HS and HN conditions, we performed a two-way ANOVA for task (HS and HN e) and day (Days 1 and 2) on the accuracy data. Significant main effects of task and day were observed (p < 0.0001), though the interac- tion of task by day was not significant (p > 0.5). This latter result indicates that the degree of accuracy im- provements between days in HS 1 was parallel to that in HN l」 .
Voxel-wise ANOVA Maps for Task and Day
Because we targeted HS as a major learning condition of interest for matching speech sounds with newly learned letters, a voxel-wise ANOVA for task and day was first conducted separately for the data sets of the HS 1 and KS tasks and for those of the HS 1 and HN 1 tasks. These results were then combined to assess consistent activa- tions with equal statistical significance (corrected p < 0.05), thereby taking the AND of two statistical maps. First, we examined the main effects of task, i.e., HS- selective activations, Day 1 + Day 2: (HS 1 − KS ) and (HS 1 − HN 11 ). Note that this is an abbreviated notation; its exact statistical model is: [Day 1 + Day 2: (HS 1 – KS ))] and [Day 1  + Day 2: (HS 1  – NN )  – (HN 1  – NN 7 )]. We observed significant bilateral activations in multiple regions, including the dorsal frontal cortex, PITG (the lateral portion of the FG/PITG), the parieto-occipital (PO) cortex, the calcarine fissure and surrounding cortex, and the cerebellum (Figure 3A and Table 1). According to automated anatomical labeling (AAL) (Tzourio-Mazoyer et al., 2002), the left temporal activation was observed mostly in the left PITG (72%) without extension to the left FG (the medial portion of the FG/PITG); the right temporal activation was observed in the right PITG (98%) with minimum extension to the right FG (2.0%). Next, we e xamined the main effects of day, Day 2 – Day 1: (HS i + KS i) and (HS i + HN il); the exact statistical model of this notation is: [Day 2 − Day 1: (HS 1 + KS i) − 2NN,] and [Day 2 – Day 1: (HS, − NN,) + (HN,1 − NN,1)]. This analysis showed significant activations in the medial occipital cortex and the right Rolandic operculum (Fig- ure 3B and Table 1). The other direction of main effects of day, Day 1  – Day 2: (HS 1 + KS 1) and (HS 1 + HN 1), resulted in activations of the dorsal frontal cortex, the bilateral angular gyrus (AG), and the median cingulate gyrus (Table 1). These regions showing the main effects of day may reflect task-independent factors of facilita- tion or habituation. In contrast to these nonspecific factors, the interac- tion of task by day, Day 2 – Day 1: (HS  – KS) and (HS 1 − HN 1 ), would reveal task-related learning effects on activations. We found significant activations in a region along the left precentral sulcus (PrCS), the left PITG, the left superior parietal gyrus, the calcarine fissure and surrounding cortex, and the left cerebellum (Figure 3C and Table 1). On the other hand, the interaction of task by day, Day 1 – Day 2: (HS i – KS i ) and (HS i – HN i ), resulted in significant activations in the orbital part of the left frontal cortex and a portion of the bilateral AG.
Task-Related Activations
Based on the fact that the main effects of task resulted in activations over widespread regions (Figure 3A), some of the activated regions may be specifically involved in matching speech sounds with newly learned letters, and other regions might reflect general effortful processes for the most difficult HS condition. To identify the brain regions involved in matching speech sounds with letters in the absence of the general effortful efects, we per- formed the comparison of KS 1 − NN 1 . Using the data in Days 1 and 2, KS 1 – NN 1 resulted in significant bilateral activations of the superior and middle temporal gyri (STG/MTG), the FG/PITG, and the PO (Figure 4A and Table 1). Significant activations in all of these regions were replicated in Experiment II. We found that none of the regions showing the main effects of day (both Day 2 – Day 1 and Day 1 – Day 2; Table 1) extended into the regions identified by KS 1 – NN 1. Among the regions showing significant task by day interaction (Figure 3C), only the left PITG overlapped with the regions identified by KS 1 – NN 1. In the comparison of KS 1 – NN 1, the FG/ PITG and the PO activations in the left hemisphere were much greater than those in the right hemisphere, as shown by their spatial extent. Activation in the left PO Figure 3.  Voxel-wise ANOVA Maps for Task and Day Activations were projected onto a surface- rendered standard brain. Left to right: the left (L) lateral view, the posterior view, and the right lateral view. (A) Main effects of task, HS-selective. Its ex- act statistical model is: [Day 1  + Day 2: (HS; − KS)] and [Day 1 + Day 2: (HS; − NN;) − (HN;) − NN t)]. The subscripts denote Experiment I or Il. (B) Main effects of day, Day 2-selective. Its exact statistical model is: [Day 2 － Day 1: (HS) + KS)) − 2NN.] and [Day 2 − Day 1: (HS) − NN) + (HN, - NN,)]. (C) Interaction of task by day, HS- and Day 2-selective. Its exact statistical model is: [Day 2 − Day 1: (HS, − KS,)] and [Day 2 − Day 1: (HS, − NN,) − (HN, − NN,)]. The activated regions may reflect task-related learning ef- fects. partially spanned multiple regions including the middle occipital gyrus, the inferior parietal gyrus, the superior parietal gyrus, and AG. When KS i — NN i was performed for each separate day, the same bilateral activation in the STG/MTG, as well as the left-dominant activations in the FG/PITG and the PO, were consistently observed. We confirmed that there was no significant interaction of task by day in KS 1 − NN 1 for both Day 2 − Day 1 and Day 1  – Day 2 (corrected p > 0.05). Among the multiple regions with the main effects of task (i.e., HS-selective; Figure 3A), it should be noted that a portion of the bilateral PITG and PO showed acti- vations also in KS 1 − NN (Figure 4A), while activations in frontal regions were not significant in KS 1 – NN 1. Al- though the localized activations for an interaction of task by day (Figure 3C) suggest the presence of more specific learning effects, further analyses are thus re- quired to narrow down the most critical regions. To identify and characterize functionally unitary regions within the PITG and PO regions, the activated regions in the comparison of KS 1 − NN 1 (Figure 4A) were used as an inclusive mask for subsequent analyses. These regions primarily represent the processes of matching speech sounds with already acquired letters, but for this reason they are free from any general effortful pro- cesses. A comparison of HS  – KS 1 , using the data in Days 1 and 2 with an inclusive mask of KS 1 – NN 1 , revealed significant activations in the left PITG with minimum extension to the left FG (3.5% according to AAL), as well as in the left PO and the right FG/PITG (Table 1, red regions in Figure 4B). The activations occupied a considerable portion of the left PO (75%), the left FG/ PITG (37%), and the right FG/PITG (17%) that were iden- tified by KS 1 – NN 1. The prominent activations in the left PITG and the left PO, as well as the right FG/PITG activations, were replicated in the comparison of HS 1 – KS  for the respective separate days. In contrast, a comparison of KS  − HS , using the data in Days 1 and 2 with an inclusive mask of KS 1 − NN 1 , revealed significant activations in the left FG with mini- mum extension to the left PITG (2.6% according to AAL), as well as in the bilateral STG/MTG (Table 1, blue regions in Figure 4B). The temporal activations occupied a large portion of the left STG/MTG (72%) and the right STG/ MTG (87%) that were identified by KS 1 – NN 1 , whereas the left FG activation occupied a smaller portion of the left FG/PITG (10%). These activations were replicated in the comparison of KS 1 − HS 1 for the respective separate days. The fact that the regions identified by KS 1 – NN 1 can be complementarily divided into the KS- and HS- selective regions is intriguing. Moreover, the KS- and HS-selective activations are not due to auditory factors alone, because the same set of speech sounds were presented under both KS and HS conditions. The main effects of task, Day 1 + Day 2: (HS i − KS i) and (HS i − HN ), which incorporates the direct comparison between HS ı and HN ı , further controls the stimulus-specific visual effects of the Hangul letters. Given that the complexity of unfamiliar Hangul letters was basically equivalent be- tween the HS and HN conditions (Figure 1B), we also excluded the possibility that the HS-selective activa- tions in the left PITG and the left PO were reflective solely of visual exposure to new and complex letters.
in the contrasts indicated. All statistical thresholds were set at corrected p < 0.05. L, left; R, right; M, medial; BA, Brodmann's area; PrCS, precentral sulcus; F3op and F3t, opercular and triangular parts of inferior frontal gyrus, respectively; F3O, F2O, and F1O, orbital part of the inferior, middle, and superior frontal gyri, respectively; F1M, medial part of superior frontal gyrus; SMA, supplementary motor area; STG, superior temporal gyrus; MTG, middle temporal gyrus; FG, fusiform gyrus; PITG, posterior inferior temporal gyrus; PO, parieto-occipital cortex; SPG, superior parietal gyrus; AG, angular gyrus. Figure 4.  Selective Activations for the KS and HS Conditions (A) Regions identified by the contrast KS, − NN, (green), using the
Learning Effects on the Left PITG Activation
To investigate for a more direct effect of training under the HS condition, we examined the interaction of task by day in HS 1 – NN 1 for Day 2 – Day 1 with an inclusive mask of KS 1 − NN 1 . Significant activation was observed in the left PITG alone (Figure 5A and Table 1; green region in Figure 5B), which was in agreement with the observation that only the left PITG, among the regions showing significant task by day interaction (Figure 3C), overlapped with the regions identified by KS 1 − NN 1 . We further confirmed that this region was a portion of the left PITG identified by HS 1 – KS 1 (red region in Figure 5B). In contrast, the interaction of task by day in HN n − NNu for Day 2 – Day 1 did not show any significant activation, although the improvements in accuracy were observed in both HS and HN conditions (see Behavioral Data). When the regions identified by HS 1 − KS 1 (red) and KS」 – HS」 (blue) are overlaid onto the same sections (Figure 5B), it is evident that the left PITG identified by HS 1 – KS 1 is clearly segregated from the left FG identified by KS 1 −HS 1 along the lateral occipitotemporal sulcus (arrows). It is intriguing that the functional separation between the two adjacent regions of the left PITG and the left FG followed this anatomical boundary. To confirm that an activation increase in the left PITG was consistent among the subjects, we also performed individual analyses treating subjects as a random effect. At its local maximum (−54, −51, −18), we calculated the averaged signal changes under KS, HS, and HN separately for Days 1 and 2 with reference to a corre- sponding NN condition (Figure 5C). A two-way ANOVA for the individual data revealed a significant interaction of task by day (p < 0.05), as well as main effects of task (p < 0.05), but without significant main effects of day (p > 0.1). HS 1 for Day 2 elicited significantly larger activa- tion than HS 1 for Day 1 (paired t test, p < 0.01), and HS 1 for both Days 1 and 2 showed clearly greater activations than those of KS 1 and HN 11 . In contrast, KS 1 and HN 11 showed comparable activations for the two days (p > 0.1). We also performed individual analyses at the local maximum (−36, −42, −24) of the left FG (Figure 5D). A two-way ANOVA revealed significant main effects of task for the KS-selective activation (p < 0.01), though neither significant main effects of day nor significant interaction of task by day (p > 0.1) were observed. These results suggest that the activation increase in the left PITG is a possible neural correlate for forming a new link between orthography and phonology. We therefore tested whether the activation change in this region was predictive of how well each individual subject improved his/her performance over the two days. The performance of HS resulted in clear improvement (paired t test, p < 0.0001), though improvement varied among subjects (Figure 6A). We found a significant posi- tive correlation between performance improvement and activation increase in the left PITG under the HS condi- tion (r = 0.68, p = 0.01) (Figure 6B). These results sug- gest that activation in the left PITG is a good indicator of the degree of performance improvement in learning letters.
Plasticity of Functional Connectivity between
the Left PITG and the Left PO
We next examined how functional networks changed as subjects improved their performance. A functionally unitary region of the left PITG identified by HS 1 – KS 1 (Figure 4B) was selected as a region of interest (ROI), which served as a reference region for functional con- nectivity analyses (see Experimental Procedures). Acti- vations temporally correlated with this ROI were evalu- ated in a voxel-wise manner under each of the KS, HS, and HN conditions (Figures 7A–7F and Supplemental Table S1 at http://www.neuron.org/cgi/content/full/42/ 2/311/DC1), separately for Days 1 and 2. When com- pared with Day 1, more distinct clusters of significant voxels were observed in the right FG/PITG on Day 2 under all the three conditions, which may reflect general learning effects of the crossmodal matching tasks. On Day 2 under the HS and HN conditions, significant voxels were also observed in the left dorsal frontal cortex. In contrast to these regions with nonspecific effects, only the left PO showed specific changes under the HS condi- tion, such that significant voxels were more clearly mani- fested on Day 2 than on Day 1. To quantify the HS-selective changes in functional connectivity between the left PITG and the left PO, we calculated an r value for each voxel within the left PO (187 voxels) with reference to the left PITG. Under the KS condition, there were no voxels that reached the significance level for both Days 1 and 2 (Figures 7G and 7H). Under the HS condition, however, the peak of distribution was just below the significance level for Day 1 (Figure 7l); the peak reached a level well above that of significance for Day 2 (Figure 7J). The positive shift between these distributions from Day 1 to Day 2 was statistically significant (Mann-Whitney test, p < 0.0001). This enhancement of functional connectivity is specific to the HS condition; the HN condition did not show such a shift (Figures 7K and 7L). These results demonstrate that the functional connectivity between the left PITG and the left PO is selectively and rapidly enhanced dur- ing the formation of a new link between orthography and phonology.
Discussion
By employing the novel audiovisual matching paradigm to examine crossmodal learning effects on cortical plas- Figure 6. Activation Increase in the Left PITG under the HS Condi- tion that Predicts Performance Improvement (A) Individual data for accuracy (%) under the HS condition. Each Figure 5.  The   Anatomical   and   Functional Segregation between the Left PITG and the Left FG (A) A region exhibiting an activation increase induced by learning Hangul letters with speech sounds. The interaction of task by day in HS 1 − NN 1 for Day 2 − Day 1 resulted in significant activation in the left PITG (green), projected onto a surface-rendered standard brain. The location of its local maximum is shown by blue crosshairs (y − −51, z − −18). (B) The separation within the left FG/PITG. On the coronal (y − −51) and axial (z − −18) slices of a standard brain, the regions identi- fied by KS 1 - HS and HS 1 - KS 1 are shown in blue and red, respectively. Arrows indicate the lateral occipitotemporal sulcus that cor- responds to the border between the PITG (lat- eral) and the FG (medial). Note that the region (green) with the interaction of task by day shown in (A) is localized within the left PITG. (C) Averaged activation data among subjects at the local maximum (−54, −51, −18) of the left PITG. Mean percent signal changes for KS ; (open bars), HS ; (filled bars), and HN ; (crosshatched bars) on Days 1 and 2 are shown with reference to the corresponding NN condition. Error bars denote standard er- rors among the subjects. Note the selective activation increase under the HS condition from Day 1 to Day 2. (D) Averaged activation data at the local max- imum (−36, −42, −24) of the left FG. In con- trast to the left PITG, the left FG shows larger activation for KS, than for HS, or HN,, with no significant activation change between Days 1 and 2. ticity, we obtained three striking results. First, we found that the regions recruited for integrating newly learned orthographic and phonological information, the left PITG and the left PO, are indeed separable from the specific regions, the bilateral STG/MTG and the left FG, neces- sary for processing associative information including already acquired orthography. Moreover, the left PITG and the left PO activations, which are selectively en- hanced under the HS condition in comparison with those under the KS and HN conditions, are not due to auditory factors alone, because the same set of speech sounds were presented under the KS and HS conditions. We also excluded the possibility that they reflect visual ex- posure to new and complex letters alone, given that the complexity of the visual stimuli used under the HS and HN conditions was basically equivalent. Therefore, we conclude that the left PITG and the left PO subserve the formation of a new link between letter and speech representations. Second, the left PITG showed a selec- tive activation increase under the HS condition over 2 days, the degree of which was predictive of individual performance improvement. Behavioral results also indi- cate that learning effects of HS and HN on Day 1 were clearly manifested on Day 2. These results suggest that the left PITG plays a primary and critical role in the cortical plasticity involved in learning new letters. Third, we demonstrated that functional connectivity between the left PITG and the left PO was selectively enhanced under the HS condition, but not under the KS or HN condition. The present fMRI study thus establishes that the human adult brain is still adaptable to new associa- tions between orthographic and phonological stimuli, and that this type of learning is achieved in only 2 days by the rapid plasticity of a functional system critically involving the left PITG in association with the left PO. Previous imaging studies have reported a repetition priming effect of visual objects that resulted in an activa- tion decrease in the occipitotemporal cortex (Van Turen- nout et al., 2000; Dehaene et al., 2001; Soon et al., 2003). It is possible that this activation decrease reflects a general habituation effect or less novelty-related atten- tion as a result of repeated presentation. In contrast, we found a learning-related activation increase in the left PITG. It is unlikely that the activation in the left PITG primarily reflects novelty-related attentional effects or general task difficulty, because the activation in HS for Day 2 was larger than that for the more demanding HS for Day 1. We also excluded the possible involvement of general familiarization or associative skills per se as a major factor, because the activation under the HS condition was larger than that for familiar kana letters under the KS condition, and such an increase was not observed under the HN condition. Therefore, we suggest that this activation increase represents the specific de- velopment of a new link between orthography and pho- nology. It remains to be studied whether the left PITG activation returns to the level of the KS condition when new letters are fully mastered. By analyzing the learning curves, we observed that a clear performance improve- ment under the HS condition occurred between Days 1 and 2, during which time there was no exposure to the Hangul letters. Recent behavioral studies have sug- gested that consolidation of new learning occurs during sleep, which results in significant improvements in lan- guage learning tasks (Ficca et al., 2000; Fenn et al., 2003), as well as in perceptual or skill-learning tasks (Karni et al., 1994; Walker et al., 2002). It is thus possible that the period of sleep between Days 1 and 2 in the present study facilitated the acquired association be- tween orthography and phonology. We also observed task-related learning effects in the left dorsal frontal cortex, the medial occipital cortex, and the cerebellum. The left dorsal frontal cortex has been implicated in complex phonological tasks, such as rhyming judgment and syllable counting tasks (Paulesu et al., 1993; Poldrack et al., 1999). It is thus possible that the initial stages of learning letters may also recruit the regions that are involved in effortful pho- nological processing. A recent imaging study trained Japanese subjects on Hangul words for 2 weeks and then examined the training effect on activations for read- ing Hangul words (Lee et al., 2003). It was found that the left occipital cortex showed larger activation for reading Hangul words than for reading Japanese words, which is consistent with the present finding of significant main effects of task in the medial occipital region (Figure 3A). However, we also found significant main effects of day as well as the interaction of task by day (Figures 3B and 3C), neither of which was reported by Lee et al. (2003). It is possible that the occipital activation is enhanced by novel letters, whereas learning effects on its activa- tion may depend on multiple factors, such as task de- mands, learning materials (words or nonsense letter strings), and training periods. Indeed, a learning-related activation increase in the left medial occipital region has been reported in a previous imaging study in which subjects were trained for verb generation tasks (Raichle et al., 1994). On the other hand, activation decrease in the cerebellum has been reported during performance improvement (Raichle et al., 1994; Vaina et al., 1998). Nevertheless, cerebellar contributions to cognitive func- tions such as attention, memory, and language have been suggested (Allen et al., 1997; Desmond and Fiez, 1998), and cerebellar involvement in skill automatization, disruption of which may lead to reading impairment, has been recently proposed (Nicolson et al., 2001). The ability of learning to read is selectively impaired in patients diagnosed with developmental dyslexia (Shaywitz, 1996; Habib, 2000). With regard to the neuro- logical basis of this impairment, previous imaging stud- ies have revealed reduced activation in the left FG/PITG of dyslexic adults relative to that in normal controls when reading words or pseudowords (Shaywitz et al., 1998; Brunswick et al., 1999; Paulesu et al., 2001). A mag- netoencephalography (MEG) study revealed cortical re- sponses selective to letter strings in the left inferior oc- cipitotemporal cortex in normal adults; these responses were not detectable in dyslexics (Helenius et al., 1999). The findings in the present study are consistent with those in these studies, in that they suggest that the lesions in the left FG/PITG may impair linking orthogra- phy to phonology. Furthermore, we have clearly estab- lished differential roles for the left FG and the left PITG; the left FG processes information about already ac- quired orthography, whereas the left PITG integrates newly learned orthography and phonology. We suggest that in the case of the literate adult readers, a selective lesion of the left FG or the left PITG might lead to double dissociation of deficits in reading letters, such that a left FG lesion selectively impairs already acquired orthogra- phy, whereas a left PITG impairment selectively impairs the formation of a new link between orthography and phonology. A previous fMRI study reported that skill learning for reading mirror-reversed words was associated with in- creased activation in the bilateral FG/PITG as well as in the frontal and cerebellar regions (Poldrack and Gabrieli, 2001). The activation increase was also observed for inverted-reversed and spelled-backward text without separate training sessions, which suggested that these increased activations were not due to learning particular orthographic representations, but, rather, to increased engagement of lexical/phonological processes. How- ever, as shown by the HS and HN tasks in the present study, an association between novel orthography and speech or nonspeech sounds can be rapidly learned during scanning sessions alone, which may parallel the generalization of spatial transformations in their study. The present study further excludes the possibility that the enhanced activation in the left PITG is due to in- creased engagement of lexical and semantic processes, based on the fact that nonsense words were used as stimuli. The left PITG activation in HS 1 – HN 1 suggests that proper training in unfamiliar letters must include phonology to elicit activation increase. Our previous fMRI studies have established that the bilateral STG and the superior temporal sulci (STS) are critically involved in tasks that explicitly require target detection of speech stimuli (Hashimoto et al., 2000; Su- zuki and Sakai, 2003). Given that a portion of these regions is also related to visual language processing such as lip-reading (Calvert et al., 1997), it is likely that STG and STS, as well as the adjacent MTG, also play pivotal roles in crossmodal integration of letters and speech sounds. Indeed, the bilateral STS regions have previously been suggested to be critical sites where the phonemes of speech and the graphemes of letters are integrated (Raij et al., 2000). The findings in the present study are consistent with those in these studies, in that the KS condition elicited larger activation than the HS condition, which may reflect the successful audiovisual integration of letters and speech sounds, given that kana letters had already been acquired in the present sub- ject group. In contrast to the case in the STG/MTG, the left PO showed larger activation under the HS condition than under the KS condition. The left PO and the adjacent AG have been considered to function in a letter-to-sound conversion according to a neuropsychological model of reading (Geschwind, 1979). A recent imaging study indicated that this region is part of the network for verbal working memory (Jonides et al., 1998). In either case, selective activation under the HS condition in the left PO may reflect controlled processes given rise to by insufficient reading skills. A previous imaging study re- vealed reduced activation in both the left PITG and the left PO in dyslexics (Shaywitz et al., 1998). Moreover, it has been reported that the left parietal cortex of dyslex- ics is functionally disconnected from the left striate, extrastriate, and occipitotemporal cortices, when per- forming nonsense word reading tasks (Horwitz et al., 1998; Pugh et al., 2000). These observations are consis- tent with the present results, in that the enhancement of functional connectivity between the left PITG and the left PO was selective to the HS condition, which involved a new link to phonology. Within the visual system, effec- tive connectivity is known to be modulated by attention (Büchel and Friston, 1997) and paired associates learn- ing (Büchel et al., 1999). Therefore, we suggest that the dynamic modulation of the functional connectivity between the left PITG and the left PO is critically involved in learning the association between visual stimuli and phonology.
Conclusion
The present study revealed dynamic plasticity in the adult brain for forming a new link between orthography and phonology, as demonstrated by an enhanced func- tional connectivity between the left PITG and the left PO, as well as by an activation increase per se in the left PITG that was correlated with performance improve- ment. These cortical mechanisms would be critical for adult learners to attain literacy, whereas, as suggested by previous imaging studies (Castro-Caldas et al., 1998; Turkeltaub et al., 2003), different systems might be em- ployed in children when they are first learning to read letters of the alphabet. In contrast to the plasticity of the left PITG, we found that the adjacent FG is immune to short-term training in letters. The lack of plasticity in this region may explain the stable reading proficiency that adult leamers have already achieved. In claiming that the FG is recruited in processes including already acquired orthography, we are not making any claim about its exclusivity. The FG region is also known as a color center (Sakai et al., 1995; McKeefry and Zeki, 1997) and a face area (Kanwisher et al., 1997), but it is possible that this region is included in distributed and overlapping cortical representations for some other visual features (Haxby et al., 2001). Taken together, the present results provide the initial step toward understanding the neural mechanisms that underlie the scope and limits of learn- ing new letters in adulthood. Research in this direction could contribute to the neuroscience of education and could be utilized to develop effective training programs for language learners and for patients with language dis- abilities.
Experimental Procedures
Subjects
Twelve native Japanese speakers (nine males and three females, aged 18–27, all right-handed) participated in Experiment I. We re- cruited 12 matching native Japanese speakers (ten males and two females, aged 19–29, all right-handed) for Experiment II. These sub- jects showed more than 80% accuracy on average for each day under the NN conditions. They participated in fMRI sessions over two consecutive days and slept normally during the night of Day 1. Informed consent was obtained from each subject after the nature and possible consequences of the study had been explained. Ap- proval for these experiments was obtained from the institutional review board of the University of Tokyo, Komaba.
Task Designs and Stimuli
All visual stimuli were shown in yellow against a dark background. No successive repetition of identical stimuli appeared in a string of three visual stimuli. The subjects responded by button press, judg- ing whether strings of visual stimuli matched auditory stimuli. The audiovisual stimuli were presented with a sound delivery system and an eyeglass-like MRI-compatible display (VisuaStim XGA; Reso- nance Technology, Inc., Northridge, CA). All tasks were carried out in a block design. Under the KS and HS conditions, we tested the following nine syllables: /a/, /i/, /o/, /ka/, /ki/, /ko/, /ma/, /mi/, and /mo/, each of which was unambiguously associated with one of nine kana letters and one of nine Hangul letters (Figure 1B). These speech stimuli were digitally synthesized (16 bit; the normal audio cut-off, 11,025 Hz) using speech synthesis software (Oshaberi-mate; Fujitsu, Tokyo, Japan). Under the HN condition, we tested eight pairs, each com- prised of a Hangul letter and a nonspeech sound (Figure 1B). Each Hangul letter was associated with one of the following nonspeech sounds: low-frequency pure tone (T-L, 400 Hz), high-frequency pure tone (T-H, 800 Hz), low-frequency white noise (N-L, the low-pass cut-off at 500 Hz), and high-frequency white noise (N-H, the low- pass cut-off at 4000 Hz). Hangul letters for pure tone and white noise could appear in a single string, as shown in Figure 1A, but each string of nonspeech sounds consisted of either three pure tones or three white noises with a gap of 80 ms. Under the NN condition, the geometric patterns of downward triangles were asso- ciated with N-L, whereas those of upward triangles were associated with N-H (Figure 1B). Before the scanning sessions in Experiment I, the subjects under- went initial exposure to the nine Hangul letters used for HS;. First, the subjects were shown the Hangul letters through simultaneous presentation of a single Hangul letter and its corresponding syllable three successive times, at a rate of 1 Hz. Second, they judged whether each letter was correctly followed by its corresponding syllable at a rate of 1 Hz. After reaching a 90% accuracy level for two consecutive sessions of 20 trials, the subjects proceeded to scanning sessions in the same day and received no further training outside the scanner. Subjects completed this initial exposure in 3.1 ± 0.9 sessions (mean ± standard deviation, n − 12). Before the scanning sessions in Experiment II, the subjects underwent initial exposure to the eight Hangul letters used for HN 11 , in which they performed the same audiovisual matching task as that performed in the scanning sessions and at the same presentation rate. After reaching a 70% accuracy level for one session of 20 trials, the subjects proceeded to scanning sessions in the same day and re- ceived no further training outside the scanner. They completed this initial exposure in 1.8 ± 1.1 sessions. Each subject underwent ten scanning sessions per day. In a single scanning session of Experiment I, KS 1 and HS 1 blocks were alter- nately presented twice, with an intervening NN, block between them as a baseline task. In a single scanning session of Experiment II, KS n and HN n blocks were alternately presented twice, with an intervening NN n block between them. Each of the KS n , KS n , HS t , and HN n blocks consisted of nine trials, whereas each of NN t and NN t blocks con- sisted of six trials. There were two types of scanning sessions, during which auditory stimuli followed or preceded visual stimuli. The order of KS t , HS t , and HN t , blocks, as well as that of the two types of scanning sessions, were counterbalanced within and across subjects. During these scanning sessions, as well as during the initial exposure sessions, no feedback on performance was given to any subject.
fMRI Data Acquisition and Analyses
The fMRI scans were conducted using a 1.5 T scanner (Stratis II, Premium; Hitachi Medical Corporation, Tokyo, Japan). With a gradi- ent echo echo-planar imaging sequence (repetition time, 4 s; echo time, 50 ms; acquisition time, 1750 ms; resolution, 3 × 3 mm²), we scanned over 16 horizontal slices, each 6 mm thick with a 1 mm gap, covering from z − −49 to z − 62 mm. The scanning sounds were confined within the interstimulus interval by using a clustered volume-acquisition sequence. For analyses of fMRI data, we used statistical parametric mapping software (SPM99, Wellcome Depart- ment of Imaging Neuroscience, London, UK). Sessions that included data with a translation of more than 2 mm in one of the three direc- tions or with a rotation of more than 1.5 were removed. We also removed the sessions in which accuracy under the KS condition was below 60% from the analyses. The data were normalized to the standard brain, resampled every 3 mm using bilinear interpola- tion, and smoothed with an isotropic Gaussian kernel of 8 mm full width at half maximum. Task- and/or day-specific effects were esti- mated with a general linear model (fixed effects model) using a boxcar waveform convolved with the canonical hemodynamic re- sponse function. The significant activation was determined by using the ANOVA or t statistics on a voxel-by-voxel basis (corrected p < 0.05, with an extent threshold of nine voxels). For the use of an inclusive mask of KS 1 − NN 1 , an exclusive mask of NN 1 − HS 1 (uncor- rected p < 0.05) was always applied to eliminate the nonspecific activities that occur during unconstrained states (Binder et al., 1999; Raichle et al., 2001). For the analyses of functional connectivity, we first identified ROIs by the comparison HS, − KS,. Following the preprocessing proce- dures described previously (Bokde et al., 2001; Homae et al., 2003), we averaged the data for each subject separately for the two session types (auditory stimuli following and preceding visual stimuli, re- spectively) and further collapsed each data set among subjects. We then calculated the representative time courses of the MR signals for the ROI by averaging the time courses of all the voxels within that ROI. After shifting one time point for a hemodynamic delay, the time course data were segmented into task blocks, and the first time point was removed from nine time points in each of the KS, HS, and HN blocks in order to minimize the effect of hemodynamic changes from the preceding NN blocks. A task-specific time course was then constructed for each task by merging the time courses of the four independent blocks. The resultant task-specific time courses therefore contained a total of 32 time points. Finally, we evaluated task-specific temporal correlations between the prede- fined ROI and each voxel in all scanned regions. At each voxel, the correlation coefficient with respect to the ROI was calculated between the time course of one voxel and an averaged time course in the ROI. This procedure resulted in a correlation map for the reference region. The significance level was set at r > 0.56 (uncor- rected p < 0.001, with an extent threshold of nine voxels), which was adopted in our previous study (Homae et al., 2003).
Acknowledgments
We would like to thank Yasushi Miyashita for encouragement; Kei Suzuki, Yoshinori Tatsuno, Fumitaka Homae, and Natsuki Saeki for technical assistance; and Minako Maki for administrative assis- tance. This work was supported by a Core Research for Evolutional Science and Technology (CREST) grant from the Japan Science and Technology Agency (JST) and by a Young Investigators' Grant from the Human Frontier Science Program (HFSP) to K.L.S. Received: May 16, 2003 Revised: February 11, 2004 Accepted: March 4, 2004 Published: April 21, 2004