CREST WorkshoponLanguage Acquisition and Brain Functional Imaging
The University of Tokyo, Komaba, Japan
June 10-12, 1999
ProgramJune 10 (Thursday)
10:00 − 10:10

Opening address
Shunichi Amari
Director, Brain-Style Information Systems ResearchGroup, RIKEN Brain Science Institute, Japan

10:10 − 11:00

Introduction to functional imaging studies of language
information processing
Kuniyoshi L. Sakai
Department of Cognitive and Behavioral Science,
The University of Tokyo, Komaba, Japan

11:00 − 12:00

The anatomical organization of the brain for language
Albert M. Galaburda
Emily Fisher Landau Professor of Neurology and
Neuroscience, Harvard Medical School, U.S.A.

12:00 − 13:30< Lunch Break >13:30 − 14:30

Cognitive neuroscience of speech sound processing
Alec Marantz
Department of Linguistics and Philosophy,
Massachusetts Institute of Technology, U.S.A.

1

14:30 − 15:30

From speech perception to word learning
Janet F. Werker
Department of Psychology, The University of British
Columbia, Canada

15:30 − 16:00< Coffee Break >16:00 − 17:00

Bilingualism: Learning and mastery of two languages
Núria Sebastián GallésDept. de Psicologia Basica, Universitat de Barcelona,
Spain

17:00 − 18:00

Cortical Activity and Language Acquisition
Jacques Mehler
Directeur d'Etudes l'Ecole des Hautes Etudes en Sciences
Sociales, Paris, France

18:00 − 19:30< Welcome Reception >June 11 (Friday)
10:00 − 17:00

Meeting of the “Cortical Plasticity and Linguistic Constraints”
project group (closed), phase one

June 12 (Saturday)
10:00 − 17:00

Meeting of the “Cortical Plasticity and Linguistic Constraints”
project group (closed), phase twoThis workshop is under the auspices of Japan Science and Technology Corporation (JST),Tokyo, Japan.

2Introduction to functional imaging studies oflanguage information processing
Kuniyoshi L. Sakai
Department of Cognitive and Behavioral Science, The University of Tokyo, Komaba, Japan

In this introductory talk, I will summarize relative advantages of functional
imaging techniques and their possible application to language studies. Because
none of these techniques are “perfect,” we have to combine multiple techniques
to overcome their limitations. For example, functional magnetic resonanceimaging (fMRI) is useful for functional localization only in adults, whereas
optical topography (OT) can be applied to infants. Moreover, these imaging
techniques should be combined with anatomical studies for understanding both
structural and functional bases of language information processing.

My current working hypothesis is that the human auditory cortex is speciallydesigned to work as a part of Language Acquisition Device. Therefore it isessential to study the auditory cortex to elucidate language information
processing. Here I will present our recent fMRI study, which demonstrates
attentional influence on multiple auditory areas using the dichotic speech
recognition paradigm. We tested two experimental conditions: diotic listening(DIO) and dichotic listening (DIC). In a single run, DIO and DIC blocks werealternately presented and those blocks were intervened by a control condition, in
which white noise and tone were presented. We have identified at least nine
auditory areas in the left superior temporal cortex. Speech sound-selective
cortical responses emerge even in A1, as well as A2 and Wernicke’s area. We
found clear differential effects of attention on these multiple auditory areas. The
degrees of attentional modulation of cortical activity indicate the progressive
stages of speech recognition in the auditory cortex, which are further segregatedinto multiple pathways.
3

The anatomical organization of the brain for language
Albert M. Galaburda
Emily Fisher Landau Professor of Neurology and Neuroscience, Harvard Medical School, U.S.A.

Although in the past few years there have been no paradigmatic changes in our
knowledge of how the cerebral cortex is organized, our understanding of how
language maps on to this anatomy has indeed changed. This is due to a large
extent to findings made by functional imaging in normal subjects. In this lecture
I will review what is known about the organization of the cerebral cortex andrelevant subcortical structures with respect to language function. In hearing
subjects the auditory system figures importantly in this organization; theextension of the auditory system includes most of the superior temporal gyrus, a
far greater area that depicted in classical maps. Auditory areas on the superior
temporal gyrus, including some quite anterior near the temporal pole, maintainorderly relations with visual areas in the middle and inferior temporal gyri, as
well as with a number of parietal and frontal areas. Furthermore, there is a
special interrelation between the perisylvian areas involved in language anddorsomedial fronto-parietal regions involved in motivation for and initiation of
linguistic activity. Relations between the cortical auditory areas and thethalamus are also more widespread than commonly understood. Finally,
interhemispheric connectivity plays a role in the interindividual variability seen
in cerebral dominance for language. When this broader anatomical picture istaken into account, it is easier to understand why functional studies activate
areas that fall outside the 'classical' language areas.

4Cognitive neuroscience of speech sound processing
Alec Marantz
Department of Linguistics and Philosophy, Massachusetts Institute of Technology, U.S.A.

I will report the results from two recent MEG studies on speech perception.

Magnetic Mismatch Field elicited by phonological feature contrast
Colin Phillips, Thomas Pellathy, Alec Marantz

Phonological theory shows that the phonemes of a language are not primitives, but arecomposed of subphonemic units known as features. The phonemes /b/, /d/, and /g/ differ inplace of articulation but share the feature [+voice], which distinguishes them from thephonemes /p/, /t/, /k/, which are all [-voice]. Existing research has shown that the generator of
the Magnetic Mismatch Field (MMF) in human auditory cortex accesses representations ofboth the discontinuous acoustic sensitivity which underlies speech perception (Näätänen et al.
1997), and phonemic categories themselves (Phillips et al. 1998). The current study
investigates whether the generator of the MMF can also access subphonemic feature
representations, specifically the feature [voice].

Evoked magnetic fields were recorded using a 64-channel whole head magnetometer while
subjects passively listened to sequences of standard and oddball stimuli. [+voice] stimuli werechosen randomly from a set of 12 synthetic syllables, consisting of /ba/, /da/, and /ga/ eachwith 4 different voice onset times (VOT); [-voice] stimuli were chosen randomly from acorresponding set of examples of /pa/, /ta/ and /ka/. There was no many-to-one ratio among
stimuli at either the acoustic or phonemic level, but there was at the level of phonologicalfeatures.

Comparison of responses to standard and deviant stimuli revealed a MMF component which
could only be accounted for if the generator of the MMF has access to phonological feature
representations. The MMF was statistically significant only in the left hemisphere.

5Is speech mode of auditory cortex related to knowledge ofarticulation/acoustic mappings?
David Poeppel, Colin Phillips, David Embick, Krishna Govindarajan, Alec Marantz

The sound categories of language are simultaneously perceptual and articulatory. There ismuch evidence that we hear according to the way that sounds are made and that we articulate
speech sounds to create reliable acoustic categories. The sound categories of speech, then are
well suited to the human articulatory and auditory/perceptual systems. Much of the researchon the initial state of a child and on brain plasticity for speech perception has explored thesound space of actual human languages, i.e., the sounds that are within the articulatory andperceptual borders of language. We must ask whether the learning mechanisms used by thechild are tuned to the sounds within a space delimited by properties of the human articulatory
tract or whether any sounds with a formant structure and with timing properties similar tohuman speech will trigger the same category and contrast learning as speech.

As background to investigating these questions about the initial state of the child, the present
study examines whether the adult auditory cortex is specially tuned to sounds that could be
produced by a human articulatory system. Previous research from our group has suggested
that the M100 latency in the auditory evoked MEG response correlates with the speech vs.
non-speech status of the stimuli: for speech sounds, M100 latency is a function of the formant
structure of the sounds while for non-speech, M100 latency is a function of the fundamental
frequency. The present experiment tests the scope of this generalization and also tests
whether the speech/non-speech distinction for auditory cortex is related to ecologicalplausibility, i.e., to what a human vocal tract can produce. Subjects are presented with sounds
from a continuum between a 2-formant /a/ and a non-speech perceived-pitch stimulus withenergy concentrated at what corresponds to F2 of the vowel. The continuum is produced byraising the relative energy at F2 over F1 for the vowel, simultaneously spreading the
bandwidth of F2. As the energy at F2 increases, the sound becomes impossible for a humanvocal tract to produce (and is perceived as a complex tone with a pitch at the fundamental
frequency of the vowel). We predict a shift in M100 latency from a latency determined by F1
to a latency determined by F0 at the point in the continuum where the sound stops being
perceived as a vowel.

6

From speech perception to word learning
Janet F. Werker
Department of Psychology, The University of British Columbia, Canada

During the first year of life, infants become exquisitely tuned to just those phonetic
differences which are used to distinguish meaning in their native language. This languagespecific tuning would seem to ideally prepare the infant for the task of mapping sound on tomeaning. However, in recent work we reported that although infants of 14-months who are in
the initial stages of word learning are easily able to discriminate native-language contrasts in
simple perception tasks, when they attempt to associate minimal pair words to two different
objects, they become confused. This suggests a functional reorganization in the use ofphonetic detail at the onset of word-learning. In ongoing work, we have been using bothbehavioural and ERP methods to ascertain whether this lack of sensitivity at 14-months
reflects a deficit in the phonetic detail in the representation of words, or instead if it simply
reflects a difficulty in access. As well, we have been probing to see whether older infants,
who are more proficient word learners, can learn to associate minimal pair nonsense words todifferent objects.

To examine the question of representation vs. access, we chose to conduct an ERP study incollaboration with the labs of Neville and Mills. In previous work, Coffey-Corrina, Mills, andNeville (1995) found that infants of both 13-17 and 20-months show different infant ERP
responses to known and unknown words. In our work, we tested infants of 14- and 20-monthsof age on their ERP response to known, phonetically dissimilar unknown, and minimal pair
nonsense words. At 14-months the ERPs to the minimal pair unknown words patterned likethe ERPs to known words, indicating that in its earliest stages, the lexicon may not include
fine phonetic detail. By 20-months, the ERPs to the phonetically similar unknown words
looked like those of the phonetically dissimilar unknown words, indicating that the brain wasable to use fine phonetic detail to distinguish comprehended from unknown words. On thebasis of these ERP results, we conducted a behavioural word-object association task with
infants aged 20-months. As predicted by the ERP results, we found that infants of 20-months
have no difficulty mapping minimal pair words to two different objects. The match between
the behavioural and ERP work points to exciting new directions for study.
7

Bilingualism: Learning and mastery of two languages
Núria Sebastián GallésDept. de Psicologia Basica, Universitat de Barcelona, Spain

Research performed cross-linguistically has shown that speakers of different languages
process the speech signal in quite different, language-specific ways. Moreover, they seem toapply maternal language knowledge when listening to foreign languages. But, what is it this“maternal language knowledge” for individuals raised in bilingual environments? In this talk,I will present some data analyzing the way in which people become bilinguals and how thesepeople process and represent the two languages they have.

Though there are different lines of research dealing with this issue, I will concentrate my
exposition on two specific domains. The two topics that I will expand both refer tophonological processing:(a) tuning to the maternal language:- language discrimination abilities in young infants (4 month olds) by monolingual andbilinguals- acquisition of phonotactic constraints (10 month olds) by monolingual and bilinguals(b) phonemic representation in adult bilinguals.

There are many different types of bilinguals, but we are interested in studying two types ofthem:(1) people who during the first years of their lives had only been exposed to one language andthat from age 3-4 onwards, they have been intensively exposed to two languages (and who
currently use both languages in their everyday life)(2) people who have been exposed to both languages from birth

Our hope is that the first type of bilinguals will allow us to address the issue: to which extent
can a second language be learnt when early + intensive exposure to a second language is
produced (but the first one has already been established)? Can L2 override (modify) the initialvalues of L1? The second type can allow us to address a different issue: is it possible to learn
two languages at the same time (or our language learning “device” is designed to learn justone? Is it the same to learn one and two languages from the beginning?
8

Cortical Activity and Language Acquisition
Jacques Mehler
Directeur d'Etudes l'Ecole des Hautes Etudes en Sciences Sociales, Paris, France

During language acquisition the first observable adjustment to the infant’s
perceptual processing system is determined by the prosodic features of theparental language. In the last ten years several studies have established that atbirth infants discriminate any two languages that have very different prosodic
structures. How can the prosodic differences between two languages bemeasured? Phonologists and psycholinguists postulated that languages
instantiate different timing properties, like syllable-timing, stress-timing, and soforth. Recently a more general hypothesis called the phonological classhypothesis (PCH) has been formulated. This hypothesis asserts that thelanguages of the world cluster into a few groups giving rise to perceptualclasses. We will present evidence compatible with PCH. Clearly, PCH becomesinteresting if it can be shown that some of the essential parameters that have to
be fixed to learn the syntactic properties of one's language, can be correlatedwith the prosodic properties that characterize these classes or clusters.

Over time some of the abilities present at birth change. Two month old infants
do not discriminate two languages if neither is the native language. However,two month olds have no trouble discriminating their native language fromlanguages that belong to a different class even though they no longer distinguishit from other languages in the same class.

By the age of five months, infants have made considerable progress. Indeed,
they now can distinguish their maternal language from other languages in thesame class showing that they have assimilated properties other than the prosodic
ones that characterize the class. It is known that it is about at this time that9

babies begin to move from a phonological to a phonetic characterization of
language. Learning is very important during this period. To illustrate this point I
will present results from studies of infants raised in bilingual environments. I
will argue that similar studies on bilingual infants who are learning two very
different languages are necessary if we are to understand whether one cangeneralize from the study of Spanish/Catalan bilinguals to say English/Mandarin
bilinguals.

In the second part of the talk, I will first show that the vertebrate brain is very
plastic and that re-mapping of the somatosensory cortex can be observed even in
adults. Next I will show that language behavior is far more rigid than the corticalrepresentation underlying behavior. I will review results from brain-imaging
studies showing that the cortical representation of language changes with thedegree of proficiency with which the language is spoken. In particular, I willshow that the greater the proficiency of a bilingual with L2, the more extensive
the overlap between the cortical representations of L1 and L2. Moreover, degreeof proficiency is more important for our understanding of the macrorepresentation of L2 than age of acquisition. We will discuss results obtainedusing PET and fMRI over the last five years.

To conclude, I will argue that we still need to study the stable state of thelanguage used in conjunction with the linguistic dispositions of the human infant
in order to be able to understand how language learning takes place. Moreover,
it is important to compare the infants' dispositions with those in other higher
vertebrates to gain a better understanding of the biological foundations oflanguage.

10