Brain and Language (1998)

One of the most enduring and significant findings from neuropsychology is the left hemisphere dominance for language processing. Studies both past and present converge to establish a widespread language network in the left peri-sylvian cortex which encompasses at least four main regions: Broca’s area, within the inferior prefrontal cortex; Wernicke’s area, within the posterior two-thirds of the superior temporal lobe; the anterior portion of the superior temporal lobe; and the middle prefrontal cortex (14Neville H.J Bavelier D Curr. Opin. Neurobiol. 1998; 8: 254-258Crossref PubMed Scopus (94) Google Scholar). While the language processing abilities of the left hemisphere are uncontroversial, little is known about the determinants of this left hemisphere specialization for language. Are these areas genetically determined to process linguistic information? To what extent is this organization influenced by the language experience of each individual? What role does the acoustic structure of languages play in this pattern of organization? To date, most of our understanding of the neural bases of language is derived from the studies of spoken languages. Unfortunately, this spoken language bias limits our ability to infer the determinants of left hemisphere specialization for human language. For example, we are unable to assess whether left hemisphere dominance arises from the analysis of the sequential/hierarchical structures that are the building blocks of natural languages or rather is attributable to processing of the acoustic signal of spoken language. American Sign Language (ASL), which makes use of spatial location and motion of the hands in encoding linguistic information, enables us to investigate this issue. The comparison of the neural representations of spoken and signed languages permits the separation of those brain structures that are common to all natural human languages from those that are determined by the modality in which a language develops, providing new insight into the specificity of left hemisphere specialization for language. In this paper, we will first review some properties of ASL and then discuss the contribution of the left hemisphere and that of the right hemisphere to ASL processing. The idea that sign languages are not a concatenation of universally understood pantomimes or concrete gestures has been hard to unroot. In fact, just as there are many spoken languages, there are many unique and different signed languages. Recent advances in linguistics have revealed that sign languages such as ASL encompass the same abstract capabilities as spoken languages and contain all the different levels of linguistic representations found in spoken language, including phonology, morphology, syntax, semantics, and pragmatics (12Lillo-Martin D.C Universal Grammar and American Sign Language. Kluwer Academic Publishers, Boston1991Crossref Google Scholar, 5Corina D Sandler W Phonology. 1993; 10: 165-207Crossref Scopus (63) Google Scholar). Thus, similar linguistic structures are found in spoken and signed languages. A number of authors have proposed that the left hemisphere recruitment for language results from a specialization of these areas for the analysis of linguistic structures. By this view, the structural similarity between signed and spoken languages predicts that left hemisphere language areas should also be recruited during ASL processing. On the surface, however, ASL differs markedly from spoken languages. For example, in ASL, phonological distinctions are created by the positions and shape of the hands relative to the body rather than by acoustic features such as nasality and voicing found in spoken languages. The fact that signed and spoken languages rely on different input and output modalities carries important consequences for theories on the origin of the left hemisphere dominance for language. It is often argued that the left hemisphere specialization for language originates from a left hemisphere advantage to execute fine temporal discrimination, such as the fast acoustic processing required during speech perception (20Tallal, P., Miller, S., and Fitch, R.H. (1993). In Temporal Information Processing in the Nervous System, with Special Reference to Dyslexia and Dysphasia, P. Tallal, A.M. Galaburda, R. Llinas, and C. von Euler, eds. (New York: New York Academy of Sciences), pp. 27–47.Google Scholar). By this view, the standard left hemisphere language areas may not be recruited during the processing of visuo-spatial languages such as ASL. Signed and spoken languages also differ by the way they convey linguistic information. While most aspects of spoken languages rely on fast acoustic transitions (e.g., consonant contrast) and temporal ordering of constituents (e.g., suffixation, prefixation, word order, etc.), sign languages make significant use of visuo-spatial devices. For example, the use of signing space as a staging ground for the depiction of grammatical relations is a prominent feature of ASL syntax. As shown in Figure 1, in ASL, nominals introduced into the discourse are assigned arbitrary reference points in a horizontal plane of signing space. Signs with pronominal function are directed toward these points, and verb signs obligatorily move between such points in specifying grammatical relations (subject of, object of). Thus, grammatical functions served in many spoken languages by case marking or by linear ordering of words are fulfilled in ASL by spatial mechanisms; this is often referred to as “spatialized syntax” (12Lillo-Martin D.C Universal Grammar and American Sign Language. Kluwer Academic Publishers, Boston1991Crossref Google Scholar, 18Poizner H Klima E.S Bellugi U What the Hands Reveal about the Brain. MIT Press, Cambridge, MA1987Google Scholar; but see 11Liddell, S.K. (1998). In Language and Gesture: Window into Thought and Action, D. McNeill, ed. (Cambridge: Cambridge University Press), in press.Google Scholar, for an alternative view). Another example of ASL processing that makes special use of visuo-spatial information is the classifier system. Classifiers are morphologically complex forms that often convey salient visual properties of the objects they signify. For example, when talking about a car being parked in the garage, signers may use a “vehicle” classifier, but the orientation and direction of motion of their hands specifies whether the car is parked face-forward or backward in the garage (17Newport, E.L., and Supalla, T. (1980). In Signed and Spoken Language: Biological Constraints on Linguistic Form, U. Bellugi and M. Studdert-Kennedy, eds. (Deerfield Beach, FL: Verlag Chemie), pp. 187–211.Google Scholar). These examples illustrate how unique ASL is in its integration of language and visuo-spatial properties. Interestingly, the right hemisphere appears to play a greater role than the left hemisphere in visuo-spatial processing. Right-lesioned patients exhibit a wide range of visuo-spatial deficits, such as problems in processing different aspects of spatial relationships, route finding, drawing, and visually guided reaching. Moreover, visuo-spatial neglect following right hemisphere lesions tends to be more severe than that following left hemisphere lesions (8Heilman, K.M., Watson, R., and Valenstein, E. (1985). In Clinical Neuropsychology, K.M. Heilman and E. Valenstein, eds. (Oxford: Oxford University Press), pp. 243–293.Google Scholar). The reliance of ASL on visuo-spatial processing raises the possibility of a greater contribution from right hemisphere areas during sign language processing. The next two sections consider in turn the contribution of left hemisphere language areas and right hemisphere areas to ASL processing. The sign aphasia literature is rich in examples of right-handed signers who, like hearing persons, exhibit language disturbances when critical left hemisphere areas are damaged (18Poizner H Klima E.S Bellugi U What the Hands Reveal about the Brain. MIT Press, Cambridge, MA1987Google Scholar, 9Hickok G Klima E.S Bellugi U Nature. 1996; 381: 699-702Crossref PubMed Scopus (119) Google Scholar, 10Hickok G Bellugi U Klima E.S Trends Cogn. Sci. 1998; 2: 129-136Abstract Full Text Full Text PDF PubMed Scopus (63) Google Scholar, 4Corina, D.P. (1998). In Aphasia in Atypical Populations, P. Coppens, Y. Lebrun, and A. Basso, eds. (Mahwah, NJ: Lawrence Erlbaum Associates), pp. 261–309.Google Scholar). In hearing individuals, severe language comprehension deficits are associated with left hemisphere posterior lesions, especially posterior temporal lesions. Similar patterns have been observed in users of signed languages. For example, after damage to posterior temporal structures, patient W. L. (a congenitally deaf signer) evinced marked comprehension deficits, including difficulty in single sign recognition, moderate impairment in following commands, and severe sentence comprehension problems. Similarly, left hemisphere anterior lesions are associated with language production impairment with preserved comprehension in users of spoken languages, and they are also implicated in sign language production impairment. A representative case is patient G. D. (a congenitally deaf signer), who experienced damage to the left frontal lobe, including Brodmann’s areas 44 and 45 (18Poizner H Klima E.S Bellugi U What the Hands Reveal about the Brain. MIT Press, Cambridge, MA1987Google Scholar). G. D.’s signing was effortful and dysfluent and reduced largely to single sign utterances, yet his sign language comprehension remained intact (for a recent review of 21 case studies, see 4Corina, D.P. (1998). In Aphasia in Atypical Populations, P. Coppens, Y. Lebrun, and A. Basso, eds. (Mahwah, NJ: Lawrence Erlbaum Associates), pp. 261–309.Google Scholar). Recent imaging studies have confirmed the left hemisphere participation during sign processing in healthy native signers (deaf or hearing). Recordings of event-related potentials during sentence comprehension have revealed an increased anterior negativity over the left temporal electrodes similar for function words/signs in native English speakers and native signers (15Neville H.J Coffey S.A Lawson D.S Fischer A Emmorey K Bellugi U Brain Lang. 1997; 57: 285-308Crossref PubMed Scopus (177) Google Scholar). The few tomographic studies of native signers unambiguously indicate a recruitment of the standard left peri-sylvian language areas during viewing of ASL sentences (19Soderfeldt B Ingvar M Ronnberg J Eriksson L Serrander M Stone-Elander S Neurology. 1997; 49: 82-87Crossref PubMed Scopus (73) Google Scholar, 16Neville H.J Bavelier D Corina D.P Rauschecker J.P Karni A Lalwani A Braun A Clark V.P Jezzard P Turner R Proc. Natl. Acad. Sci. 1998; 95: 922-929Crossref PubMed Scopus (366) Google Scholar). A recent imaging study in which native signers were asked to imagine the production of signs confirms a strong recruitment of left Broca’s area during sign execution (13McGuire P.K Robertson D Thacker A David A.S Kitson N Frackowiak R.S Frith C.D Neuroreport. 1997; 8: 695-698Crossref PubMed Scopus (85) Google Scholar). These results establish that left hemisphere language areas are recruited by the language system independently of the modality and surface properties of the language and suggest that these areas are biologically determined to process the kind of structure specific to natural languages. The question of right hemisphere involvement in linguistic processing has received new interest recently. While left hemisphere lesions lead to marked impairment on tests of sentence production and comprehension, speakers with right hemisphere lesions show more subtle deficits when tested on such materials (3Caplan D Hildebrandt N Makris N Brain. 1996; 119: 933-949Crossref PubMed Scopus (183) Google Scholar). Hearing patients with right hemisphere lesions are commonly impaired in the processing of prosody, discourse, and pragmatic aspects of language use. Thus, the ability to make inferences regarding the emotional tone of language (affective prosody) or to integrate meanings across sentences and to appreciate jokes and puns in language appears to rely on the integrity of the right hemisphere (see 1Beeman M Chiarello C Right Hemisphere Language Comprehension. Lawrence Erlbaum Associates, Mahwah, NJ1998Google Scholar, for references). The available data indicate that both left and right hemispheres contribute to the processing of the complexities of spoken languages. Interestingly, the right hemisphere processes language rather differently than the left hemisphere. While the right hemisphere appears strongly tuned toward broad-based semantic interpretation and global meaning, the left hemisphere seems necessary for the fine aspects of on-line sentence processing and literal meaning. Worth noting in this context are recent studies that have begun to document a greater participation of the right hemisphere during the comprehension of signs. Event-related potentials during sentence processing in native signers reveal larger bilateral components than in native speakers (15Neville H.J Coffey S.A Lawson D.S Fischer A Emmorey K Bellugi U Brain Lang. 1997; 57: 285-308Crossref PubMed Scopus (177) Google Scholar). A functional magnetic resonance imaging study has further established a larger participation of right temporal areas during the comprehension of ASL sentences than during comprehension of written English (16Neville H.J Bavelier D Corina D.P Rauschecker J.P Karni A Lalwani A Braun A Clark V.P Jezzard P Turner R Proc. Natl. Acad. Sci. 1998; 95: 922-929Crossref PubMed Scopus (366) Google Scholar). As Figure 2 illustrates, the classical left hemisphere dominance for sentence processing was not observed in native signers in an imaging study that compared ASL sentence comprehension to the processing of arbitrary meaningless signs. This result indicates that the right hemisphere recruitment in ASL occurs above and beyond the processing demands of arbitrary gestures. The lack of left hemisphere dominance in this ASL comprehension task contrasts with the left hemisphere dominance that has been consistently observed in imaging studies that compared sentence comprehension to listening to backward speech in native speakers. These findings suggest different final organizations of the brain systems for language in speakers and signers and are a first indication that the cerebral organization for language may be altered by the structure and processing requirements of the language. An outstanding question at present concerns the functional role of right hemisphere areas during ASL. The right hemisphere areas that participate in ASL do not seem to be entirely homologous to those areas that mediate visuo-spatial cognition in general. Striking examples of dissociations between ASL and visuo-spatial cognition are found in the few studies of right hemisphere–lesioned signers. For example, following a right posterior lesion, patient J. H. was unable to recognize visual object stimuli presented to the left visual field but showed preserved performance for ASL signs in the same field (see 6Corina D Kritchevsky M Bellugi U Cogn. Neuropsychol. 1996; 13: 321-356Crossref Scopus (16) Google Scholar, for details). Studies of the neural substrate for nonlinguistic gestures in hearing subjects support a dissociation between the right hemisphere areas for ASL and those involved in the perception of biological motion. In native signers, the comprehension of ASL sentences results in robust and extensive activation of the right superior temporal lobe (Figure 2; 16Neville H.J Bavelier D Corina D.P Rauschecker J.P Karni A Lalwani A Braun A Clark V.P Jezzard P Turner R Proc. Natl. Acad. Sci. 1998; 95: 922-929Crossref PubMed Scopus (366) Google Scholar). In contrast, this area is not recruited when hearing nonsigners perceive meaningful gestures (e.g., combing hair7Decety J Grezes J Costes N Perani D Jeannerod M Procyk E Grassi F Fazio F Brain. 1997; 120: 1763-1777Crossref PubMed Scopus (758) Google Scholar). Overall, available studies support the view of distinct brain systems for ASL and for nonlinguistic visuo-spatial abilities, such as the processing of visuo-spatial and biological motion information. It is likely that the right hemisphere in signers is recruited for prosody and/or discourse functions, as it is in speakers (2Brentari D Poizner H Kegl J Brain Lang. 1995; 48: 69-105Crossref PubMed Scopus (37) Google Scholar, 10Hickok G Bellugi U Klima E.S Trends Cogn. Sci. 1998; 2: 129-136Abstract Full Text Full Text PDF PubMed Scopus (63) Google Scholar). However, as discussed above, the contribution of right hemisphere areas to language processing appears larger in native signers than in native speakers. Do these right areas mediate the processing of linguistic information? Are they recruited because of the visuo-spatial processing demands of ASL? One deficit documented after right hemisphere lesions concerns aspects of ASL syntax that rely heavily on the use of space. As 4Corina, D.P. (1998). In Aphasia in Atypical Populations, P. Coppens, Y. Lebrun, and A. Basso, eds. (Mahwah, NJ: Lawrence Erlbaum Associates), pp. 261–309.Google Scholar reports, right hemisphere–damaged signers show performance well below controls on tests of spatial syntax. However, these patients generally suffered from visuo-constructive and visuo-perceptual deficits; thus, it is unclear whether these deficits in spatial syntax should be treated as a linguistic deficit per se (i.e., as an aphasia) or as a processing deficit owing to disordered appreciation of spatial relations. Recently4Corina, D.P. (1998). In Aphasia in Atypical Populations, P. Coppens, Y. Lebrun, and A. Basso, eds. (Mahwah, NJ: Lawrence Erlbaum Associates), pp. 261–309.Google Scholar has also documented the case of a right hemisphere–damaged signer who suffered impaired comprehension and production of classifier forms but only mild neglect. The limited number of studies on this topic prevents any firm conclusion; however, available evidence suggests it will be important for future research to map the neural substrate that mediates syntactic operations involved in spatialized syntax and the ASL classifier system. The participation of standard left hemisphere language regions to sign processing suggests strong biases that render those left hemisphere regions well suited to processing natural language independently of the surface properties of the language, such as its modality or form. The detailed comparison of the brain systems for signed and spoken languages, however, reveals differences as well. Present studies point to a larger recruitment of right hemisphere areas during the comprehension of signed than that of spoken languages. The recruitment of the right hemisphere in early learners of ASL suggests that the cerebral organization for language can be altered by the structure and processing requirements of the language. Thus, while standard left hemisphere language areas may be common to all natural languages, the final organization of the language system appears to be determined by the exact language experience of the individual.