HANDBOOK OF FUNCTIONAL NEUROIMAGING OF COGNmON

Semantic Representations Thus far, our discussion of the representation of semantic memory has been confined to physical properties of concrete objects. When Allport articulated his theory of the representation of meanings, he limited himself, "for simplicity," to the domain of object concepts. Of course, any complete theory of semantic memory has a bit more work to do: there are abstract concepts (e.g., peace), abstract features of object concepts (e.g., "alive" as a feature of a plant), and abstract relations between concepts (e.g., the ways in which a "bat" is more similar to a "bear" than to a "bird"). There is ample neuropsychological evidence for a dissociation between the representation of abstract and concrete concepts (e.g., Breedin et al., 1994) that may reflect any number of qualitative differences in their acquisition and representational format. Neuroimaging comparisons of abstract and concrete words have identified an inconsistent array of regions associated with abstract concepts in the left superior temporal gyrus (Wise et al., 2000), right anterior temporal pole (Kiehl et al., 1999), and left posterior middle temporal gyrus (M. Grossman et al., 2002). Noppeney and Price (2004) compared fMRI activation while subjects made similarity judgments about triads of visual words, sound words, action words, and abstract words that were matched for difficulty. Relative to the three other conditions, retrieval of abstract concepts activated the left inferior frontal gyrus, middle temporal gyrus, superior temporal sulcus, and anterior temporal pole. The authors suggest that these differences reflect activation of areas involved in sentence comprehension, although this is clearly an area in need of more investigation. There is, to date, even less work addressing abstract features. One particular problem with using functional neuroimaging to compare abstract and co,? crete features is that abstract semantic decisions typically take longer to resolve. The left inferior PFC has been implicated in extended or controlled semantic processing on the basis of studies that might confound effort with abstractness (Roskies et al., 2001); for example, PFC activity increased when subjects decided that "candle" and "halo" were similar, compared to deciding that "candle" and "flame" were similar 165 Functional Neuroimaging of Semantic Memory (Wagner et aI., 2001), arguably a comparison that confounds effort with abstraction. To unconfound these processes, Goldberg and colleagues (2004) compared the effects of increasin& semantic abstractness and increasing difficulty on activity in PFC while subjects verified perceptual or abstract facts about animals. fMRI activation in left PFC (BA 47) was specifically associated with an increased reliance on abstract properties but not increased semantic difficulty. This finding is consistent with recent evidence showing that ne_urons in a primate analogue of this region represent abstract rules (Wallis et aI., 2001). Many cognitive models of semantic memory have described hierarchical networks that reflect abstract relations between concepts (e.g., a tree is a plant and a plant is a living thing). This description is quite different from the distributed representation we have been describing thus far. However, Rogers and colleagues have articulated a fonnal model of semantic memory that includes units which integrate infonnation across all of the attribute domains (including verbal descriptions and object names; McClelland & Rogers, 2003). As a consequence, "abstract semantic representations emerge as a product of statistical learning mechanisms in a region of cortex suited to performing cross-modal mappings by virtue of its many interconnections with different perceptual-motor areas" (Rogers et aI., 2004, 206). The interaction between content-bearing perceptual representations and verbal labels produces a similarity space that is not captured in any single attribute domain; rather, they argue, it reflects abstract similarity (d. Caramazza et aI., 1990; A. R. .,,-, Damasio, 1989; Plaut, 2002; Tyler et aI., 2000). as a candidate for these abstract representations is the temporal pgle, based both on the anatomical connectivity of this region and the degener-atien:lOf this region in semantic dementia. The notion that interactions between perceptual and verbal representations lead to the emergence of new, abstract representations may be relevant for a puzzle that has emerged in neuroimaging tests of Allport's (1985) sensorimotor model of semantic memory: that there is a consistent trend for retrieval of a given physical attribute to be associated withactivatlon 6f'coiifcal et aI. , 1995; Thomps'on-Schill, 2003). This pattern, which has been interpreted as coactivation of the "same areas" involved in sensorimotor processing, as Allport hypothesized, could alternatively be used as grounds to reject the Allport model. What does this anterior shift reflect? We believe the answer may lie in the ideas developed by Rogers and colleagues (2004). The process of abstracting away from modality-specific representations may occur gradually across a number of cortical regions (perhaps converging on the temporal pole). As a result, a gradient of abstraction may emerge in the representations throughout a given region of cortex (e.g., the ventral extrastriate visual pathway), and the anterior shift may reflect activation of a more abstract 166 s. L. Thompson-Schill, L P. Kan, and R. T. Oliver representation (Kosslyn & Thompson, 2000). The tasks that have been used to study conceptual retrieval of visual attributes have not consistently required the subject to retrieve perceptual information. For example, in order to recall that a banana is "yellow," activation of color representations that are more abstract than those necessary for perception could suffice. The conceptual similarity space in more anterior regions may depart a bit from the similarity space in the environment, perhaps moving in the direction of abstract relations. More work is needed to uncover the nature of the representations-and how the similarity space may gradually change across different cortical regions. Categories of Semantic Memory-Redox Thus far, we have presented two potentially orthogonal views about the organization of semantic memory. We initially considered the hypothesis that representations of specific categories of semantic knowledge are instantiated in spatially distinct neural regions. As we saw, there is ample support for this hypothesis from the neuropsychologicalliterature, but only partial support from neuroimaging studies. Then, we reviewed neuroimaging studies that support models of distributed representations of semantic memory, where different attribute domains of object knowledge are represented in distinct sensorimotor systems. As the reader having even a passing familiarity with these literatures will know, these two hypotheses about the organization of semantic memory are intertwined, by virtue of the fact that the taxonomic category of an object and its associated attribute domains are not at all orthogonal. The confound between these two putative organizing principles has made it challenging to uncover the neural architecture of semantic memory. Warrington and McCarthy (1983) first called attention to implications of this relation for the interpretation of category-specific deficits: whereas sensory attributes are important for discriminating between members of the category of living things, functional attributes are more important for discriminating between members of the category of nonliving things. Thus, category-specific deficits could result from the degradation of an attribute domain of semantic memory. Since the mid-1980s, this sensory-functional theory has persisted in a variety of accounts of categoryspecific deficits, all of which hold that semantic knowledge is stored in sensorimotor channels, and that the relative importance of information contained in these channels varies across items in different categories (e.g., Farah & McClelland, 1991; Martin et aI., 2000; Saffran & Schwartz, 1994; Simmons & Barsalou, 2003). Accordingly, one explanation of category-specific activation in neuroimaging studies is that these differences reflect the differential weighting of visual and functional knowledge across categories (e.g., Martin et aI., 1996). In order to test this account, Patterson and colleagues have reported two PET studies that have uncon167 FlUlctional Neuroimaging of Semantic Memory founded object category and attribute domain. In the first, subjects made similarity judgments about living or nonliving things on the basis of either visual or nonvisual information. With this fully crossed design, the authors compared the magnitude of category-specific effects and attribute-specific effects directly, and concluded that the latter were more prominent neurally (Mummery et al., 1998). In a second study, subjects generated visual or nonvisual features in response to an object name (Lee, Graham, et al., 2002). Although they found no category-specific effects, they did find an effect of attribute type: visual retrieval activated left posterior inferior temporal cortex, and nonvisual retrieval activated left middle temporal cortex and right fusiform cortex. The relationship between visual processing demands and object category was elegantly demonstrated in a PET study by Rogers and colleagues (2005). Subjects categorized photographs of animals and vehicles at one of three levels ofspecificity (e.g., animal, bird, or robin; vehicle, boat, or ferry). Posterior fusiform activation was greater for animals than for vehicles only when subjects were categorizing pictures at an intermediate level (e.g., bird). The authors argued that the fusiform gyrus responds to the discrimination of items with similar visual representations, and that at the intermediate level of description only, animals have more overlappin