Grounding Meaning in Affordances

Consider which of the following sentences is sensible, and which is less so: la. After wading barefoot in the lake, Erik used his shirt to dry his feet. lb. After wading barefoot in the lake, Erik used his glasses to dry his feet. Sentence la is sensible, whereas entence lb does not make much sense. But why? Both sentences are grammatical. Both meet obvious semantic constraints (e.g., shirts and glasses are inanimate). Both sentences generate coherent, integrated, well-formed propositions. Finally, the sentences do not differ in the associations among the terms. That is, both "shirt" and "glasses" are associatively unrelated to "dry. " The ability to discriminate between sentences such as these is taken as support for the Indexical Hypothesis. The hypothesis is based on three claims about how we understand words and sentences. First, words and phrases are indexed to objects in the world or to analogical representations of those objects such as pictures or perceptual symbols (Barsalou, under review). Second, we derive affordances from the objects. Third, and most importantly, the affordances, not the words, constrain the way ideas can be coherently combined or meshed. Affordances (Gibson, 1979) are ways in which perceiver with a particular type of body can interact with an object. For an adult, a chair affords sitting, standing-on to change a light bulb, and throwing to use as a weapon. For a child, a chair affords sitting and standing on (to get to the cookies), but not throwing. For an agile person, a floppy disk affords grasping and it affords scratching. A thread also affords grasping, but it does not afford scratching. Because we use affordances to understand sentences, we judge a sentence such as "Danielle used a small floppy disk to scratch her back" as sensible and understandable: Everyone knows that a floppy disk can be used to scratch one’s back even if no one has ever tried it or read about it. Similarly, everyone knows that the sentence "Danielle used a short length of thread to scratch her back" does not make much sense. According to the Indexical Hypothesis, grammatical constructions direct the combination, or mesh (Glenberg, 1997), of affordances through a process of envisioning. That is, if we can envision how the combination of affordances can accomplish the goal described by the sentence, then we understand the sentence and judge that it makes sense. Because humans can grasp and manipulate a floppy disk, and because floppy disks are rigid enough to afford scratching, we can envision how Danielle can use the disk to scratch her back. In contrast, because short lengths of thread are not rigid enough to afford scratching, we can make little sense of the alternative sentence. Consider another example taken from a compass learning task used in one experimental investigation of the Indexical Hypothesis. In using a compass to identify landmarks, one step is "Point the direction of travel arrow at the mountain. " Comprehension of this sentence requires an understanding of what it means to point, but how one points depends in part on the affordances of what one is pointing with. It means one thing to point an index finger, another to point a rifle, another to point a car, and quite another to point a direction of travel arrow. Do we have in memory a huge listing of what it means to point, and select the appropriate meaning based on a search of memory? Perhaps, but consider this alternative. Our knowledge consists of a general notion of what it means to point (aligning the long axis of an object with a reference, or to-be-pointed-at, object). Also, we have a repertoire of bodily skills, such as how to grasp and turn objects of different sizes and hefts. To understand the sentence about pointing the direction of travel arrow, we need to index the words ("direction of travel arrow") to the actual object so that its affordances are made available. Then,we can mesh our general knowledge of pointing, the affordances of the direction of the travel arrow, and our repertoire of bodily skills to envision