Time Course of Visual Attention in Statistical Learning of Words and Categories

Time Course of Visual Attention in Statistical Learning of Words and Categories Chi-hsin Chen 1 , Chen Yu 2 ({chen75, chenyu}@indiana.edu) Damian Fricker 2 , Thomas G. Smith 2 , Lisa Gershkoff-Stowe 1 Department of Speech and Hearing Sciences 1 , Department of Psychological and Brain Sciences 2 Indiana University, IN 47405 USA Abstract Previous research indicates that adult learners are able to use co-occurrence information to learn word-to-object mappings and form object categories simultaneously. The current eye- tracking study investigated the dynamics of attention allocation during concurrent statistical learning of words and categories. The results showed that the participants’ learning performance was associated with the numbers of short and mid-length fixations generated during training. Moreover, the learners’ patterns of attention allocation indicated online interaction and bi-directional bootstrapping between word and category learning processes. Keywords: Eye-tracking; statistical learning; word learning; category learning. Introduction Over the past few decades, researchers have found that humans are sensitive to statistical regularities in the environment. People are able to use statistical information in non-linguistic tasks, such as making inferences (e.g., Xu & Denison, 2009) or finding predictive features of complex visual scenes (e.g., Fiser & Aslin, 2001). They can use statistical information in linguistic tasks as well, such as learning phonetic distributions (e.g., Maye et al., 2002), word boundaries (e.g., Saffran et al. 1996b), word and meaning mappings (e.g., Smith & Yu, 2008), and rudimentary syntax (e.g., Gomez & Gerken, 1999). These studies suggest that statistical learning is a domain-general ability in human cognition. An earlier cross-linguistic study conducted in our laboratory (Chen et al., 2009) also showed that adult English and Mandarin speakers were able to use co- occurrence information to learn word-to-object mappings and to form object categories at the same time. However, even though these two groups of learners had comparable performance in learning word-to-object mappings, they showed different levels of sensitivity to the cues associated with category learning. Participants were better at learning the types of regularities that were present in their native language than the ones that were incongruent with their linguistic input. In Experiment 1 of the study, objects from the same category had similar attached object parts and their labels ended with the same final syllable. This syllable-to- category association simulated a prevalent linguistic feature in Mandarin in that the final syllables of object names often indicated category membership. The results showed that Mandarin speakers were able to learn individual word-to- object mappings and to form syllable-to-category associations under cross-situational learning contexts. On the other hand, English speakers tended not to use the final syllables of labels as cues in category learning. In Experiment 2 of that study, the category markers were moved to the beginning of labels to simulate a more frequent feature in English (e.g., the adjectives in noun phrases). As the structures of the training stimuli were more congruent with the input in the naturalistic environment, the English speakers’ category learning performance became significantly better. More importantly, they also had better performance in the word learning task. One possible explanation of the improvement of word learning performance is that category learning bootstraps word learning. That is, learning which objects belong to the same category helps the learners to focus on relevant features of the stimuli and to rule out certain distractors as possible referents of a word. However, from the design of that study, we were not able to draw a conclusive link between the English speakers’ success in forming categories and their improvement in word learning. The present study was designed to address this issue by using eye-tracking techniques. Category learning studies using eye-tracking techniques have shown that learners generally attend to all possible dimensions early in learning. But during the process of learning, they gradually shift their attention to relevant dimensions (e.g., Rehder & Hoffman, 2005; Blair et al., 2009). Based on previous studies, similar patterns might be observed in statistical word learning and category learning. Our prediction is that at the beginning of training, learners will pay attention to all objects on the screen when hearing a word. Across learning, they will gradually tune their attention to the most probable referent of a word. Moreover, after successfully forming a few word- to-object mappings, the learners should notice that the objects (and their labels) can be grouped into different categories, each having its own distinctive feature. After establishing primitive category structures, the learners should then use this information to rule out certain distractors as possible referents of a word. The goals of the current study are to examine the dynamics of attention allocation in statistical learning of words and categories and to investigate the real-time interaction between word learning and category formation. Method Participants Participants were 23 undergraduates (14 females, mean age: 19.1 years) who received course credit for volunteering.