Recognizing Scenes Containing Consistent or Inconsistent Objects

Recognizing Scenes Containing Consistent or Inconsistent Objects Michael L. Mack (michael.mack@vanderbilt.edu) Department of Psychology, 111 21 st Avenue South Nashville, TN 37240 USA Thomas J. Palmeri (thomas.j.palmeri@vanderbilt.edu) Department of Psychology, 111 21st Avenue South Nashville, TN 37240 USA Abstract How does object perception influence scene perception? A recent study of ultrarapid scene categorization (Joubert et al., 2007) reported facilitated scene categorization for scenes with consistent objects compared to scenes with inconsistent objects. One proposal for this consistent-object advantage is that ultrarapid scene categorization is influenced directly by explicit recognition of particular objects in the scene. We instead asked whether a simpler mechanism that relied only on scene categorization without any explicit object recognition could explain the consistent-object advantage. We combined a computational model of scene recognition based on global scene statistics (Oliva & Torralba, 2001) with a diffusion model (Ratcliff, 1978) of perceptual decision making. Simulations show that this model is sufficient to account for the consistent-object advantage. Importantly, this effect need not arise from explicit object recognition, but from the inherent influence certain objects have on the global scene statistics diagnostic for scene categorization. Keywords: scene categorization; object recognition Introduction What is the relationship between scene perception and object perception? Past research has examined how objects are recognized in consistent or inconsistent scenes (e.g., Biederman, Mezzanotte, & Rabinowitz, 1982; Davenport & Potter, 2004; Palmer, 1975). The general finding is that it is easier to recognize objects in semantically consistent scenes, such as recognizing a toaster in a kitchen compared to recognizing a toaster in a bedroom (Davenport & Potter, 2004; Henderson & Hollingworth, 1999; Palmer, 1975). One proposed mechanism for facilitated recognition of objects contained in consistent scenes is an interacting, dual system account (Davenport, 2007; Davenport & Potter, 2004). At the same time that the object recognition system is extracting information for an object categorization, the scene perception system is extracting evidence for a scene categorization. Object and scene perception systems operate in parallel, sharing information and converging on a full description of the environment, facilitating categorizations that are consistent with one another. The interacting, dual-system account is supported by evidence for scene perception facilitating object recognition. Of course, the converse should be the case as well. Scene recognition can also be influenced by object perception. Indeed, Davenport and Potter (2004) found that scene categorization was facilitated when the scene contained a consistent object (e.g., a football field with a football player) compared to an inconsistent object (e.g., a football field with a priest). A recent study (Joubert, Rousselet, Fize, & Fabre-Thorpe, 2007) reported a similar advantage for scenes containing consistent objects versus inconsistent objects in ultrarapid scene categorization. Participants were presented with scenes for only 26ms and performed a go/no-go decision about the scene’s superordinate category (natural or man- made). As illustrated in Figure 1, scene images either contained objects consistent with the scenes’ category (e.g., an urban street scene with a parked car) or contained objects inconsistent with the scenes’ category (e.g., an urban street scene with a large tree). A post-hoc analysis comparing these two types of scenes showed a consistent-object advantage such that participants made fewer errors and were faster to respond when categorizing a scene containing a consistent object. Joubert et al. explained this consistent- object advantage with the interacting, dual-system account: Information extracted by the object recognition system influences the rapid processing and categorization decision by the scene perception system. For a scene containing an inconsistent object, the object information conflicts with the evidence for the scene’s category, leading to more errors and slower reaction times. Previous work has shown that ultrarapid scene categorization is largely determined by coarse, global scene properties (Oliva & Schyns, 1997; Schyns & Oliva, 1994). Furthermore, computational models that represent scenes based on their global spatial structure are sufficient for ultrarapid scene categorization (Oliva & Torralba, 2001). Importantly, such models capture only the diagnostic global features of scenes without explicitly representing any local content of the scene, such as the location, presence, or identity of particular objects (Greene & Oliva, 2009). The feature set used by these models is based on global image statistics calculated across the entire scene, such as the scene’s spatial frequency content. A possibility suggested by the Joubert et al. (2007) results is that ultrarapid scene categorization based on global image properties is influenced in some way by ultrarapid categorization of particular objects in the scene that are either consistent or inconsistent with the scene’s category. We instead asked whether the consistent-object advantage could be explained by a simpler mechanism that relied only