Coherence in the Visual Imagination: Local Hill Search Outperforms Thagard's Connectionist Model

Coherence in the Visual Imagination: Local Hill Search Outperforms Thagard’s Connectionist Model Michael O. Vertolli (michaelvertolli@gmail.com) Jim Davies (jim@jimdavies.org) Carleton University, Institute of Cognitive Science, 1125 Colonel By Dr. Ottawa, ON K1S 5B6 Canada Abstract A cognitive model of the visual imagination will produce “incoherent” results when it adds elements to an imagined scene that come from different contexts (e.g., “computer” and “cheese” with “mouse”). We approach this problem with a model that infers coherence relations from co-occurrence probabilities of labels in images. We show that this algorithm’s serial traversal of networks of co-occurrence relations for a particular query produces greater coherence than one leading model in the field of computational coherence: Thagard’s connectionist model. Keywords: imagination; coherence; artificial intelligence Introduction The imagination is implicated in a wide range of abilities related to human cognition. The list of abilities includes but is not limited to planning, problem solving, hypothetical thinking, counterfactual thinking, theory of mind, and mental time travel (Davies, Atance, & Martin Ordas, 2011). Despite a plethora of research on the imagination as a facilitator for these abilities (see, for example, Markman, Klein & Suhr, 2012), the generative capacity of the imagination is an untapped area of research in this domain. This work focuses on the visual faculty as it is the most studied. When someone constructs a visual scene with the imagination (e.g., a mouse eating a piece of cheese) they might use visual memories from many different experiences as the components of the new scene. How these components are selected from the range of possible experiences is not obvious. If more than just the mouse and cheese are included in the scene, it is unclear what makes the selection of some elements (e.g., a cat, mousetrap, floorboards, or a countertop) more likely or appropriate than others (e.g., a rollercoaster, map of Spain, or cruise ship). What is known is that people do not arbitrarily select the components for their imaginings, even if those imaginings are entirely fictional (see Cockbain, Vertolli, Davies, in press). There is an intuitive coherence imposed on imagined scenes that inhibits unusual and sometimes even highly creative combinations. One way that humans might make this selection is through the co-occurrence of objects in visual memory (by visual memory we mean only the memory of visual things, and not a specific subsystem like the visuo-spatial sketchpad). Thus, when one is imagining a scene given an environmental query (e.g., a novel, question, or problem), mental processes might search visual memory for other objects that often occur with that query. Recent research in cognitive neuroscience supports this co-occurrence view. Under the title of “scene construction theory,” this research has demonstrated that the hippocampus plays a primary role in the construction of a “coherent spatial context” for the integration of the components of imagined experiences among other cognitive phenomena (Hassabis & Maguire, 2007; Maguire & Mullally, 2013). Parallel research on the role of the hippocampus in the emergence of conceptual knowledge and knowledge transfer (see Kumaran, Summerfield, Hassabis & Maguire, 2009) endorses the “memory space” hypothesis of hippocampal function. In this view, neurons in the hippocampus encode the co-occurrence of the components of a given experience or event through their spatiotemporal associations (Konkel & Cohen, 2009). This means that objects can co-occur as a consequence of associations due to spatial relationships (e.g., mice often are seen next to cheese) or temporal relationships (e.g., gunshots are often followed by death or injury). We only address the former in the current work. Figure 1. An incoherent scene generated by SOILIE for the query ‘mouse,’ containing elements from both the computer and animal senses of ‘mouse. ’ In sum, when one imagines a mouse eating a piece of cheese, it is not surprising that mousetraps, cats, etc. are more likely to come to mind than unrelated elements. They all often occur together in the world and, thus, are associated in the brain. In this case, “a mouse eating a piece of cheese” serves as the query and the other elements are what are returned by some imagination process. One way to further explore this idea is through the use of computational models.