Modeling Continuous Representations in Visual Working Memory

Modeling Continuous Representations in Visual Working Memory Johannes Lohmann (johannes.lohmann@uni-tuebingen.de) Martin V. Butz (martin.butz@uni-tuebingen.de) Cognitive Modeling, Department of Computer Science, Department of Psychology, Sand 14, T¨ubingen, 72076, Germany Abstract Visual working memory (VWM) is a crucial part of our cogni- tive system. Currently there is an active debate how the appar- ent limitations of VWM should be described. Limited-slot and flexible-resource theories are discussed, but so far the tempo- ral dynamics of representations stored in VWM are not fully understood. In this paper we present data that supports the no- tion of dynamic VWM contents with changing precision. To account for these observations in a qualitative way, we propose a neural network that is able to account for emerging capacity limits as well as for changes in the precision of stored infor- mation. Keywords: VWM; Visual Attention; Neural Network Introduction Successful interaction with our environment requires the stor- age and maintenance of task-relevant information. Planning of behavior as well as evaluation of the outcome would not be possible without this ability. The fact that we are able to perform these tasks indicates that there is some kind of inter- face between our environment and cognition. This interface is provided by the working memory system. Due to its im- portance in linking cognition to the external world, working memory has been studied extensively for decades. An apparent limitation of working memory and especially visual working memory (VWM) with respect to the amount of preserved information has been observed throughout the years (see Miller, 1956 for an early review). Until today it is still in discussion how these limitations might be charac- terized. While limited-slot theories (Zhang & Luck, 2008) state that the maximum number of stored representations is limited and cannot be increased by decreasing the precision of individual representations, flexible-resource theories (Bays & Hussain, 2008) assume a mnemonic resource that can be used to either store a large number of low-precision represen- tations or a small number of high-precision representations. The differentiation between both theories is fundamental as the alternatives suggest rather different bases of cognition. There is empirical evidence for both alternatives (see Fukuda, Awh, & Vogel, 2010 for a recent review), even if resource models seem more plausible from a modeling per- spective (Berg, Shin, Chou, George, & Ma, 2012). Recent studies investigated the neural basis for capacity limits, but again the results are mixed. While Anderson, Vogel, and Awh (2011) found behavioral as well as electrophysiological evi- dence for a limited number of slots in humans, Buschman, Siegel, Roy, and Miller (2011) reported mixed evidence in rhesus monkeys. The results imply the existence of discrete slots, containing some kind of resource that determines the precision of the stored representations. To sum up, it remains unclear if capacity limits of VWM can be described in terms of slots or resources, but the em- pirical evidence indicates that it is unlikely that a strict re- source or slot model will be the consensus – some kind of mixed model seems most probable. Interestingly, in contrast to the limitations with respect to the number of stored items, the temporal dynamics of VWM contents are far less investi- gated. Also in this respect the results are inconclusive. Zhang and Luck (2009) reported clear evidence for abrupt loss of information, whereas Salmela, Lahde, and Saarinen (2012) found evidence for the gradual loss of information. The re- sults of Zhang and Luck (2009) are more in line with a slot model. The gradual decay observed by Salmela et al. (2012) is more in line with a resource model, in which the precision of the representations degrades over time. To investigate the temporal dynamics of information maintained in VWM we applied the same experimental paradigm as Zhang and Luck (2009). Furthermore, we developed a neural network model that can qualitatively account for the observations. In the next section we describe the experimental setup. Next we report the obtained results. After this we give an outline of the neural model. A short discussion concludes the paper. Experimental Setup Since the influential study of Luck and Vogel (1997), change detection paradigms have become the standard approach in VWM research. As it was highlighted by Brady, Konkle, and Alvarez (2011), however, change detection paradigms do not allow to assess the precision of the representation that under- lies the response of the participants. Zhang and Luck (2009) proposed a paradigm that allows to obtain a measure for the precision of the stored representations. Participants had to remember Fourier descriptors which varied continuously in their phase (see Fig. 1 for the trial se- quence and the stimuli). After the presentation of the stim- uli a response screen appeared after a variable interstimulus interval (ISI). The position of one of the presented shapes was highlighted and participants had to indicate the presented stimuli on a “shape-wheel” containing the whole shape di- mension. If the critical shape was in working memory, par- ticipants should report a shape close to the original shape, the response distribution should be bell-shaped and its deviation would be a measure of the precision of the representation. Without a representation participants should guess. Guess- ing should follow a uniform distribution. Together these pro- cesses result in a mixed distribution, consisting of a bell- shaped distribution reflecting the precision (referred to as σ