Evidence for false memory before deletion in visual
short-term memory
Eiichi Hoshino1,2, Ken Mogi2,
1

Tokyo Institute of Technology, Department of Computational Intelligence and Systems
Science. 4259 Nagatsutacho Midori-ku Yokohama-shi Japan 226-8502.
2Sony Computer Science Laboratories, Fundamental Research Laboratory, Takanawa Muse
Bldg. 3-14-13, Higashigotanda, Shinagawa-ku, Tokyo, Japan 141-0022.
eiichi@brn.dis.titech.ac.jp

Abstract. Forgetfulness results in interference and/or deletion. Visual shortterm memory (VSTM) gradually decays as the retention time elapses, causing
forgetfulness. Little is known about forgetfulness in VSTM, while substantial
studies on VSTM have focused on the process of memory encoding, often with
control of attention. Evidences suggest that the prefrontal cortex may contributeto maintain short-term memory during extended retention periods while theposterior parietal cortex may support the capacity-limited store of visual items.
Here we conduct a visual memory experiment to measure the levels and sourceof memory decay. In particular, multiple retention intervals were used between
the presentation of a study array and a cue. The results show that the correct
response to cued objects decreased as retention interval increased while that touncued and novel objects remain unchanged. These data indicate thatforgetfulness in VSTM is primarily due to interference rather than memory
deletion.
Keywords: Visual short-term memory, forgetfulness, retro-cue, interference

1

Introduction

In delayed matching tasks, such as change blindness, there is more failure inrecognition as the retention time increases. Memory of an item to which subjects is
exposed to tends to be maintained by employing voluntary attention to the memory
itself (recall) during the time of retention and results in a better performance in a later
recognition task [2][3]. Although much studies on visual memory have focused on
how a better performance in recognition task is achieved, little is known about thenature of this failure. The failure of retrieval accompanying forgetfulness is possibly
related to memory deletion or false memory.
Visual short-term memory (VSTM) is known to be memory storage of visual
information that has a limited capacity of at least four items, depending on stimulus
complexity and the organization of objects in the memory array [4], and lasts
approximately from one second to several seconds from the onset of the memory
array. Memory before one second from the onset is best understood with iconic orsensory memory which stores a representation with unlimited capacity but decaying
more quickly and is more distracted by eye movements and intervening stimuli
compared with VSTM. It has been shown that several brain regions are associatedwith VSTM. The prefrontal cortex is associated with the maintenance of working
memory and shows increased activity with memory load [6][7][8]. The inferior
intraparietal sulcus (IPS) participates in encoding and maintenance of about four
objects at different spatial locations independent of object complexity whereas
superior IPS and lateral occipital complex participates in that of variable capacity
depending on object complexity [5].
Employing attention to an external visual stimulus [1][9][10] as well as that to aninternal representation of a stimulus that is stored in VSTM [2][3] gives rise to abetter memory performance. Visual attention to an item in the external world
enhances processing of information for that item, whereas suppressing that forunattended items through gating the processing resources on perception [1][3],activating posterior parietal cortex and extrastriate cortex [12]. On the other hand,attention to internal representation is different from the visual attention in respect oftargets because the internal representation in VSTM is encoded in the brain in
advance, giving a better performance in a later recognition task [2][3]. In addition,
attention to internal representation selectively activates several frontal lobe regions,although there is also an overlapping mechanism associated with orienting attention tolocations in external and internal representations in parietal, frontal, and occipitalareas [11]. A question is whether memory of unattended items that were once
encoded in the brain in advance attenuates with or without attention to memory of an
item in VSTM. In other words, memory of unattended items, or not recalled items,
may be deleted or mistaken with other items.
Griffin and Nobre first described attention directed to internal representations ofenhanced object memories in a similar manner to attention directed to external stimuli[3]. In their experiment, an informative cue was presented either before (pre-cue) orafter (retro-cue) an array of four coloured crosses followed by a memory probe. Eachcue had 80 % validity to targets, consisting of two distinct conditions valid andinvalid (in the so-called Posner paradigm [12]) together with a neutral condition thatwas cued with a non-informative signal. The results showed that, as well as theenhancement of accuracy for valid objects, accuracy for invalid objects was
significantly impaired when directing attention to both the external and internal. They
did not, however, address contents of memory degradation, namely memory deletion
and interference in the invalid condition. In case of retro-cues that prompt attention to
the internal representation, differing from pre-cues, the array might be once encodedand stored in VSTM.
Here we investigate these issues by instructing the subjects to direct attention tovisual memory of an object after multiple retention intervals. A delayed matchparadigm was designed to investigate the effect of retention intervals when attention
was oriented to visual memory (internal representation) of an object within VSTM
during its retention (Figure 1). Attention was controlled by a predictive symbolic cue
which indicates a possible target identity, using the Posner paradigm [12]. In 67 % of
total trials, targets appeared with the cued identities. In half of the remaining trials,
targets appeared with an identity other than the cued identity. In the other half, targets
appeared with a novel identity (But see table 1). Nonsensical figures were used tocontrol subjective familiarity with the stimuli and to minimize the effect of verbal
encoding. They were also easily distinguishable and prone to forgetting so as tofacilitate investigation of the nature of mere VSTM.

2Materials and methods

Figure 1. Time course of a trial. A predictive cue
indicating a possible probe identity was presented after
multiple retention intervals (0, 1500, 3000 ms). Subjects
were instructed to answer VALID, INVALID or
NOVEL in a three-alternative forced-choice at the probe
phase. Four nonsensical figures consisting of 5 lines
were randomly generated by the computer.

Table 1. Percentage of numbers oftrials in the designed conditions in
the experiment.

Subjects: 8 subjects (4 females and 4 males, age 25-32, with an average of 28.1)
participated in this experiment. All subjects were right-handed by self-report and hadnormal or corrected-to-normal vision. The subjects gave written informed consent
after being explained about the purpose and nature of the experiments. The stimuli
were presented on a computer screen. The subjects responded by key pressing.
Stimuli and apparatus: Nonsensical figures were generated by the following
algorithm. 20 invisible vertices were randomly chosen by the computer on a 120 x
120 pixel bitmap with black background, of which 2 vertices were randomly chosen
to draw a white line segment between them. The nonsensical figures were composedof 5 line segments each. The 5 segments composition was designed to avoid aresemblance to the letters while minimizing the complexity of the figure. A single
vertex may be shared by more than one line segments.
Procedure: the experiment consisted of 18 blocks with 24 trials each. A trial started
with a 300 ms fixation followed by the study phase. In the study phase, an array of 4
nonsensical figures was presented in the centre of the screen for 800 ms followed byretention intervals of multiple durations (0, 1500, 3000 ms). After the retention
intervals, a cue was presented for 1400 ms. After the presentation of the cue and a 100
ms interval, the probe was displayed in the centre of the screen until the subjects
responded with a key press. The cue was an arrow (i.e. top left, top right, left bottom,
and right bottom) displayed in the centre of screen predicting one of four identities ofa figure that was previously shown in the study phase, whereas the probe was one offour figures displayed at the study phase of the same trial or a novel figure. There
were 67 % “valid-trials” (where the correct answer is “valid”), 16.5 % “invalid-trials”(where the correct answer is “invalid”) and 16.5 % “novel-trials” (where the correct
answer is “novel”). Further details of number of trials are shown in Table 1. One
picture was displayed in only a single trial so that all pictures apparently had
comparable familiarity level. The inter-trial interval was 2000 ms.
Before the experiment, the subjects were informed that a single block consisted of
24 trials and there would be a total of 18 blocks. They sat comfortably in front of thescreen at the distance of 60 cm. They were instructed to look at the crisscrossthroughout its presentation, to remember the object identity in the study phase and toanswer a task question (“valid” “invalid” or “novel” with a key press using rightindex, middle or third finger, respectively) as accurately and quickly as possible.
After the instruction, the subjects practiced the tasks.

3

Results

Hit rate of attended objects (VALID response in the valid trial) decreased
significantly when retention intervals were increased (one way ANOVA, p<0.001;
between 0 and 1500 ms retention interval, Tukey post hoc test, p<0.05; between 0 and3000 ms retention interval, Tukey post hoc test, p<0.001) while the hit rate of invalid
objects (INVALID response in the invalid trial) and of novel objects (NOVEL
response in the novel trial) remained unchanged (p>0.1) (Figure 2 (a)). Comparison of
trial types on 0 ms retention interval by Tukey post hoc test revealed significant
differences between the valid and novel trials (p<0.001) and between the invalid andnovel trials (p<0.05) (Figure 2 (a)). d’ in each trial was calculated in a case of threealternative forced choice with chance level 33% (Figure 2 (c)). The statisticalsignificances were the same as the hit rate above.
With an increase in failure in recognition for valid objects after longer retention
intervals, wrong answers of invalid and novel objects were increased (one way
ANOVA for invalid objects, p<0.001; between 0 and 1500 ms retention interval,
Tukey post hoc test, p<0.05; between 0 and 3000 ms retention interval, Tukey post
hoc test, p<0.05 and one way ANOVA for novel objects, p<0.05; but none of
combinations among 0, 1500 and 3000 ms retention interval by Tukey post hoc test
revealed significant differences) (Figure 2 (b)). Paired t-tests were performed on two
types of the wrong answers for each retention interval. Only with the interval of 1500
ms, the rate of INVALID response was significantly higher than that of NOVEL
response (p<0.05), indicating that the failure in recognition of a valid object may
result in having false memory (i.e. of an unattended object) rather than memory
deletion at that retention interval. As hit rate of the invalid trial (INVALID responsein the invalid trial) and that of the novel trial (NOVEL response in the novel trial)
were not changed with increasing retention interval, neither wrong answers in theinvalid trial (i.e. VALID and NOVEL response) or that in the novel trial (i.e. VALID
and INVALID response) were changed with increasing retention interval.

(f)Figure 2. (a) Hit rate in each trial type (the rate of VALID response in valid trial, INVALID
response in invalid trial or NOVEL response in novel trial). (c) Values of d’ in each trial type.
(b),(d),(e) Rate of response types in the valid, invalid, and the novel trials, respectively (seeTable 1). (f) Line type and colour. Colours of the line indicate the trial types: black for the valid
trials, red for the invalid trials and green for the novel trials. Types of the line indicate theresponse type: solid lines for valid, long dashed lines for invalid and short dashed lines fornovel. The horizontal axis is the retention intervals (ms). The bars give the standard errors.

Figure 3. Rate of responses irrespective to the trial types. Colours of the line indicate theresponse types: black for the VALID response, red for INVALID response and green for
NOVEL response. Note that numbers of trials conducted in the experiment are shown in Table
1.

There is also the possibility that the set of data in the valid trial includes false alarms
from other trial types. To exclude the mutual effects of trial types, the mere rates of
response types (= number of each response / total number (= 432) of trials) wereanalyzed (Figure 3). Although the numbers of trials were designed asymmetrically tocontrol for the validity of cues (Table 1), the result shows significantly different
tendencies (Figure 3). The rate of VALID response decreased (one way ANOVA,
p<0.001; between 0 and 1500 ms retention interval and 0 and 3000 ms retentioninterval, Both Tukey post hoc test, p<0.001) whereas the rate of both INVALID andNOVEL response increased (both one way ANOVA, p<0.001; between 0 and 1500
ms retention interval with NOVEL response. Tukey post hoc test, p<0.0001; between
0 and 3000 ms retention interval with NOVEL response. Tukey post hoc test, p<0.01;
between 0 and 3000 ms retention interval with NOVEL response. Tukey post hoc test,
p<0.05). When the retention intervals were 1500 ms, the rate of INVALID response
was significantly higher than that of both VALID and NOVEL responses (Tukey post
hoc test, both p<0.05). In post-experimental interview, all subjects reported that theyrealized that the valid trials were more frequent but did not realize the difference
among retention intervals within the valid trial (see Table 1).

4Discussion

The nature of memory degradation when orienting attention to the internal
representation (one of remembered objects) in VSTM was investigated. Unlike
previous studies [2][3], our experiments provide the time course effect on retrieval
VSTM. As the retention interval between the offset of stimuli and orienting attention
increased, only memory of attended objects degenerated while hit rate of unattended
objects (i.e. INVALID response in the invalid trial and NOVEL response in the novel
trial) did not (Figure 2 (a)). Thus, attention did not simply bring about a better
memory performance but rather it seemed to make a gradient on VSTM. Orienting
attention to more abstract VSTM with a long retention may lead to a quick decay of
VSTM. Further study is necessary to investigate whether orienting attention tointernal representation actually causes a quick decay after a long retention interval.
To investigate the nature of forgetfulness in VSTM, we focused on wrong answers
in the valid trial. Both INVALID response and NOVEL response increased withincreasing retention interval, while the rate of INVALID response and NOVEL
response did not make a difference at both 0 and 3000 ms retention interval (Figure 2(b)). The responses of both INVALID and NOVEL have a similar tendency and these
responses in daily life may often appear coincidently. Here we show a separation of
INVALID and NOVEL response in wrong answers at the time of 1500 ms retention
interval from the offset of memory array. In the valid trial, response of INVALID isinterpreted as interference within VSTM. Having a false memory and response of
NOVEL is considered to be the deletion of memory. Thus, as forgetfulness increases
with a longer retention, there is a tendency to have a false memory more than memory
deletion at an early time of retention. And later, the difference in the level of having
false memory and memory deletion becomes insignificant. Once the VSTM isestablished, it would be maintained by recurrent neural activation during short-term
period [13] rather than by physical changes on the structure of connectivity such as
the long-term potentiation. Such recurrent activation would maintain the VSTM,
degrading over time. When the VSTM is retained at a certain level at 1500 ms
retention interval, attention that activates neurons in several brain regions may distract
the recurrent activation of VSTM, as much as bringing out a mistake for the otherremembered objects, but not as much as causing a deletion of VSTM. This may
explain why VSTM may be distorted by orienting attention to memory of an object

that leads to a new wave of activations on a circuit. Consequently, it is possible thatforgetfulness in VSTM is primarily due to interference rather than memory deletion.
The effect of Posner paradigm was analyzed in the mere rates of response types inall of trial types (Figure 3). The experiment was designed in an asymmetrical manner
regarding the number of trials (Table 1). Only the number of the valid trial at 0 ms
retention interval was large and the subjects actually chose VALID the most often at 0
ms retention interval. If subjects had been successfully biased with this asymmetricnumber of trials (Posner paradigm), they would have tended to choose VALID themost frequently. However, at 1500 ms and later the rates of VALID response were
low and stayed around the chance level of 33 %. Interestingly, the rate of INVALID
response at 1500 ms retention interval was significantly higher than that of VALID
and NOVEL. There are two possibilities: The subjects were not biased. Or, they werebiased but tended to choose INVALID response more. According to the subject’sreports, they did not strategically respond to retention intervals, such that they
responded equally to three alternatives for the longer intervals. Suppose they did so
without being aware, the rate of response cannot be reversed at 1500 ms retention
interval. Thus the latter hypothesis may be more appropriate. As discussed above, if
there is distractive effect with attention to internal representation at the intermediate
retention period, interference of memory may be more often the case than deletion ofmemory.

5Conclusion

When the retention intervals increased, success in the recognition of attended (valid)
objects decreased while that of unattended or novel objects did not. This failure inrecognition of attended objects as early as 1500 ms retention interval from offset of amemory array is possibly due to having a false memory. Further study is necessary toinvestigate the effect of attention to internal representation stored in VSTM.

Acknowledgments. We thank all laboratory members for assistance and effective
discussion.

References
1. Mack A, Rock I.: Inattentional Blindness. Cambridge, MIT Press. (1998)
2. Makovski T, Sussman R, Jiang YV.: Orienting Attention in Visual Working Memory
Reduces Interference From Memory Probes. Journal of Experimental Psychology: Learning,
Memory, and Cognition. Mar 34(2), 369-380. (2008)
3. Griffin IC, Nobre AC.: Orienting Attention to Locations in Internal Representations. J Cogn
Neurosci. Nov 15(8), 1176-94. (2003)
4. Sligate IG, Scholte HS, Lamme VA.: Are there multiple visual short-term memory stores?:
PLOS one. Feb 3(2), e1699. (2008)
5. Xu Y, Chun MM.: Dissociable neural mechanisms supporting visual short-term memory forobjects. Nature. Mar 2, 440(7080) 91-5 (2006)

6. Pessoa L, Gutierrez E, Bandettini P, Ungerleider L.: Neural correlates of visual working
memory: fMRI amplitude predicts task performance. Neuron. 29.Aug 35(5), 975-87 (2002)
7. Cohen JD, Perlstein WM, Braver TS, Nystrom LE, Noll DC, Jonides J, Smith EE.: Temporal
dynamics of brain activation during a working memory task. Nature.10.Apr 386(6625) 6048 (1997)
8. Funahashi S.: Prefrontal cortex and working memory processes. Neuroscience. 28.Apr
139(1),
251-61 (2006)
9. Vuilleumier P, Schwartz S, Duhoux S, Dolan RJ, Driver J.: Selective attention modulates
neural substrates of repetition priming and "implicit" visual memory: suppressions andenhancements revealed by FMRI. J Cogn Neurosci Aug 17(8) 1245-60 (2005)
10. Prinzmetal W, McCool C, Park S.: Attention Reaction Time and Accuracy Reveal Different
Mechanisms. J Exp Psychol Gen. Feb, 134(1), 73-92 (2005)
11. Nobre AC, Coull JT, Maquet P, Frith CD, Vandenberghe R, Mesulam MM.: Orienting
Attention to Locations in Perceptual Versus Mental Representations. J Cogn Neurosc, Apr
16(3), 363-73 (2004)
12. Posner MI.: Orienting of attention. Q J Exp Psychol. Feb, 32(1), 3-25 (1980)
13. Constantinidis C, Wang XJ.: A neural circuit basis for spatial working memory.
Neuroscientist. Dec, 10(6), 553-65 (2004)
14. Hoshino E, Mogi, K.: Trade-off in the Effect of Attention for Visual Short-term Memory.
Association for the Scientific Study of Consciousness 14 (2010)