# An ecologically valid examination of event-based and time-based prospective memory using immersive virtual reality: The influence of attention, memory, and executive function processes on real-world prospective memory

Panagiotis Kourtesis, Sarah E Macpherson

► To cite this version:

Panagiotis Kourtesis, Sarah E Macpherson. An ecologically valid examination of event-based and time-based prospective memory using immersive virtual reality: The influence of attention, memory, and executive function processes on real-world prospective memory. Neuropsychological Rehabilitation, 2023, 33 (2), pp.255-280. 10.1080/09602011.2021.2008983. hal-03472981

HAL Id: hal-03472981

https://inria.hal.science/hal-03472981

Submitted on 9 Dec 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
An ecologically valid examination of event-based and time-
based prospective memory using immersive virtual reality: the
influence of attention, memory, and executive function
processes on real-world prospective memory.

Panagiotis Kourtesis<sup>a,b,c,d,e,f,g,h*</sup> and Sarah E. MacPherson<sup>a,b</sup>

<sup>a</sup>Human Cognitive Neuroscience, Department of Psychology, University of Edinburgh,
Edinburgh, UK;

<sup>b</sup>Department of Psychology, University of Edinburgh, Edinburgh, UK;

<sup>c</sup>Lab of Experimental Psychology, Suor Orsola Benincasa University of Naples, Naples, Italy;

<sup>d</sup>Interdepartmental Centre for Planning and Research "Scienza Nuova", Suor Orsola
Benincasa University of Naples, Naples, Italy;

<sup>e</sup>National Research Institute of Computer Science and Automation, INRIA, Rennes, France;

<sup>f</sup>Univ Rennes, Rennes, France;

<sup>g</sup>Research Institute of Computer Science and Random Systems, IRISA, Rennes, France;

<sup>h</sup>French National Centre for Scientific Research, CNRS, Rennes, France.

<footnote>* Panagiotis Kourtesis, Department of Psychology, University of Edinburgh, 7 George Square, Edinburgh, EH8 9JZ, United Kingdom. Email: pkourtes@exseed.ed.ac.uk</footnote>
Abstract

Studies on prospective memory (PM) predominantly assess either event- or time-based PM by implementing non-ecological laboratory-based tasks. The results deriving from these paradigms have provided findings that are discrepant with ecologically valid research paradigms that converge on the complexity and cognitive demands of everyday tasks. The Virtual Reality Everyday Assessment Lab (VR-EAL), an immersive virtual reality (VR) neuropsychological battery with enhanced ecological validity, was implemented to assess everyday event- and time-based PM, as well as the influence of other cognitive functions on everyday PM functioning. The results demonstrated the importance of delayed recognition, planning, and visuospatial attention on everyday PM. Delayed recognition and planning ability were found to be central in event- and time-based PM respectively. In order of importance, delayed recognition, visuospatial attention speed, and planning ability were found to be involved in event-based PM functioning. Comparably, planning, visuospatial attention accuracy, delayed recognition, and multitasking/task-shifting ability were found to be involved in time-based PM functioning. These findings further suggest the importance of ecological validity in the study of PM, which may be achieved using immersive VR paradigms.

Keywords: prospective memory; episodic memory; attention; executive functions; ecological validity
**Introduction**

Prospective memory (PM) involves the ability to remember to initiate an action in the future. The PM action may be associated with either a certain event (i.e., event-based PM; e.g., when you pass the supermarket, buy a pint of milk) or a specific time (time-based PM; e.g., at 3 pm watch the weather forecast). PM functioning is crucial for many aspects of everyday life, varying from minor (e.g., brushing your teeth after having breakfast) to vital importance (e.g., having an insulin injection after dinner). PM impairment is common in healthy older individuals (Huppert, Johnson, & Nickson, 2000), as well as in clinical populations such as Parkinson's disease (Kliegel, Altgassen, Hering, & Rose, 2011), individuals with mild cognitive impairment (Costa, Caltagirone, & Carlesimo, 2011), Alzheimer's disease (Jones, Livner, & Bäckman, 2006), and autism spectrum disorder (Sheppard, Bruineberg, Kretschmer-Trendowicz, & Altgassen, 2018). The study of everyday PM may assist in our understanding of real-life PM functioning, and consequently provide ways to improve functioning in individuals with PM impairments.

*Cognitive functions associated with prospective memory*

The two main theoretical models of PM are the preparatory attentional and memory (PAM) theory and the multiprocess theory (Anderson, McDaniel, & Einstein, 2017). These two theoretical models have attempted to explain the implications of attentional and memory processes in everyday PM functioning. Based on the PAM theory, PM functioning relies on attentional processes for monitoring both environmental and internal cues, which may assist an individual in recalling their intended PM action (Smith, 2003; Smith, Hunt, McVay, & McConnell, 2007). The multiprocess framework suggests that passive monitoring may also assist in detecting target cues, resulting in reflexive associative retrieval of the intended PM action (McDaniel & Einstein, 2000; McDaniel & Einstein, 2007). More recently, the dynamic multiprocess framework has been proposed, which suggests a dynamic interplay between top-down and bottom-up monitoring and retrieval processes for PM functioning (Scullin, McDaniel, & Shelton, 2013; Shelton & Scullin, 2017). The dynamic multiprocess framework emphasises the importance of contextual information and/or formulating an effective plan of action in PM functioning (Scullin et al., 2013; Shelton & Scullin, 2017). Contextual information may indicate the probability of detecting a target cue related to the intended PM action in the current environment (Marsh, Hicks, & Cook, 2008; Scullin et al., 2013; Shelton & Scullin, 2017). In addition, an effective plan of action is facilitated by associating potential environmental cues with the intended PM action (Scullin et al., 2013; Shelton & Scullin, 2017).

The formulation of an effective plan of action is based on Gollwitzer's *implementation intention*, which is an encoding technique for future actions by associating a cue with the intended action (i.e., "if I encounter X then I will do Y"; Gollwitzer, 1999). Planning ability, an executive function, involves thinking about the future, organizing and prioritizing future actions (Miyake *et al.*, 2000; Morris & Ward, 2004). In everyday life, planning defines when and where an action will take place, and updates/prioritises the plan of action based on the received information (e.g., I received a letter notifying me of my overdue electricity bill, so, tomorrow I will need pay it after work; Morris & Ward, 2004). Indeed, effective planning has
been found to improve PM functioning (Azzopardi, Auffray, & Kermarrec, 2017; Gonneaud et al., 2011; Liu & Park, 2004; McDaniel & Scullin, 2010; McFarland & Glisky, 2009; Milne, Orbell, & Sheeran, 2002; Mioni & Stablum, 2014; Vanneste, Baudouin, Bouazzaoui, & Taconnat, 2016; Zuber, Mahy, & Kliegel, 2019). Another executive function which is often found to contribute to PM functioning is task-shifting (Azzopardi et al., 2017; Gonneaud et al., 2011; McFarland & Glisky, 2009; Mioni & Stablum, 2014; Vanneste et al., 2016; Zuber et al., 2019). Task-shifting pertains to the ability to inhibit one's attention on the current task, and shift it to another task (Miyake et al., 2000).

However, as mentioned above, everyday PM consists of both event- and time-based PM (Einstein & McDaniel, 1996). Hence, including only event-based tasks in the assessment of PM does not allow for generalisation of the results to PM functioning in the daily life (e.g., Einstein, Smith, McDaniel, & Shaw, 1997; Einstein, McDaniel, Manzi, Cochran, & Baker, 2000; Mullet, Scullin, Hess, Scullin, Arnold, & Einstein, 2013; Scullin, McDaniel, Shelton, & Lee, 2010; Smith, 2003; Smith et al., 2007). Time-based tasks may have similar aspects to event-based tasks (e.g., a cue which reminds an individual of the intended PM action), but they significantly differ in other aspects (Glickson, & Myslobodsky, 2006; Zuber et al., 2019). A key difference is the requirement to monitor the time in time-based tasks (Glickson, & Myslobodsky, 2006; Zuber et al., 2019). However, individuals do not appear to constantly monitor time. Instead, the most common behaviour is to monitor time strategically (Cona, Arcara, Tarantino, & Bisiacchi, 2012; Kliegel, Martin, McDaniel, & Einstein, 2001). By implementing strategic time monitoring, individuals check the time and then estimate when they should check it again, and adjust correspondingly the frequency of their monitoring (Cona et al., 2012; Kliegel et al., 2001).

## The importance of ecological validity

Evidence for the role of attentional, memory, and executive functioning processes in PM functioning derives predominantly from experiments which have implemented laboratory tasks. These laboratory tasks incorporate simple, static stimuli within a highly controlled environment, which do not resemble the complexity of real-life situations (Parsons, 2015). For example, a PM laboratory experiment may incorporate a single on-going task such as a computerised lexical decision task where the participant should indicate whether the presented word is a real or fictional word, together with a comparable computerised PM task, where the participant should indicate when a specific word appears. Hence, the laboratory tasks incorporate a simple testing environment (e.g., two-dimensional) and stimuli (e.g., static), with almost automatic responses (i.e., in milliseconds) provided using a keyboard, a relatively short testing duration (e.g., 15 minutes) and delay from encoding the PM intention (e.g., up to 10 minutes), and a single on-going task and a PM task with low demands.

Importantly, the implementation of non-ecologically valid PM tasks may provide discrepant results compared to ecologically valid research paradigms (Marsh, Hicks, & Landau, 1998). Ecologically valid tasks substantially converge with the complexity and cognitive demands of corresponding everyday tasks (Chaytor & Schmitter-Edgecombe, 2003; Franzen & Wilhelm, 1996; Spooner & Pachana, 2006), which allows the findings to be generalised to everyday functioning (Chaytor & Schmitter-Edgecombe, 2003; Haines et al., 2007).
2019; Higginson, Arnett, & Voss, 2000; Mlinac & Feng, 2016; Parsons, 2015; Phillips, Henry, & Martin, 2008). The implementation of ecologically valid tasks is therefore required to examine the role of attentional, memory, and executive functioning processes in everyday PM functioning.

## *Naturalistic Experiments*

Ecologically valid research paradigms have predominantly involved assessments in real-life settings such as performing errands in a shopping centre or a pedestrianized street (e.g., Garden, Phillips, & MacPherson, 2001; Marsh et al., 1998; Shallice & Burgess, 1991). In PM research, these naturalistic experiments do not tend to consider diverse types of PM tasks within the same paradigm. The most replicated discrepancy between the findings from laboratory and naturalistic tasks is the so-called age-prospective-memory-paradox (Kvavilashvili, Cockburn, & Kornbrot, 2013; Niedźwieńska & Barzykowski, 2012; Phillips et al., 2008; Uttl, 2008). In laboratory tasks, there are significant age-related differences on both event-based and time-based tasks between younger and older adults, while in naturalistic tasks, older adults perform at least comparably to younger adults (Kvavilashvili et al., 2013; Niedźwieńska & Barzykowski, 2012; Phillips et al., 2008; Uttl, 2008). This paradox may be attributed to factors such as individual differences (Phillips et al., 2008; Shelton & Scullin, 2017; Uttl, 2008) in the use of compensatory strategies and external memory aids, which are common strategies in daily life for performing PM tasks (Phillips et al., 2008; Uttl, 2008).

However, real-world tasks suffer from several limitations such as an inability to be standardized for use in other clinics or laboratories, reduced feasibility with certain neurological populations (e.g., schizophrenia patients), augmented time, fiscal, procedural and bureaucratic demands, as well as diminished experimental control over external factors (Parsons, 2015). An example of diminished control can be seen in the study of Niedźwieńska and Barzykowski (2012) where the scoring was dichotomous in terms of 0 or 1, without considering other factors (e.g., using external aids) that assist in performing PM tasks, and this may have caused ceiling effects.

## *Virtual Reality*

An alternative approach to achieve ecological validity is the implementation of non-immersive virtual environments, which are able to simulate real-life tasks, are cost-effective, require less administration time, enable increased experimental control, and can be implemented in other clinical or research settings (Parsons, McMahan, & Kane, 2018; Werner & Korczyn, 2012; Zygouris & Tsolaki, 2015). However, non-immersive virtual reality (VR) tasks can be difficult for individuals without gaming backgrounds (e.g., competency in using a keyboard and mouse to interact with the virtual environment; Zaidi, Duthie, Carr, & Maksoud, 2018) and they are substantially less ecologically valid than immersive VR tests (Parsons et al., 2018; Rizzo, Schultheis, Kerns, & Mateer, 2004). On the other hand, immersive VR tests benefit from the same advantages as non-immersive ones, but they also benefit from enhanced ecologically validity (Bohil, Alicea, & Biocca, 2011; Parsons, 2015; Rizzo et al., 2004; Teo et al., 2016). Notably, the first-person view and ergonomic/naturalistic interactions within immersive VR environments are proximal to real-life actions, as a result they significantly alleviate the differences in performance between gamers and non-gamers (Zaidi et al., 2018). Therefore, an immersive VR research paradigm appears to be the most efficient approach to study everyday PM functioning.
**_The Virtual Reality Everyday Assessment Lab_**

The Virtual Reality Everyday Assessment Lab (VR-EAL) was recently developed as an immersive VR neuropsychological battery assessing everyday cognitive functions such as PM (i.e., event-based and time-based), episodic memory (i.e., immediate and delayed recognition), attentional processes (i.e., visual, visuospatial, and auditory), and executive functioning (i.e., planning and task-shifting abilities; Kourtesis, Korre, Collina, Doumas, & MacPherson, 2020b). The convergent, construct, and ecological validity of the VR-EAL has been examined against established ecologically valid paper-and-pencil tests including the Cambridge Prospective Memory Test (Wilson *et al.*, 2005), the Rivermead Behavioral Memory Test-III (Wilson, Cockburn, & Baddeley, 2008), the Test of Everyday Attention (Robertson, Ward, Ridgeway, & Nimmo-Smith, 1996), and the Behavioral Assessment of Dysexecutive Syndrome (Wilson, Alderman, Burgess, Emslie, & Evans, 1997; Kourtesis, Collina, Doumas, & MacPherson, 2020a). Importantly, there is no performance difference in the VR-EAL between gamers and non-gamers. Also, participants rated the VR-EAL scenarios and tasks as significantly more similar to everyday tasks than the equivalent ecologically valid paper-and-pencil tasks (Kourtesis *et al.*, 2020a).

Hence, the VR-EAL appears to better resemble the complexity and cognitive demands of real-life tasks, as well as assessing both event- and time-based PM, and cognitive processes (i.e., attentional, memory, and executive functioning) which have been found to be associated with PM functioning. The VR-EAL is significantly closer to real life situations due to attributes such as the complexity and synthesis of the environment (i.e., a realistic and complex 360° environment), the quality of the stimuli (i.e., dynamic and realistic), the duration of the experiment (i.e., approximately 70 minutes), the naturalistic and ergonomic interactions within the testing environment, the response type (i.e., naturalistic) and time (i.e., in seconds), the length of the delay between encoding the PM intention and performing the PM action (i.e., 15 – 60 minutes), as well as including several cognitively demanding ongoing and PM tasks.

**_The current study_**

To our knowledge, the present study is the first to explore the cognitive functions that are involved in everyday event- and time-based PM functioning using an ecologically valid immersive VR paradigm in a single sample. As discussed above, the relevant literature postulates that attentional, memory, and executive processes are implicated in PM functioning. Thus, we hypothesize that attention, memory, and/or executive functioning will be involved in everyday PM functioning. However, since ecologically valid and laboratory paradigms does not provide convergent findings, we expect that the implementation of an ecologically valid assessment may indicate the increased importance of certain cognitive functions (e.g., delayed recognition, visuospatial attention, and planning), while it may decrease the importance of other cognitive functions (e.g., immediate recognition, visual attention, and multitasking). In terms of delayed recognition, the VR-EAL scenario lasts approximately 70 minutes which allows one to investigate the importance of recognising a cue after a long delay. Also, the inclusion of several diverse everyday tasks (PM and non-PM) within a realistic scenario may indicate the importance of prioritising activities (i.e., planning ability) in real life PM. Furthermore, the VR-EAL activities are performed in a naturalistic way, which substantially differ from the almost immediate responses in laboratory paradigms
(i.e., reaction times in milliseconds). Hence, allowing the examinee to perform the tasks in a
naturalistic way (e.g., in seconds) may reveal the decreased importance of multitasking/task-
shifting ability in everyday PM functioning. Finally, the VR-EAL scenario takes place within
a complex, dynamic, and immersive 360° environment, hence, the importance of visuospatial
attention processes (i.e., directing attention to a specific area of the environment and
identifying items of interest within this specific area) may be seen as more important than
visual attention processes (i.e., identifying items of interest within the visual field).

Methods

Participants and Procedure

Participants were recruited via social media and the internal mailing list of the University of Edinburgh and were the same cohort as in Kourtesis et al. (2020a). Forty-one participants (21 females) with a mean age of 29.15 years (SD = 5.80, range = 18-45) and mean education of 13.80 years (SD = 2.36, range = 10-16) were recruited. The study was approved by the Philosophy, Psychology and Language Sciences Research Ethics Committee of the University of Edinburgh. Written informed consent was obtained from each participant. All participants received verbal and written instructions regarding the procedures, possible adverse effects of immersive VR (e.g., cybersickness), utilization of the data, and general aims of the study. The participants attended the session individually. Upon their arrival, they were instructed by the examiner how to use the VR equipment. Then the participants were immersed into the VR-EAL, where they had a practice period without time restrictions to get familiarised with the basic controls and configuration. When the participants felt comfortable interacting within the virtual environment of VR-EAL, the VR-EAL's scenario commenced. All participants completed all tasks (i.e., non-PM and PM) and the full VR-EAL scenario. After exposure to VR, the Virtual Reality Neuroscience Questionnaire (VRNQ; Kourtesis, Collina, Doumas, & MacPherson, 2019b) was administered to assess cybersickness symptomatology.

Materials

Hardware

An HTC Vive HMD with two lighthouse stations for motion tracking and two HTC Vive
wands with six degrees of freedom (6DoF) for navigation and interactions within the virtual
environment were implemented in accordance with our previously published technological
recommendations for immersive VR research (Kourtesis, Collina, Doumas, & MacPherson,
2019a). The spatialized (bi-aural) audio was facilitated by a pair of Senhai Kotion Each
G9000 headphones. The size of the VR area was 5m², which provides an adequate space for
immersion and naturalistic interaction within virtual environments (Borrego, Latorre, Alcañiz,
& Llorens, 2018). The HMD was connected to a laptop with an Intel Core i7 7700HQ
2.80GHz processor, 16 GB RAM, a 4095MB NVIDIA GeForce GTX 1070 graphics card, a
931 GB TOSHIBA MQ01ABD100 (SATA) hard disk, and Realtek High Definition Audio.
## VR-EAL

VR-EAL assesses everyday cognitive functions such as PM, episodic memory (i.e., immediate and delayed recognition), executive functioning (i.e., planning, multitasking) and selective visual, visuospatial and auditory (bi-aural) attention within a realistic immersive VR scenario lasting around 70 minutes (Kourtesis et al., 2020a). See Table 1 and Figures 1 and 2 for a summary of the VR-EAL tasks assessing each cognitive ability. Also, a brief video recording of the VR-EAL may be accessed at this hyperlink:

https://www.youtube.com/watch?v=IHEIvS37Xy8&t.

<table><caption>Table 1. VR-EAL tasks and score ranges</caption><thead><tr><td>Scene</td><td>Cognitive Function</td><td>Task</td><td>Score</td></tr></thead><tbody><tr><td>3</td><td>Prospective memory</td><td>Write down the notes for the errands.</td><td>0 – 6</td></tr><tr><td>3</td><td>Immediate recognition</td><td>Recognising items on the shopping list.</td><td>0 – 20</td></tr><tr><td>3</td><td>Planning</td><td>Drawing the route to be taken.</td><td>0 – 19</td></tr><tr><td>6</td><td>Multitasking</td><td>Cooking task (preparing breakfast).</td><td>0 – 16</td></tr><tr><td>6</td><td>Prospective memory – event based</td><td>Take medication after breakfast.</td><td>0 – 6</td></tr><tr><td>8</td><td>Selective visuospatial attention</td><td>Collect items from the living room.</td><td>0 – 20</td></tr><tr><td>8</td><td>Prospective memory – event based</td><td>Take the chocolate pie out of the oven.</td><td>0 – 6</td></tr><tr><td>10</td><td>Prospective memory – time based</td><td>Call Rose at 10 am.</td><td>0 – 6</td></tr><tr><td>12</td><td>Selective visual attention</td><td>Find posters on both sides of the road.</td><td>0 – 16</td></tr><tr><td>14</td><td>Delayed recognition</td><td>Recognising items from the shopping list.</td><td>0 – 20</td></tr><tr><td>15</td><td>Prospective memory – time based</td><td>Collect the carrot cake from the bakery at 12 pm.</td><td>0 – 6</td></tr><tr><td>16</td><td>Prospective memory – event based</td><td>False prompt before going to the library.</td><td>-6 – 0</td></tr><tr><td>17</td><td>Prospective memory – event based</td><td>Return the red book to the library.</td><td>0 – 6</td></tr><tr><td>19</td><td>Selective auditory attention</td><td>Detect sounds from both sides of the road.</td><td>0 – 32</td></tr><tr><td>20</td><td>Prospective memory – time based</td><td>False prompt before going back home.</td><td>-6 – 0</td></tr><tr><td>21</td><td>Prospective memory – event based</td><td>Back home, give the extra pair of keys to Alex.</td><td>0 – 6</td></tr><tr><td>22</td><td>Prospective memory – time based</td><td>Take the medication at 1pm.</td><td>0 – 6</td></tr></tbody></table>

<footnote>*The tasks are presented in the same order as they are performed within the scenario. The table derives from Kourtesis et al. (2020a)</footnote>
<figure><img src="image_2.png" /><figcaption>Figure 1. VR-EAL Storyline: Scenes 3 - 12.</figcaption></figure>

Derived from Kourtesis et al. (2020b).
<figure><img src="image_3.png" /><figcaption>Figure 2. VR-EAL Storyline: Scenes 14 – 22</figcaption></figure>

Derived from Kourtesis et al. (2020b).
*Prospective Memory*. VR-EAL considers both event-based and time-based PM tasks. The PM tasks are divided into five event-based tasks and four time-based tasks. In the event-based tasks, the participant should remember to perform a PM action when a specific event occurs (e.g., take medicines after breakfast). In the time-based tasks, the examinee should remember to perform a planned action at a specific time (e.g., call Rose at 12 pm). The following is a scoring example. At the end of a scene, the examinees should press a button to confirm that all the tasks in the scene have been completed. If the examinees have already taken their medication (i.e., PM task) before pressing the final button, then the scene ends, and the examinees receive 6 points. Otherwise, the first prompt appears (i.e., “You Have to Do Something Else”). If the examinees then follow the prompt and take their medication, they receive 4 points. If the examinees press the final button again, then the second prompt appears (i.e., “You Have to Do Something After Having your Breakfast”). If the examinees follow this prompt and take their medication, they receive 2 points. If the examinees press the final button again, then the third prompt appears (i.e., “You Have to Take Your Meds”). If the examinees follow this prompt and take their medication, they then receive 1 point. If the examinees repress the final button without ever taking their medication, they get zero points, and the scene ends. Scores range from 0 to 6.

There are also two misleading tasks. One is related to an event (i.e., event-based) and the other to a specific time (i.e., time-based). However, in the misleading tasks, the examinee should respond negatively to the virtual human prompts to perform a task because there is not a task to perform. The misleading tasks serve two purposes. The first is deception to examine whether the examinee is simply responding affirmatively to all PM task prompts. If the examinee responds affirmatively, then s/he loses points (see Table 1). The second is that these false prompts are common in everyday life (e.g., a friend invites you to watch an episode of a TV series together, but you decline because you remember that the episode will be aired tomorrow and not today). Hence, their inclusion increases the ecological validity of VR-EAL.

*Episodic Memory*. Both immediate and delayed episodic memory are assessed. Firstly, the participant needs to memorize a shopping list which is presented audio-visually. Immediately after the presentation of the list, the participant is presented with 30 grocery items (i.e., immediate recognition). The examinee should choose the ten target items (i.e., create the shopping list) from an extensive array of items, which also contains five qualitative distractors (e.g., semi-skimmed milk versus skimmed milk), five quantitative distractors (e.g., 1 kg potatoes versus 2 kg potatoes), and ten false items (e.g., bread, bananas etc.). Similarly, the participant should choose the same 10 items when they arrive at the supermarket approximately 20 minutes later (delayed recognition). The examinee gains 2 points for each correctly chosen item, 1 point for selecting a qualitative or quantitative distractor, and 0 points for the false items. Scores range from 0 to 20.

*Multitasking/Task-Shifting*. The ability to multitask is examined using a cooking task, where the participant should prepare and serve his or her breakfast (e.g., sausages, omelette, and a cup of tea/coffee) and place a chocolate pie in the oven. Scoring relies on the animations from each game object (i.e., the omelette and the sausages). At the beginning of the animation, both items have a reddish (raw) colour which gradually turns to either a
yellowish (omelette) or brownish (sausages) colour, and finally both turn to black (burnt). The score for each pan depends on the time that the examinee removes the pan from the stove and places the pan on the kitchen worktop. Equally, the score for boiling the kettle is measured in relation to the stage of the audio playback (e.g., the kettle whistles when the water is ready) when the kettle is placed on the kitchen worktop. Scores range from 0 to 16.

***Planning.*** Planning ability is assessed by asking participants to draw their route around the city (e.g., visiting the bakery, supermarket, library, and returning home) on a 3D interactive board. The road system comprises 23 street units. When the examinee selects a unit, 1 point is awarded. The ideal route to visit all three destinations is 15 units; hence, any extra or missing units are subtracted from the total possible score of 15. Up to 4 more points are awarded for the time taken to complete the task. Scores range from 0 to 19.

***Visuospatial attention.*** Visuospatial attention is assessed by asking the participant to find and collect 6 specific items (i.e., a mobile phone, a £50 note, a library card, the flat keys, a red book, and car keys) from the living-room. A reminder of these items remains on the wall (i.e., the items are displayed as 3D objects with labels). However, there are also distractors (i.e., magazines, books, a remote control, a notebook, a pencil, a chessboard, and a bottle of wine) in the room. The examinee receives 2 point for each target item collected (6 target items). Also, up to 4 points are awarded for the speed of detecting the items. If the examinee attempts to collect one of the distractors, it counts as an error. Up to 4 points are awarded for the accuracy of detecting items. Scores range from 0 to 20.

***Visual attention.*** Visual attention is measured while the participant is seated as a passenger in a car next to a driver. The participant should identify all the targets (i.e., 16 posters of a radio station) on both sides of the road, while s/he needs to avoid any distractors (i.e., 8 posters that are a different shape and 8 posters with a different background colour). The examinee is awarded 1 point when a target poster is “spotted” and subtracted 1 point when a distractor poster is “spotted”. Scores range from 0 to 16.

***Auditory Attention.*** Auditory attention is also examined while the participant is seated as a passenger next to a driver. The participant should detect all the target sounds (i.e., 16 bell sounds) presented on both sides of the road, while avoiding the distractor sounds (i.e., 8 high pitch bells, and 8 dongs). If the examinee presses the trigger on the right controller to detect a target sound originating on the right side (i.e., controller and sound on the same side), then s/he gets 2 points. If the examinee presses the trigger on the right controller to detect a target sound originating on the left side (i.e., controller on the opposite side), s/he gains only 1 point. If the examinee responds to a distractor sound, irrelevant of its origin or the controller used to respond, 1 point is deducted. Scores range from 0 to 32.

***Statistical Analyses***

All the statistical analyses were conducted using R language (R Core Team, 2020) in RStudio (RStudio Team, 2020). The stats (R Core Team, 2020) and the car (Fox & Weisberg,
2019) packages were used for conducting the linear regression analyses and the diagnostics.
The pwr package (Champely, 2020) was used to perform the statistical power analysis for
each model. The ggstatsplot package (Patil, 2018) was used for visualising the results.

Linear regression analyses

The normality assumption for the event- and time-based scores was examined using the Shapiro-Wilk Normality test. The Non-Constant Error Variance test was used to check the homoscedasticity assumption for the models. The variance inflation factor for each model's predictors was calculated and checked to detect multicollinearity. A statistical power analysis was conducted for each model. Linear regression analyses were performed to investigate which everyday cognitive functions predict everyday PM functioning. The models were compared by conducting analyses of variance. The criteria for the comparisons between the models were the Akaike's information criterion (AIC), the F statistic and its significance level, the variance explained (i.e., R²), and the statistical power of each model.

1st step. Initially, 9 separate models were created, each with a single cognitive predictor. The 9 models were compared against one another. The best model among them was then used to build the models in the next step.

2nd step. In this step, another 8 models were created, each with two cognitive predictors. In each model, the first predictor was the predictor from the best model in the previous step. The other predictor was one of the other cognitive function scores. These 8 models were then compared against each other. The best model was then compared to the best model from the previous step. If the best model from the current step was better than the best model from the previous step, then the predictors of the best model from the current step were used to build the models in the next step.

*Following steps.* The procedure described above was continued until the best model from the previous step was better than the best model from the current step. Then, the best model from the previous step was accepted as the best final model.

**Results**

The majority of participants reported no cybersickness symptoms (i.e., nausea, dizziness, disorientation, fatigue, or instability), while no one reported cybersickness symptoms stronger than a mild feeling, which postulates that cybersickness symptoms did not affect the participants' performance. The descriptive statistics for the VR-EAL scores are displayed in Table 2. The Shapiro-Wilk Normality test yielded non-significant results for both the event-based (p = .106) and time-based (p = .179) scores, postulating that the event- and time-based scores were normally distributed. Equally, the results of the Non-Constant Error Variance test for each model were not significant, indicating that each model's residuals were homogeneous. The variance inflation factors for the models' predictors varied from 1 – 1.6, suggesting an absence of multicollinearity.
<table><caption>Table 2. Descriptive statistics for the VR-EAL scores</caption><thead><tr><th></th><th>N</th><th>Mean (SD)</th><th>Range</th></tr></thead><tbody><tr><td>Event-based (max = 24)</td><td>41</td><td>18.15 (3.26)</td><td>8-24</td></tr><tr><td>Time-based (max = 18)</td><td>41</td><td>11.63 (3.10)</td><td>6-18</td></tr><tr><td>Immediate Recognition (max = 20)</td><td>41</td><td>15.51 (1.98)</td><td>10-18</td></tr><tr><td>Delayed Recognition (max = 20)</td><td>41</td><td>17.17 (2.42)</td><td>12-20</td></tr><tr><td>Visual Attention Accuracy (max = 32)</td><td>41</td><td>22.98 (3.84)</td><td>17-30</td></tr><tr><td>Visual Attention Speed (max = 32)</td><td>41</td><td>23.61 (3.69)</td><td>18-30</td></tr><tr><td>Visuospatial Attention Speed (max = 16)</td><td>41</td><td>11.90 (1.50)</td><td>8-14</td></tr><tr><td>Visuospatial Attention Accuracy (max = 16)</td><td>41</td><td>12.10 (1.18)</td><td>8-13</td></tr><tr><td>Auditory Attention (max = 32)</td><td>41</td><td>29.56 (3.66)</td><td>20-32</td></tr><tr><td>Planning (max = 19)</td><td>41</td><td>14.90 (1.51)</td><td>11-17</td></tr><tr><td>Cooking Task (max = 16)</td><td>41</td><td>9.68 (2.57)</td><td>2-13</td></tr></tbody></table>

*Event-based score - linear regression models*

The models with a single predictor are displayed in Table 3. The delayed recognition model best predicted event-based PM functioning. The model explained 17% of event-based PM functioning and had adequate statistical power. Visuospatial speed also significantly predicted everyday event-based PM functioning and the model explained 11% of the variance. However, the model had insufficient statistical power.

<table><caption>Table 3. Event-based prospective memory models with a single predictor</caption><thead><tr><th>Model</th><th>F (1,39)</th><th>t (39)</th><th>p (&gt;|t|)</th><th>β coefficient</th><th>AIC</th><th>R<sup>2</sup></th><th>SP</th></tr></thead><tbody><tr><td>Immediate Recognition</td><td>3.29</td><td>1.82</td><td>p = .078</td><td>0.46</td><td>215</td><td>.08</td><td>47%</td></tr><tr><td>Delayed Recognition</td><td>7.99</td><td>2.83</td><td>p = .007**</td><td>0.56</td><td>211</td><td>.17</td><td>82%</td></tr><tr><td>Visual Accuracy</td><td>2.14</td><td>1.46</td><td>p = .151</td><td>0.19</td><td>216</td><td>.05</td><td>29%</td></tr><tr><td>Visual Speed</td><td>2.24</td><td>1.50</td><td>p = .142</td><td>0.21</td><td>216</td><td>.05</td><td>29%</td></tr><tr><td>Visuospatial Accuracy</td><td>0.54</td><td>0.74</td><td>p = .466</td><td>1.78</td><td>218</td><td>.01</td><td>10%</td></tr><tr><td>Visuospatial Speed</td><td>4.63</td><td>2.15</td><td>p = .038*</td><td>0.71</td><td>214</td><td>.11</td><td>58%</td></tr><tr><td>Auditory Attention</td><td>3.00</td><td>1.73</td><td>p = .091</td><td>0.24</td><td>215</td><td>.07</td><td>42%</td></tr><tr><td>Planning</td><td>2.52</td><td>1.59</td><td>p = .120</td><td>0.53</td><td>216</td><td>.06</td><td>34%</td></tr><tr><td>Multitasking</td><td>1.72</td><td>1.31</td><td>p = .198</td><td>0.26</td><td>217</td><td>.04</td><td>24%</td></tr></tbody></table>

AIC = Akaike's information criterion; A smaller AIC indicates a better fit for the model. (R² x 100) indicates the percentage of variance explained by the model.

SP = statistical power. β coefficient = standardised regression coefficient.
After several steps, the best model for event-based PM functioning is visualised in Figure 3. Based on the F statistic and its p-value, the combining effect of delayed recognition, visuospatial speed, and planning ability on everyday event-based PM functioning was found to be significant. The model explained 25% of the variance, and it had substantial statistical power. Based on the standardised regression coefficients, delayed recognition, visuospatial speed, and planning ability predict everyday event-based PM functioning. However, only the effect of delayed recognition was found to be statistically significant.

<figure><img src="image_4.png" /><figcaption>Figure 3. Best model for event-based prospective memory</figcaption></figure>

*Time-based score - linear regression models*

The models with a single predictor are displayed in Table 4. The model with planning ability was found to best predict time-based PM functioning. The model explained 12% of the variance. However, its statistical power was insufficient. Moreover, the effect of visuospatial accuracy on time-based PM functioning was marginally insignificant. The model appeared to
explain only the 9% of the variance of time-based PM functioning, and its statistical power was also inadequate.

<table><caption>Table 4. Time-based prospective memory models with a single predictor</caption><thead><tr><th>Model</th><th>F (1,39)</th><th>t (39)</th><th>p (&gt;|t|)</th><th>β coefficient</th><th>AIC</th><th>R<sup>2</sup></th><th>SP</th></tr></thead><tbody><tr><td>Immediate Recognition</td><td>0.16</td><td>0.40</td><td>p = .690</td><td>0.10</td><td>214</td><td>.01</td><td>10%</td></tr><tr><td>Delayed Recognition</td><td>2.36</td><td>1.54</td><td>p = .132</td><td>0.31</td><td>212</td><td>.06</td><td>34%</td></tr><tr><td>Visual Accuracy</td><td>0.86</td><td>0.93</td><td>p = .361</td><td>0.12</td><td>213</td><td>.02</td><td>14%</td></tr><tr><td>Visual Speed</td><td>1.00</td><td>1.00</td><td>p = .325</td><td>0.13</td><td>213</td><td>.03</td><td>19%</td></tr><tr><td>Visuospatial Accuracy</td><td>3.72</td><td>1.93</td><td>p = .061</td><td>4.28</td><td>210</td><td>.09</td><td>51%</td></tr><tr><td>Visuospatial Speed</td><td>0.69</td><td>0.83</td><td>p = .410</td><td>0.27</td><td>213</td><td>.02</td><td>14%</td></tr><tr><td>Auditory Attention</td><td>2.80</td><td>1.67</td><td>p = .102</td><td>0.22</td><td>211</td><td>.07</td><td>42%</td></tr><tr><td>Planning</td><td>5.06</td><td>2.25</td><td>p = .030*</td><td>0.69</td><td>209</td><td>.12</td><td>65%</td></tr><tr><td>Multitasking</td><td>1.21</td><td>1.10</td><td>p = .279</td><td>0.21</td><td>213</td><td>.03</td><td>19%</td></tr></tbody></table>

AIC = Akaike's information criterion; A smaller AIC indicates a better fit for the model.

(R² x 100) indicates the percentage of variance explained by the model.

SP = statistical power. β coefficient = standardised regression coefficient.

The best final model for time-based PM functioning is illustrated in Figure 4. The F statistic and its p-value indicate that the combined effect of planning, visuospatial accuracy, delayed recognition, and multitasking/task-shifting abilities on everyday time-based PM functioning is significant. The model explained 32% of the variance and it had a substantially high statistical power. The standardised regression coefficients indicated that planning, visuospatial accuracy, delayed recognition, and multitasking/task-shifting predict everyday time-based PM functioning. However, only the effects of planning and visuospatial accuracy were found to be statistically significant.
<figure><img src="image_5.png" /><figcaption>Figure 4. Best model for time-based prospective memory</figcaption></figure>

Discussion

This study aimed to examine the cognitive functions implicated in everyday PM functioning using immersive VR. The findings show that delayed recognition and planning ability significantly predict event- and time-based PM functioning respectively. The final models included delayed recognition, visuospatial attention speed, and planning ability for event-based PM functioning and planning, visuospatial attention accuracy, delayed recognition, and multitasking/task-shifting ability for time-based PM functioning. These findings highlight the importance of attention, memory, and executive functioning in everyday PM functioning.

Event-based prospective memory

Delayed recognition explained some of the variance in event-based everyday PM functioning. This is in line with previous lab-based studies that have demonstrated that memory processes influence PM functioning (e.g., Smith, 2003; Smith et al., 2007; McDaniel & Einstein, 2000, 2007). The current findings indicate similarities with episodic memory, where better memory encoding and maintenance (i.e., in this case the PM intended intention) facilitates more
efficient retrieval (Frankland, Josselyn, & Köhler, 2019; Tulving, 1983; Tulving & Thomson, 1973; Xue, 2018). In PM, the association between an X PM cue and Y PM action appear to facilitate better encoding as well as maintenance of the intended PM action (Gonneaud et al., 2011; McDaniel & Scullin, 2010; Zuber et al., 2019), which highlights the role of delayed recognition in everyday event-based PM functioning. Indeed, Gonneaud and collaborators (2011) found that retrospective processes are significant in event-based PM functioning, but they are of secondary importance in time-based PM.

Furthermore, the inclusion of visuospatial attention speed (i.e., the ability to select and scan an area of interest to detect an item of interest) in our event-based PM model emphasizes the importance of contextual information in PM functioning. Again, this supports previous findings (e.g., Marsh, et al., 2008; Scullin et al., 2013; Shelton & Scullin, 2017). It is also in line with the dynamic multiprocess framework, where attentional processes are implemented for contextual information and/or the plan of action (Scullin et al., 2013; Shelton & Scullin, 2017). In our best model for event-based PM functioning, both visuospatial attention speed and planning ability were included. Visuospatial attention speed may indicate the ability to rapidly scan the surrounding environment to retrieve information, which may be valuable for informing and/or updating the formulated plan of action. In line with our findings, the findings of several studies have postulated the role of planning in event-based PM (e.g., Gonneaud et al., 2011; McDaniel & Scullin, 2010; Zuber, Kliegel, & Ihle, 2016; Zuber et al., 2019).

## Time-based prospective memory

Planning ability was found to explain a substantial amount of the variance in time-based PM functioning. This highlights the important role of planning ability in time-based PM functioning, which is in line with previous studies that have also demonstrated that planning influences time-based PM (e.g., Azzopardi et al., 2017; Liu & Park, 2004; McFarland & Glisky, 2009; Mioni & Stablum, 2014; Vanneste et al., 2016). Beyond formulating an effective plan of action, the inclusion of planning may also indicate the importance of strategic time monitoring, which avoids the cognitively demanding task of constantly time monitoring (Cona et al., 2012; Kliegel et al., 2001). In strategic time monitoring, when individuals check the time, they estimate when they should check it again based on a predefined plan of action, and adjust correspondingly the frequency of time monitoring (Cona et al., 2012; Kliegel et al., 2001).

Similarly, the involvement of visuospatial attention accuracy in time-based PM may be an indication of effective time monitoring. In VR-EAL, the watch is attached to the left hand/controller of the participant which is accessible at all stages of the scenario (see scene 10 in Figure 1). Most of the time, the watch is not within the participant's view (see Figures 1 and 2). Time monitoring requires the participant to detect and focus on the watch.

Consequently, visuospatial attention accuracy highlights the crucial role of time monitoring in time-based PM functioning (McFarland & Glisky, 2009; Mioni & Stablum, 2014; Vanneste et al., 2016). However, this finding may also indicate the importance of contextual information in time-based PM functioning. For example, the detection of a medication bottle may remind
the individual that s/he needs to take her/his medicine at a specific time. This contextual
information may assist the individual in implementing strategic time monitoring, as well as
maintaining the PM intention.

The delayed recognition was also included in the time-based PM model, which
supports previous findings, where episodic memory was implicated in time-based PM
functioning (e.g., Mackinlay, Kliegel, & Mäntylä, 2009; McFarland & Glisky, 2009).
However, the effect of delayed recognition was not significant, which indicates that its role in
our model appears to be supplementary, as was previously seen in Gonneaud and
collaborators (2011). Finally, the inclusion of task-shifting in the best model of time-based
PM appears to be in accordance with the findings (e.g., Azzopardi et al., 2017; Gonneaud et
al., 2011; McFarland & Glisky, 2009; Mioni & Stablum, 2014; Vanneste et al., 2016; Zuber et
al., 2019). However, in our study, task-shifting did not appear to have an important effect on
time-based PM functioning. Its inclusion in the best model may have a secondary role, and
indicate the ability to inhibit the task at hand, and shift attention towards time monitoring.

Ecological Validity

As previously suggested, the implementation of lab-based, non-ecological PM paradigms may
provide findings which are discrepant with the findings of ecologically valid research
paradigms (Marsh et al., 1998). This has been further supported by the so-called age-
prospective memory-paradox where significant age-related differences were found on
laboratory PM tasks, while they were not found with ecologically valid PM tasks
(Kvavilashvili et al., 2013; Niedźwieńska & Barzykowski, 2012; Phillips et al., 2008; Uttl,
2008). These discrepancies may be attributed to the use of compensatory strategies and
external memory aids, which are common strategies in daily life for performing PM tasks
(Phillips et al., 2008; Uttl, 2008). Our findings further support the necessity for adopting
ecologically valid paradigms to study everyday PM functioning.

In our current study, the implementation of a long-lasting scenario (i.e., approximately
70 minutes) demonstrated the importance of recognising a cue after a long delay (i.e., delayed
recognition), especially for event-based PM. Importantly, the VR-EAL attempts to simulate a
common day in the real-world. The inclusion of several everyday tasks (e.g., preparing
breakfast, shopping, finding the keys, transporting, interacting with others, and taking meds)
highlighted the importance of prioritising the activities (i.e., planning ability) in everyday PM
functioning, especially for time-based PM. Furthermore, the implementation of a complex,
dynamic, and immersive 360° environment appeared to highlight the importance of
visuospatial attention processes in everyday PM functioning, while the simple visual attention
processes did not appear crucial. Lastly, enabling participants to perform the VR-EAL tasks
through ergonomic and naturalistic ways seemed to diminish the importance of
multitasking/task-shifting ability in PM functioning, as has been previously seen in laboratory
paradigms. Our findings, hence, postulate the importance of ecological validity for examining
PM functioning and generalising the findings to daily life. Also, our findings suggest the
importance of cognitive abilities in everyday PM functioning, such as episodic memory,
visuospatial attention, and planning, which were either undermined (e.g., delayed recognition
and planning) or not even considered (e.g., visuospatial attention) in non-ecologically valid
laboratory paradigms, where the duration is short (e.g., 5-15 minutes), there is only a single or
small number of tasks, and the testing environment is simple and limited (e.g., 90 - 180° field
of view). The role and importance of these cognitive functions should be further studied in
future ecologically valid studies.

## Limitations and future studies

Our study has some limitations. The sample size was relatively small, although it
allowed us to perform sound statistical analyses with high statistical power. Future studies
should include a larger population which would allow a factorial analysis involving the
components of PM. Also, our participants were only young adults. A more diverse population
including both younger and older adults may help to elucidate the effect of aging on PM
assessed using the VR-EAL. VR-EAL also has the limitation that it cannot measure time
monitoring and this should be integrated into a future version of the VR-EAL. Furthermore,
while simulations of everyday life like the VR-EAL are suitable for the investigation of real-
life PM, they may be susceptible to confounding factors, such as environmental stimuli (e.g.,
a building with several signs) or an interaction with a 3D character (i.e., the requirement of
communicational skills), which were not experimentally controlled. Laboratory tasks have the
advantage of isolating confounding factors and allowing the examination of specific
processes. However, laboratory PM tasks in their current form suffer from limitations such as
a two-dimensional environment, keyboard-based responses, static stimuli, and a lack of
realism. A solution would be the modernization of laboratory tasks using immersive VR
technology, which would reduce the divergence from real-life conditions, and enable a
meticulous examination of PM and the role of other cognitive functions in everyday PM
functioning.

## Conclusions

The findings of this study demonstrated the importance of delayed recognition, planning, and
visuospatial attention in everyday PM. Delayed recognition and planning ability were found
to be especially crucial in event- and time-based PM respectively. In order of importance,
delayed recognition, visuospatial attention speed, and planning ability were found to be
involved in event-based PM functioning. Comparably, planning, visuospatial attention
accuracy, delayed recognition, and multitasking/task-shifting ability were found to be
involved in time-based PM functioning. These findings further suggest the importance of
ecological validity in the study of PM, which may be achieved using immersive VR
paradigms.

## Acknowledgments

The authors declare no conflicts of interest. The current study did not receive any financial
support or grants.

## References

Azzopardi, B., Auffray, C., & Kermarrec, C. (2017). Paradoxical effect of aging on laboratory and naturalistic time-based prospective memory tasks. Role of executive functions. *Canadian Journal on Aging/La Revue canadienne du vieillissement*, 36(1), 30–40. https://doi.org/10.1017/s0714980816000738

Bohil, C. J., Alicea, B., & Biocca, F. A. (2011). Virtual reality in neuroscience research and therapy. *Nature Reviews Neuroscience*, 12(12), 752-762. https://doi.org/10.1038/nrn3122
Borrego, A., Latorre, J., Alcañiz, M., & Llorens, R. (2018). Comparison of Oculus Rift and HTC Vive: feasibility for virtual reality-based exploration, navigation, exergaming, and rehabilitation. *Games for Health Journal*, 7(3), 151-156. https://doi.org/10.1089/g4h.2017.0114

Champely S. (2020). pwr: Basic Functions for Power Analysis. R package version 1.3-0. https://CRAN.R-project.org/package=pwr

Chaytor, N., & Schmitter-Edgecombe, M. (2003). The ecological validity of neuropsychological tests: A review of the literature on everyday cognitive skills. *Neuropsychology Review*, 13(4), 181-197.
https://doi.org/10.1023/B:NERV.0000009483.91468.fb

Cona, G., Arcara, G., Tarantino, V., & Bisiacchi, P. S. (2012). Electrophysiological correlates of strategic monitoring in event-based and time-based prospective memory. *PLoS One*, 7(2), e31659.
https://doi.org/10.1371/journal.pone.0031659

Costa, A., Caltagirone, C., & Carlesimo, G. A. (2011). Prospective memory impairment in mild cognitive impairment: An analytical review. *Neuropsychology Review*, 21(4), 390-404.
https://doi.org/10.1007/s11065-011-9172-z

Einstein, G. O., & McDaniel, M. A. (1996). Retrieval processes in prospective memory: Theoretical approaches and some new empirical findings. In M. Brandimonte, G. O. Einstein, & M. A. McDaniel (Eds.), *Prospective memory: Theory and applications* (pp. 115–141). Mahwah, NJ: Lawrence Erlbaum Associates. https://psycnet.apa.org/record/2002-02930-005

Einstein, G. O., Smith, R. E., McDaniel, M. A., & Shaw, P. (1997). Aging and prospective memory: The influence of increased task demands at encoding and retrieval. *Psychology and Aging*, 12(3), 479.
https://doi.org/10.1037/0882-7974.12.3.479

Einstein, G. O., McDaniel, M. A., Manzi, M., Cochran, B., & Baker, M. (2000). Prospective memory and aging: Forgetting intentions over short delays. *Psychology and Aging*, 15(4), 671-683.
https://doi.org/10.1037/0882-7974.15.4.671

Fox, J. and Weisberg, S. (2019). An {R} Companion to Applied Regression, Third Edition. Thousand Oaks CA: Sage. URL: https://socialsciences.mcmaster.ca/jfox/Books/Companion/

Frankland, P. W., Josselyn, S. A., & Köhler, S. (2019). The neurobiological foundation of memory retrieval. *Nature Neuroscience*, 22(10), 1576-1585. https://doi.org/10.1038/s41593-019-0493-1

Franzen, M. D., & Wilhelm, K. L. (1996). Conceptual foundations of ecological validity in neuropsychological assessment. In R. J. Sbordone & C. J. Long (Eds.), *Ecological validity of neuropsychological testing* (p. 91–112). Gr Press/St Lucie Press, Inc. https://psycnet.apa.org/record/1996-98718-005

Garden, S., Phillips, L. H. & MacPherson, S. E. (2001). Mid-life aging, open-ended planning, and laboratory measures of executive function. *Neuropsychology*, 15(4), 472-482. https://doi.org/10.1037/0894-4105.15.4.472

Glickson, J., & Myslobodsky, M. (2006). *Timing the future: The case of time-based prospective memory*. New York, NY: World Scientific Publishing. https://doi.org/10.1142/5937

Gollwitzer, P. M. (1999). Implementation intentions: strong effects of simple plans. *American Psychologist*, 54(7), 493. https://doi.org/10.1037/0003-066X.54.7.493

Gonneaud, J., Kalpouzos, G., Bon, L., Viader, F., Eustache, F., & Desgranges, B. (2011). Distinct and shared cognitive functions mediate event- and time-based prospective memory impairment in normal ageing. *Memory*, 19(4), 360–377. https://doi.org/10.1080/09658211.2011.570765

Haines, S., Shelton, J., Henry, J., Terrett, G., Vorwerk, T., & Rendell, P. (2019, February 25). Prospective Memory and Cognitive Aging. *Oxford Research Encyclopedia of Psychology*. Retrieved 7 Dec. 2019, from https://oxfordre.com/psychology/view/10.1093/acrefore/9780190236557.001.0001/acrefore-9780190236557-e-381

Higginson, C. I., Arnett, P. A., & Voss, W. D. (2000). The ecological validity of clinical tests of memory and attention in multiple sclerosis. *Archives of Clinical Neuropsychology*, 15(3), 185-204.
https://doi.org/10.1016/S0887-6177(99)00004-9

Huppert, F. A., Johnson, T., & Nickson, J. (2000). High prevalence of prospective memory impairment in the elderly and in early-stage dementia: Findings from a population-based study. *Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition*, 14(7), S63-S81. https://doi.org/10.1002/acp.771

Jones, S., Livner, Å., & Bäckman, L. (2006). Patterns of prospective and retrospective memory impairment in preclinical Alzheimer's disease. *Neuropsychology*, 20(2), 144. https://doi.org/10.1037/0894-4105.20.2.144

Kliegel, M., Altgassen, M., Hering, A., & Rose, N. S. (2011). A process-model based approach to prospective memory impairment in Parkinson's disease. *Neuropsychologia*, 49(8), 2166-2177.
https://doi.org/10.1016/j.neuropsychologia.2011.01.024
Kliegel, M., Martin, M., McDaniel, M. A., & Einstein, G. O. (2001). Varying the importance of a prospective memory task: Differential effects across time-and event-based prospective memory. *Memory*, 9(1), 1-11. https://doi.org/10.1080/09658210042000003

Kourtesis, P., Collina, S., Doumas, L. A. A., & MacPherson, S. E. (2019a). Technological competence is a precondition for effective implementation of virtual reality head mounted displays in human neuroscience: a technological review and meta-analysis. *Frontiers in Human Neuroscience*, 13, 342. https://doi.org/10.3389/fnhum.2019.00342

Kourtesis, P., Collina, S., Doumas, L. A. A., & MacPherson, S. E. (2019b). Validation of the Virtual Reality Neuroscience Questionnaire: maximum duration of immersive virtual reality sessions without the presence of pertinent adverse symptomatology. *Frontiers in Human Neuroscience*, 13, 417. doi: https://doi.org/10.3389/fnhum.2019.00417

Kourtesis, P., Collina, S., Doumas, L. A. A., & MacPherson, S. E. (2020a). Validation of the Virtual Reality Everyday Assessment Lab (VR-EAL): an immersive virtual reality neuropsychological battery with enhanced ecological validity. *Journal of the International Neuropsychological Society*, 1-16. https://doi.org/10.1017/S1355617720000764

Kourtesis, P., Korre, D., Collina, S., Doumas, L. A. A., & MacPherson, S. E. (2020b). Guidelines for the development of immersive Virtual Reality software for cognitive neuroscience and neuropsychology: the development of Virtual Reality Everyday Assessment Lab (VR-EAL), a neuropsychological test battery in immersive virtual reality. *Frontiers in Computer Science*, 1, 12. https://doi.org/10.3389/fcomp.2019.00012

Kvavilashvili, L., Cockburn, J., & Kornbrot, D. E. (2013). Prospective memory and ageing paradox with event-based tasks: A study of young, young-old, and old-old participants. *Quarterly Journal of Experimental Psychology*, 66(5), 864-875. https://doi.org/10.1080/17470218.2012.721379

Liu, L. L., & Park, D. C. (2004). Aging and medical adherence: The use of automatic processes to achieve effortful things. *Psychology and Aging*, 19, 318-325. https://doi.org/10.1037/0882-7974.19.2.318

Mackinlay, R. J., Kliegel, M., & Mäntylä, T. (2009). Predictors of time-based prospective memory in children. *Journal of Experimental Child Psychology*, 102(3), 251-264. https://doi.org/10.1016/j.jecp.2008.08.006

Marsh, R. L., Hicks, J. L., & Cook, G. I. (2008). *On beginning to understand the role of context in prospective memory*. In M. Kliegel, M. A. McDaniel, & G. O. Einstein (Eds.), *Prospective memory: Cognitive, neuroscience, developmental, and applied perspectives* (p. 77-100). Taylor & Francis Group/Lawrence Erlbaum Associates. https://psycnet.apa.org/record/2007-15727-004

Niedźwieńska, A., & Barzykowski, K. (2012). The age prospective memory paradox within the same sample in time-based and event-based tasks. *Aging, Neuropsychology, and Cognition*, 19(1-2), 58-83. https://doi.org/10.1080/13825585.2011.628374

Marsh, R. L., Hicks, J. L., & Landau, J. D. (1998). An investigation of everyday prospective memory. *Memory & Cognition*, 26(4), 633-643. https://doi.org/10.3758/BF03211383

McDaniel, M. A., & Scullin, M. K. (2010). Implementation intention encoding does not automatize prospective memory responding. *Memory & Cognition*, 38(2), 221-232. https://doi.org/10.3758/MC.38.2.221

McFarland, C. P., & Glisky, E. L. (2009). Frontal lobe involvement in a task of time-based prospective memory. *Neuropsychologia*, 47(7), 1660-1669. https://doi.org/10.1016/j.neuropsychologia.2009.02.023

Mioni, G., & Stablum, F. (2014). Monitoring behaviour in a time-based prospective memory task: The involvement of executive functions and time perception. *Memory*, 22(5), 536-552. https://doi.org/10.1080/09658211.2013.801987

Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. *Cognitive Psychology*, 41(1), 49-100. https://doi.org/10.1006/cogp.1999.0734

Mlinac, M. E., & Feng, M. C. (2016). Assessment of activities of daily living, self-care, and independence. *Archives of Clinical Neuropsychology*, 31(6), 506-516. https://doi.org/10.1093/arclin/acw049

Morris, R., & Ward, G. (Eds.). (2004). *The cognitive psychology of planning*. Psychology Press. https://doi.org/10.4324/9780203493564

Mullet, H. G., Scullin, M. K., Hess, T. J., Scullin, R. B., Arnold, K. M., & Einstein, G. O. (2013). Prospective memory and aging: Evidence for preserved spontaneous retrieval with exact but not related cues. *Psychology and Aging*, 28(4), 910. https://doi.org/10.1037/a0034347

Parsons, T. D. (2015). Virtual reality for enhanced ecological validity and experimental control in the clinical, affective and social neurosciences. *Frontiers in Human Neuroscience*, 9. https://doi.org/10.3389/fnhum.2015.00660

Parsons, T. D., McMahan, T., & Kane, R. (2018). Practice parameters facilitating adoption of advanced technologies for enhancing neuropsychological assessment paradigms. *The Clinical Neuropsychologist*, 32(1), 16-41. https://doi.org/10.1080/13854046.2017.1337932
Patil, I. (2018). ggstatsplot: 'ggplot2' Based Plots with Statistical Details. CRAN. Retrieved from https://cran.r-project.org/web/packages/ggstatsplot/index.html.

Phillips, L. H., Henry, J. D., & Martin, M. (2008). Adult aging and prospective memory: The importance of ecological validity. In M. Kliegel, M. A. McDaniel, & G. O. Einstein (Eds.), *Prospective memory: Cognitive, neuroscience, developmental, and applied perspectives* (p. 161–185). Taylor & Francis Group/Lawrence Erlbaum Associates. https://doi.org/10.4324/9780203809945

R Core Team (2020). R: A language and environment for statistical computing. *R Foundation for Statistical Computing*, Vienna, Austria. URL https://www.R-project.org/.

Rizzo, A. A., Schultheis, M., Kerns, K. A., & Mateer, C. (2004). Analysis of assets for virtual reality applications in neuropsychology. *Neuropsychological Rehabilitation*, 14(1-2), 207-239.
https://doi.org/10.1080/09602010343000183

Robertson, I. H., Ward, T., Ridgeway, V., & Nimmo-Smith, I. (1994). *The test of everyday attention (TEA)*. Bury St. Edmunds, UK: Thames Valley Test Company, 197-221.

RStudio Team (2020). RStudio: Integrated Development Environment for R. RStudio, PBC, Boston, MA URL http://www.rstudio.com/.

Scullin, M. K., McDaniel, M. A., Shelton, J. T., & Lee, J. H. (2010). Focal/nonfocal cue effects in prospective memory: Monitoring difficulty or different retrieval processes? *Journal of Experimental Psychology: Learning, Memory, and Cognition*, 36(3), 736. https://doi.org/10.1037/a0018971

Shallice, T., & Burgess, P. (1991). Deficits in strategy application following frontal lobe damage in man. *Brain*, 114, 727-741. https://doi.org/10.1093/brain/114.2.727

Sheppard, D. P., Bruineberg, J. P., Kretschmer-Trendowicz, A., & Altgassen, M. (2018). Prospective memory in autism: Theory and literature review. *The Clinical Neuropsychologist*, 32(5), 748-782.
https://doi.org/10.1080/13854046.2018.1435823

Spooner, D. M., & Pachana, N. A. (2006). Ecological validity in neuropsychological assessment: A case for greater consideration in research with neurologically intact populations. *Archives of Clinical Neuropsychology*, 21(4), 327-337. https://doi.org/10.1016/j.acn.2006.04.004

Teo, W. P., Muthalib, M., Yamin, S., Hendy, A. M., Bramstedt, K., Kotsopoulos, E., ... & Ayaz, H. (2016). Does a combination of virtual reality, neuromodulation and neuroimaging provide a comprehensive platform for neurorehabilitation?–A narrative review of the literature. *Frontiers in Human Neuroscience*, 10, 284. https://doi.org/10.3389/fnhum.2016.00284

Tulving, E., & Thomson, D. M. (1973). Encoding specificity and retrieval processes in episodic memory. *Psychological Review*, 80(5), 352. https://doi.org/10.1037/h0020071

Tulving, E. (1983). *Elements of Episodic Memory*. Oxford University Press, Oxford.
https://doi.org/10.1017/S0033291700015361

Uttl, B. (2008). Transparent meta-analysis of prospective memory and aging. *PloS one*, 3(2), e1568.
https://doi.org/10.1371/journal.pone.0001568

Vanneste, S., Baudouin, A., Bouazzaoui, B., & Taconnat, L. (2016). Age-related differences in time-based prospective memory: The role of time estimation in the clock monitoring strategy. *Memory*, 24(6), 812-825. https://doi.org/10.1080/09658211.2015.1054837

Wilson, B. A., Evans, J.J., Emslie, H., Foley, J., Shiel, A., Watson, P., Hawkins, K., & Groot, Y. (2005). *The Cambridge prospective memory test: CAMPROMPT*. London: Harcourt Assessment.

Wilson, B. A., Cockburn, J., & Baddeley, A. (2008). *The Rivermead behavioural memory test*. Bury St Edmunds, UK: Thames Valley Test Company.

Wilson, B. A., Alderman, N., Burgess, P. W., Emslie, H., & Evans, J.J. (1996). *Behavioural Assessment of the Dysexecutive Syndrome (BADS)*. Bury St. Edmunds, UK, Thames Valley Test Company.

Xue, G. (2018). The neural representations underlying human episodic memory. *Trends in Cognitive Sciences*, 22(6), 544-561. https://doi.org/10.1016/j.tics.2018.03.004

Zaidi, S. F. M., Duthie, C., Carr, E., & Maksoud, S. H. A. E. (2018, December). Conceptual framework for the usability evaluation of gamified virtual reality environment for non-gamers. In *Proceedings of the 16th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry* (p. 13). ACM. https://doi.org/10.1145/3284398.3284431

Zuber, S., Kliegel, M., & Ihle, A. (2016). An individual difference perspective on focal versus nonfocal prospective memory. *Memory & Cognition*, 44(8), 1192-1203. https://doi.org/10.3758/s13421-016-0628-5

Zuber, S., Mahy, C. E., & Kliegel, M. (2019). How executive functions are associated with event-based and time-based prospective memory during childhood. *Cognitive Development*, 50, 66-79.
https://doi.org/10.1016/j.cogdev.2019.03.001

Zygouris, S., & Tsolaki, M. (2015). Computerized cognitive testing for older adults: a review. *American Journal of Alzheimer's Disease & Other Dementias®,* 30(1), 13-28. https://doi.org/10.1177/1533317514522852