Validation of a remote and fully automated story recall task to assess for early cognitive impairment in older adults: a longitudinal case-control observational study
Authors: Skirrow Caroline , Meszaros Marton , Meepegama Udeepa , Lenain Raphael , Papp Kathryn V. , Weston Jack , Fristed Emil 
Published: October 2021

BACKGROUND

Story recall is a simple and sensitive cognitive test commonly used to measure changes in episodic memory function in early Alzheimer’s disease (AD). Recent advances in digital technology and natural language processing methods make this test a candidate for automated administration and scoring. Convenient and low-burden daily assessments may provide more reliable data than one-off lengthy assessments and be suitable for longer-term disease monitoring.

OBJECTIVES

Develop and validate a remote and fully automated story recall task, suitable for longitudinal assessment, in a population of older adults with and without mild cognitive impairment (MCI) or mild AD.

METHODS

Participants from AMYPRED-UK (NCT04828122) and AMYPRED-US (NCT04928976) studies were asked to complete optional daily self-administered assessments remotely on their smart devices over 7-8 days. Assessments included immediate and delayed recall of three stories from the Automatic Story Recall Task (ASRT), a test with multiple parallel stimuli (18 short stories, and 18 long stories) balanced for key linguistic and discourse metrics. Verbal responses were recorded and securely transferred from participants’ personal devices, and automatically transcribed and scored using text similarity metrics between the source text and retelling to derive a generalised matching score (G-match). Adherence and task performance differences were examined with logistic mixed models and linear mixed models, respectively. Correlational analysis examined parallel forms reliability of ASRTs, and convergent validity with established cognitive tests (Logical Memory Test, and Preclinical Alzheimer’s Cognitive Composite with semantic processing (PACC5)). Acceptability and usability data were obtained via remotely administered questionnaire.

RESULTS

Out of 200 participants recruited into the AMYPRED studies, a total of 151 participants (75.5%, 78 cognitively unimpaired (CU), 73 MCI/mild AD) engaged in optional remote assessments. In these participants, adherence to daily assessment was moderate, did not decline over time, but was higher in cognitively unimpaired participants (66% MCI/mild AD and 78% CU participants completed at least one ASRT story per day). Participants reported favourable task usability: few technical problems, that the application was easy to use, and the tasks were broadly interesting. Task performance improved modestly across the week and was better for immediate recall. G-match scores were lower in participants with MCI/mild AD. Parallel forms reliability of ASRTs were moderate to strong for immediate recall (mean rho=0.73), and delayed recall (mean rho=0.73). ASRTs showed moderate convergent validity with established cognitive tests.

CONCLUSIONS

The unsupervised, self-administered ASRT task is sensitive to cognitive impairments in MCI/mild AD. The task shows good usability, high parallel forms reliability and convergent validity with established cognitive tests. Remote, low cost, low burden and automatically scored speech assessments could be used to support diagnostic screening, healthcare and treatment monitoring.