Feasibility and utility of automated processing of speech-based cognitive assessments for early Alzheimer's Disease
Keywords: audio signal processing
natural language processing
narrative speech
lexical and semantic analysis
story retelling analysis
speech-based cognitive assessments
Alzheimer's disease

Subtle changes in speech and language can be one of the earlier signs of Alzheimer's disease (AD) that is evident even decades before the appearance of observable symptoms leading to the AD diagnosis. Smartphone applications enable the frequent, extensive collection of raw speech data from a large population remotely and target specific behaviours that correlate with the disease. There is a need for automated and scalable processing of these large data to extract meaningful features for further analysis. Also, the newly developed measure's feasibility and utility must be addressed before it can be widely adopted.
A literature review of studies undertaking audio and language processing for cognitive assessment identified 131 studies and the range of techniques used. The study sizes were very small (75% of studies with < 100 participants; the largest sample size of 521), and the data were usually manually processed. Some of the evidence gaps identified were the lack of a standardised feature set with both audio and language features and the limited use of computational methods for scalability and automation.
Data available from the healthy PREVENT (n = 25) and GameChanger (n = 11239) study participants contained audio recordings of story narration (learning) from pictorial cues (18 stories in PREVENT and 7 stories in GameChanger Year-1 and an additional 5 stories in a subset of participants in Year-2) and their retellings from memory immediately and 24 hours later using the Story Time task available within the Mezurio smartphone app. In this thesis, a scalable and cohesive automated feature extraction pipeline was developed utilising automatic speech recognition (ASR) systems, natural language processing (NLP), audio signal processing (ASP), and artificial intelligence/machine learning (AI/ML) to extract audio and language features from speech recordings. A systematic comparison of transcription accuracy across six different ASR systems showed that Microsoft Azure transcription service provided transcripts with a lower word error rate (WER = 0.16) compared to other ASR systems, including KALDI (WER = 0.33), which has been mostly used in AD research. Therefore, the ASR system provided by Microsoft Azure was included in the pipeline to convert speech to text. The quality of the information provided by the participants was automatically scored as Automated Information Units (AIU) considering NLP methods, including semantic neighbourhood and part-of-speech tags. A protocol to provide a human-derived measure of information unit recall [Manual Information Unit (MIU) score] was developed and applied in PREVENT to compare the quality of AIU scores. A linear mixed model (adjusted for age, sex, and education, with participants and stories as random effects) was used to estimate the change in about 50 pipeline speech-based features cross-sectionally. Associations between Story Time pipeline features and other cognitive measures (in-clinic test, Gallery Game, and Subjective Cognitive Decline scores) available within PREVENT and GameChanger were evaluated. The association of pipeline features with well-established socio-demographic variables associated with dementia risk, including age, gender, education, family history of dementia, and deprivation, were examined. In addition, stepwise regression was used to identify the informative speech-based features associated with measures of cognition.
In the pilot of 25 participants in PREVENT, the correlation between AIU and MIU scores was very high (r = 0.98). Of about 18000 participants in GameChanger, 11239 attempted the Story Time task. Older people (60-95) in the study were more committed to the task, and on average, they fully completed more stories (4.6 stories with a full set of the subtasks learning, immediate recall, and delayed recall) than the younger group (16-39: 3.2 stories; 40-59: 3.6 stories) out of 7 stories in Year-1. The test-retest reliability (Cronbach alpha) of learning, immediate, delayed, and average AIU recall scores were between 0.70-0.79, showing moderate to good reliability. Among those with a full set of sub-tasks, the AIU score was significantly lower at 24-hr delayed recall (Mean ± SD = 0.42 ± 0.14) than immediate recall (Mean ± SD = 0.50 ± 0.12) (p < 10- 300). The AIU recall score was one of the strong pipeline predictors of the Gallery Game recall score and subjective cognitive decline measure. In the sensitivity analysis, when only the audio features were considered that would be directly applicable to studies in other languages, the model explained only 4% of the variation of Gallery Game recall accuracy scores compared to 7%, using all the pipeline features, highlighting the importance of language features, but still obtaining half of the value by using audio features alone. Association of AIU recall scores with age and education were in the expected direction. Several pipeline features, including lexical, syntactic, speech fluency, and voice quality, were also statistically significantly associated with age, education, and deprivation.
In this thesis, a novel automated feature extraction pipeline was developed using various emerging technologies to extract speech-based features of potential relevance to dementia using speech data collected via a smartphone application. This pipeline has the potential to facilitate much larger studies and trials through its scalability and wide deployment. AIU recall score showed expected associations with other cognitive scores and demographics, demonstrating its utility as a clinical assessment method for examining the performance of participants in a story-retelling task. However, the association of the score with stages of dementia could not be assessed. The Mezurio app is currently being used in other European studies (some of which include the potential for comparison with dementia status), and the participants provide speech responses in their native language. ASR in non-English languages is improving. Therefore, this pipeline could be readily adapted to produce these potentially useful language features in those studies allowing further validation of the methods, but the audio feature is directly applicable.