Interference effects from machine learning matched confusable object pairs in memory assessment tasks

For early detection of neurodegeneration, objective cognitive tests provide a capable alternative to costly and invasive traditional biomarkers, as well as to short and infrequent in-clinic cognitive assessments. These tests, that are designed to target the cognitive processes first vulnerable to change in AD, can be remotely deployed in individuals’ lives over long durations for frequent and repeated measurements using widely accessible digital devices such as smartphones. A class of such objective tests loads on the susceptibility of the short-term memory to interference from perceptually similar objects to obtain the first signal of AD-specific neurodegeneration. Evidence suggests that increasing the similarity of the confusable object pairs used in such tasks could have differential impact on the ability of individuals at different stages of cognitive health to inhibit interference from such pairs, thereby making object pair similarity a useful metric in sensitively recording individual cognitive differences. While this claim has been tested across various stimulus pair types, due to the lack of standardised datasets of confusable object pairs parametrically varying in similarity where the constituent objects in the pairs differ conceptually but still share some degree of similarity (e.g., guitar and banjo), such an assessment remains unexplored for this stimulus type. In this thesis, this gap is addressed by first using machine learning based models of vision and semantics, as well as their combinations, to produce the required datasets of object pairs parametrically varying in similarity, followed by exploring the differential effects of such confusable object pairs on interference in short-term memory among participants from varied demographics in an objective test of cognition.
First, an exploratory review of literature was conducted to study the methodologies that exist for producing datasets of confusable image pairs parametrically varying in similarity across stimulus types. The reviewed evidence suggested that while methods to produce such datasets exist for stimulus types such as face stimuli, abstract stimuli, and natural scene stimuli, such methods were lacking for generating confusable object pairs central to this thesis. In fact, a huge reliance on human input to generate such pairs was found. During this review, the relationship between stimulus pair similarity and size of interference was also noted. Studies showed a direct relationship between stimulus pair similarity and the magnitude of interference they produced. Last but not least, the interaction of interference was also examined by age. Studies unanimously agreed that older population is more susceptible to interference from confusable stimuli than younger population.
To address the gaps identified by the review, I used computer vision-based, linguistic association-based and taxonomic information-based computational models, as well as their combinations, to generate similar object pairs parametrically varying in similarity. To measure the ability of such computational models at estimating the human understanding of object pair similarity, the employed models as well as the datasets of object pairs they produced were rated against existing human-rated databases of object pair similarities. The object pairs produced by the best performing models from this analysis were then rated by 554 humans using a similarity rating experiment hosted on the Amazon mechanical Turk platform. Finally, these validated object pairs were deployed in Mezurio Gallery Blast, an objective cognitive test of short-term memory, to assess the utility of such pairs in the assessment of cognition.
The findings from the analysis of the performance of 1,455 cognitively normal participants who attempted the task as part of a remote study revealed that 1) the computationally matched object pairs employed in the task were capable of stimulating interference effects across participants in both the proactive and retroactive conditions of interference, 2) the object pair similarity has a significant and direct effect on susceptibility to retroactive interference but not to proactive interference, and 3) increasing object pair similarity in the retroactive interference condition has a larger effect on the older age group’s susceptibility to interference compared to that of the younger age group. These results establish the utility of the computationally matched object pairs and the associated object pair similarity metric, in appropriately designed smartphone-based cognitive tasks. Furthermore, they provide preliminary evidence that such object pairing may facilitate detection of AD at an earlier stage of progression, and thus motivate future work.

Keywords: statistics
early detection
tetroactive interference
object similarity
human-in-the-loop
computer vision
neurodegeneration
machine learning in healthcare
digital biomarkers
Alzheimer's disease
objective cognitive tests
mixed-effects models
interference effects
deep learning
preclinical dementia
proactive interference
Advisor: Hinds, C