Detecting Dementia from Long Neuropsychological Interviews (2022)

Abstract: Neuropsychological exams are commonly used to diagnose various kinds of cognitive impairment. They typically involve a trained examiner who conducts a series of cognitive tests with a subject. In recent years, there has been growing interest in developing machine learning methods to extract speech and language biomarkers from exam recordings to provide automated input for cognitive assessment. Inspired by recent findings suggesting that the examiner’s language can influence cognitive impairment classifications, in this paper, we study the influence of the examiner on automatic dementia identification decisions in real-world neuropsychological exams. To mitigate the influence of the examiner, we propose a systematic three-stage pipeline for detecting dementia from exam recordings. In the first stage, we perform audio-based speaker diarization (i.e., estimating who spoke when?) by incorporating speaker discriminative features. In the second stage, we employ text-based language models to identify the role of the speaker (i.e., examiner or subject). Finally, in the third stage, we employ text- and audio-based models to detect cognitive impairment from hypothesized subject segments. Our studies suggest that incorporating audio-based diarization followed by text-based role identification helps mitigate the influences from the examiner’s segments. Further, we found that the text and audio modalities complement each other, and the performance improves when we use both modalities. We also perform several carefully designed experimental studies to assess the performance of each stage.

## introduction cognitive impairment is the condition where a human may experience cognitive decline in mental ability (e.g., dementia and alzheimer's disease). alzheimer's disease is the most common form of dementia. early detection of cognitive impairment may lead to a better lifestyle and pathways to treatment @xcite . neuropsychological exams are a widely used and effective technique in assessing a subject's cognitive status @xcite . these exams often serve as the first-stage screening tool to identify cognitive impairment before performing more expensive laboratory tests (e.g., brain imaging) @xcite with an average estimated cost of $5,000 per brain scan. an accurate first-stage assessment is crucial as it helps avoiding additional cost and time burdens on healthcare system and the patient. for a typical cognitive interview evaluation, a trained examiner interacts with a subject (patient) by conducting a series of tasks that are designed to assess the memory, attention, visuoperceptual, reasoning, language, and verbal skills of the subject @xcite . during this evaluation session, the examiner asks a series of questions related to different cognitive domains. these questions include identifying objects in an image, repeating the numbers, words, or story narrated by the examiner. based on the answers to the questions posed by the examiner, the interview exam is assigned a set scores that reflect the subject's cognitive status. this is a long process, and each interview typically lasts for more than an hour, with interview recordings typically containing approximately equal amounts of speech from the examiner and subject. ## methodology figure 1 shows the proposed three stages for detection of dementia from the long neuropsychological interview examinations, including (i) diarization using audio, (ii) role id using text, and (iii) cognitive id using both text and audio. the diarization stage takes input long interview recording and divides it into smaller chunks and processes using the ecapa-tdnn model to extract speaker discriminative embeddings. these embeddings are clustered using spectral clustering to obtain two clusters. both clusters are fed to the role id module to identify the cluster belonging to the subject. the hypothesized subject cluster is then used to identify dementia using text and audio. the scores from text and audio are fused to make the final cognitive id decision. ## conclusions in this study, we found that the examiner's segment influence the cognitive id performance in long exams. to mitigate the examiner's influence, we propose a three-stage cognitive identification system for long neuropsychological interviews. our system achieves state-of-the-art performance on the fhs dataset. more importantly, the following are the main conclusions from our study: finally, our system automatically curates the conversation dataset at all stages, i.e., diarization (who spoke when?), asr (speech to text), role id (speaker identity), and cognitive id (cognitive condition of a subject) which is useful for research.